* [PATCH 1/8] iommu/amd: Refactor AMD IOMMU debugfs initial setup
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 2/8] iommu/amd: Add debugfs support to dump IOMMU MMIO registers Dheeraj Kumar Srivastava
` (6 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
Rearrange initial setup of AMD IOMMU debugfs to segregate per IOMMU
setup and setup which is common for all IOMMUs. This ensures that common
debugfs paths (introduced in subsequent patches) are created only once
instead of being created for each IOMMU.
With the change, there is no need to use lock as amd_iommu_debugfs_setup()
will be called only once during AMD IOMMU initialization. So remove lock
acquisition in amd_iommu_debugfs_setup().
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
---
drivers/iommu/amd/amd_iommu.h | 4 ++--
drivers/iommu/amd/debugfs.c | 16 +++++++---------
drivers/iommu/amd/init.c | 5 ++---
3 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/amd/amd_iommu.h b/drivers/iommu/amd/amd_iommu.h
index 6386fa4556d9..68821b62730c 100644
--- a/drivers/iommu/amd/amd_iommu.h
+++ b/drivers/iommu/amd/amd_iommu.h
@@ -29,9 +29,9 @@ void *__init iommu_alloc_4k_pages(struct amd_iommu *iommu,
gfp_t gfp, size_t size);
#ifdef CONFIG_AMD_IOMMU_DEBUGFS
-void amd_iommu_debugfs_setup(struct amd_iommu *iommu);
+void amd_iommu_debugfs_setup(void);
#else
-static inline void amd_iommu_debugfs_setup(struct amd_iommu *iommu) {}
+static inline void amd_iommu_debugfs_setup(void) {}
#endif
/* Needed for interrupt remapping */
diff --git a/drivers/iommu/amd/debugfs.c b/drivers/iommu/amd/debugfs.c
index 545372fcc72f..ff9520e002be 100644
--- a/drivers/iommu/amd/debugfs.c
+++ b/drivers/iommu/amd/debugfs.c
@@ -13,20 +13,18 @@
#include "amd_iommu.h"
static struct dentry *amd_iommu_debugfs;
-static DEFINE_MUTEX(amd_iommu_debugfs_lock);
#define MAX_NAME_LEN 20
-void amd_iommu_debugfs_setup(struct amd_iommu *iommu)
+void amd_iommu_debugfs_setup(void)
{
+ struct amd_iommu *iommu;
char name[MAX_NAME_LEN + 1];
- mutex_lock(&amd_iommu_debugfs_lock);
- if (!amd_iommu_debugfs)
- amd_iommu_debugfs = debugfs_create_dir("amd",
- iommu_debugfs_dir);
- mutex_unlock(&amd_iommu_debugfs_lock);
+ amd_iommu_debugfs = debugfs_create_dir("amd", iommu_debugfs_dir);
- snprintf(name, MAX_NAME_LEN, "iommu%02d", iommu->index);
- iommu->debugfs = debugfs_create_dir(name, amd_iommu_debugfs);
+ for_each_iommu(iommu) {
+ snprintf(name, MAX_NAME_LEN, "iommu%02d", iommu->index);
+ iommu->debugfs = debugfs_create_dir(name, amd_iommu_debugfs);
+ }
}
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 43131c3a2172..d78dc96bbec3 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -3377,7 +3377,6 @@ int amd_iommu_enable_faulting(unsigned int cpu)
*/
static int __init amd_iommu_init(void)
{
- struct amd_iommu *iommu;
int ret;
ret = iommu_go_to_state(IOMMU_INITIALIZED);
@@ -3391,8 +3390,8 @@ static int __init amd_iommu_init(void)
}
#endif
- for_each_iommu(iommu)
- amd_iommu_debugfs_setup(iommu);
+ if (!ret)
+ amd_iommu_debugfs_setup();
return ret;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 2/8] iommu/amd: Add debugfs support to dump IOMMU MMIO registers
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 1/8] iommu/amd: Refactor AMD IOMMU debugfs initial setup Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 3/8] iommu/amd: Add debugfs support to dump IOMMU Capability registers Dheeraj Kumar Srivastava
` (5 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
Analyzing IOMMU MMIO registers gives a view of what IOMMU is
configured with on the system and is helpful to debug issues
with IOMMU.
eg.
1. To get mmio registers value for iommu<x>
# echo "0x18" > /sys/kernel/debug/iommu/amd/iommu00/mmio
# cat /sys/kernel/debug/iommu/amd/iommu00/mmio_dump
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
drivers/iommu/amd/debugfs.c | 75 +++++++++++++++++++++++++++++++++++++
1 file changed, 75 insertions(+)
diff --git a/drivers/iommu/amd/debugfs.c b/drivers/iommu/amd/debugfs.c
index ff9520e002be..30d6819e9a35 100644
--- a/drivers/iommu/amd/debugfs.c
+++ b/drivers/iommu/amd/debugfs.c
@@ -15,6 +15,76 @@
static struct dentry *amd_iommu_debugfs;
#define MAX_NAME_LEN 20
+#define OFS_IN_SZ 8
+
+static int mmio_offset = -1;
+
+static ssize_t iommu_mmio_write(struct file *filp, const char __user *ubuf,
+ size_t cnt, loff_t *ppos)
+{
+ struct seq_file *m = filp->private_data;
+ struct amd_iommu *iommu = m->private;
+ char *mmio_ptr_ofs;
+ int ret = cnt;
+
+ if (cnt > OFS_IN_SZ) {
+ ret = -EINVAL;
+ goto err;
+ }
+
+ mmio_ptr_ofs = memdup_user_nul(ubuf, cnt);
+ if (IS_ERR(mmio_ptr_ofs)) {
+ ret = PTR_ERR(mmio_ptr_ofs);
+ goto err;
+ }
+
+ if (kstrtou32(mmio_ptr_ofs, 0, &mmio_offset) < 0) {
+ ret = -EINVAL;
+ goto free;
+ }
+
+ if (mmio_offset > iommu->mmio_phys_end - 4) {
+ ret = -EINVAL;
+ goto free;
+ }
+
+ kfree(mmio_ptr_ofs);
+ *ppos += cnt;
+ return ret;
+
+free:
+ kfree(mmio_ptr_ofs);
+err:
+ mmio_offset = -1;
+ return ret;
+}
+
+static int iommu_mmio_show(struct seq_file *m, void *unused)
+{
+ if (mmio_offset >= 0)
+ seq_printf(m, "0x%x\n", mmio_offset);
+ else
+ seq_puts(m, "No or invalid input provided\n");
+ return 0;
+}
+DEFINE_SHOW_STORE_ATTRIBUTE(iommu_mmio);
+
+static int iommu_mmio_dump_show(struct seq_file *m, void *unused)
+{
+ struct amd_iommu *iommu = m->private;
+ u32 value;
+
+ if (mmio_offset < 0) {
+ seq_puts(m, "Please provide mmio register's offset\n");
+ return 0;
+ }
+
+ value = readl(iommu->mmio_base + mmio_offset);
+ seq_printf(m, "0x%08x\n", value);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(iommu_mmio_dump);
void amd_iommu_debugfs_setup(void)
{
@@ -26,5 +96,10 @@ void amd_iommu_debugfs_setup(void)
for_each_iommu(iommu) {
snprintf(name, MAX_NAME_LEN, "iommu%02d", iommu->index);
iommu->debugfs = debugfs_create_dir(name, amd_iommu_debugfs);
+
+ debugfs_create_file("mmio", 0644, iommu->debugfs, iommu,
+ &iommu_mmio_fops);
+ debugfs_create_file("mmio_dump", 0444, iommu->debugfs, iommu,
+ &iommu_mmio_dump_fops);
}
}
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 3/8] iommu/amd: Add debugfs support to dump IOMMU Capability registers
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 1/8] iommu/amd: Refactor AMD IOMMU debugfs initial setup Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 2/8] iommu/amd: Add debugfs support to dump IOMMU MMIO registers Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 4/8] iommu/amd: Add debugfs support to dump IOMMU command buffer Dheeraj Kumar Srivastava
` (4 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
IOMMU Capability registers defines capabilities of IOMMU and information
needed for initialising MMIO registers and device table. This is useful
to dump these registers for debugging IOMMU related issues.
e.g.To get capability registers value for iommu<x>
# echo "0x10" > /sys/kernel/debug/iommu/amd/iommu00/capability
# cat /sys/kernel/debug/iommu/amd/iommu00/capability_dump
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
drivers/iommu/amd/debugfs.c | 77 +++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
diff --git a/drivers/iommu/amd/debugfs.c b/drivers/iommu/amd/debugfs.c
index 30d6819e9a35..c3ef15e7b918 100644
--- a/drivers/iommu/amd/debugfs.c
+++ b/drivers/iommu/amd/debugfs.c
@@ -18,6 +18,7 @@ static struct dentry *amd_iommu_debugfs;
#define OFS_IN_SZ 8
static int mmio_offset = -1;
+static int cap_offset = -1;
static ssize_t iommu_mmio_write(struct file *filp, const char __user *ubuf,
size_t cnt, loff_t *ppos)
@@ -86,6 +87,78 @@ static int iommu_mmio_dump_show(struct seq_file *m, void *unused)
}
DEFINE_SHOW_ATTRIBUTE(iommu_mmio_dump);
+static ssize_t iommu_capability_write(struct file *filp, const char __user *ubuf,
+ size_t cnt, loff_t *ppos)
+{
+ char *cap_ptr_ofs;
+ int ret = cnt;
+
+ if (cnt > OFS_IN_SZ) {
+ ret = -EINVAL;
+ goto err;
+ }
+
+ cap_ptr_ofs = memdup_user_nul(ubuf, cnt);
+ if (IS_ERR(cap_ptr_ofs)) {
+ ret = PTR_ERR(cap_ptr_ofs);
+ goto err;
+ }
+
+ if (kstrtou32(cap_ptr_ofs, 0, &cap_offset) < 0) {
+ ret = -EINVAL;
+ goto free;
+ }
+
+ /* Capability register at offset 0x14 is the last IOMMU capability register. */
+ if (cap_offset > 0x14) {
+ ret = -EINVAL;
+ goto free;
+ }
+
+ kfree(cap_ptr_ofs);
+ *ppos += cnt;
+ return ret;
+
+free:
+ kfree(cap_ptr_ofs);
+err:
+ cap_offset = -1;
+ return ret;
+}
+
+static int iommu_capability_show(struct seq_file *m, void *unused)
+{
+ if (cap_offset >= 0)
+ seq_printf(m, "0x%x\n", cap_offset);
+ else
+ seq_puts(m, "No or invalid input provided\n");
+ return 0;
+}
+DEFINE_SHOW_STORE_ATTRIBUTE(iommu_capability);
+
+static int iommu_capability_dump_show(struct seq_file *m, void *unused)
+{
+ struct amd_iommu *iommu = m->private;
+ u32 value;
+ int err;
+
+ if (cap_offset < 0) {
+ seq_puts(m, "Please provide capability register's offset\n");
+ return 0;
+ }
+
+ err = pci_read_config_dword(iommu->dev, iommu->cap_ptr + cap_offset, &value);
+ if (err) {
+ seq_printf(m, "Not able to read capability register at 0x%x\n", cap_offset);
+ return 0;
+ }
+
+ seq_printf(m, "0x%08x\n", value);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(iommu_capability_dump);
+
void amd_iommu_debugfs_setup(void)
{
struct amd_iommu *iommu;
@@ -101,5 +174,9 @@ void amd_iommu_debugfs_setup(void)
&iommu_mmio_fops);
debugfs_create_file("mmio_dump", 0444, iommu->debugfs, iommu,
&iommu_mmio_dump_fops);
+ debugfs_create_file("capability", 0644, iommu->debugfs, iommu,
+ &iommu_capability_fops);
+ debugfs_create_file("capability_dump", 0444, iommu->debugfs,
+ iommu, &iommu_capability_dump_fops);
}
}
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 4/8] iommu/amd: Add debugfs support to dump IOMMU command buffer
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
` (2 preceding siblings ...)
2024-10-08 10:40 ` [PATCH 3/8] iommu/amd: Add debugfs support to dump IOMMU Capability registers Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 5/8] iommu/amd: Add support for device id user input Dheeraj Kumar Srivastava
` (3 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
IOMMU driver sends command to IOMMU hardware via command buffer. In cases
where IOMMU hardware fails to process commands in command buffer, dumping
it is a valuable input to debug the issue.
IOMMU hardware processes command buffer entry at offset equals to the head
pointer. Dumping just the entry at the head pointer may not always be
useful. The current head may not be pointing to the entry of the command
buffer which is causing the issue. IOMMU Hardware may have processed the
entry and updated the head pointer. So dumping the entire command buffer
gives a broad understanding of what hardware was/is doing. The command
buffer dump will have all entries from start to end of the command buffer.
Along with that, it will have a head and tail command buffer pointer
register dump to facilitate where the IOMMU driver and hardware are in
the command buffer for injecting and processing the entries respectively.
Command buffer is a per IOMMU data structure. So dumping on per IOMMU
basis.
eg. To get command buffer dump for iommu<x>
#cat /sys/kernel/debug/iommu/amd/iommu<x>/cmdbuf
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
drivers/iommu/amd/amd_iommu_types.h | 7 +++++++
drivers/iommu/amd/debugfs.c | 27 +++++++++++++++++++++++++++
drivers/iommu/amd/iommu.c | 7 -------
3 files changed, 34 insertions(+), 7 deletions(-)
diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h
index 601fb4ee6900..876fa671ef91 100644
--- a/drivers/iommu/amd/amd_iommu_types.h
+++ b/drivers/iommu/amd/amd_iommu_types.h
@@ -878,6 +878,13 @@ extern struct list_head amd_iommu_list;
*/
extern struct amd_iommu *amd_iommus[MAX_IOMMUS];
+/*
+ * Structure defining one entry in the command buffer
+ */
+struct iommu_cmd {
+ u32 data[4];
+};
+
/*
* Structure defining one entry in the device table
*/
diff --git a/drivers/iommu/amd/debugfs.c b/drivers/iommu/amd/debugfs.c
index c3ef15e7b918..69d2b01bc0e4 100644
--- a/drivers/iommu/amd/debugfs.c
+++ b/drivers/iommu/amd/debugfs.c
@@ -159,6 +159,31 @@ static int iommu_capability_dump_show(struct seq_file *m, void *unused)
}
DEFINE_SHOW_ATTRIBUTE(iommu_capability_dump);
+static int iommu_cmdbuf_show(struct seq_file *m, void *unused)
+{
+ struct amd_iommu *iommu = m->private;
+ struct iommu_cmd *cmd;
+ unsigned long flag;
+ u32 head, tail;
+ int i;
+
+ raw_spin_lock_irqsave(&iommu->lock, flag);
+
+ head = readl(iommu->mmio_base + MMIO_CMD_HEAD_OFFSET);
+ tail = readl(iommu->mmio_base + MMIO_CMD_TAIL_OFFSET);
+ seq_printf(m, "CMD Buffer Head pointer register:0x%08x Tail pointer register:0x%08x\n",
+ head, tail);
+ for (i = 0; i < CMD_BUFFER_ENTRIES; i++) {
+ cmd = (struct iommu_cmd *)(iommu->cmd_buf + i * sizeof(*cmd));
+ seq_printf(m, "%3d: %08x%08x%08x%08x\n", i, cmd->data[0],
+ cmd->data[1], cmd->data[2], cmd->data[3]);
+ }
+ raw_spin_unlock_irqrestore(&iommu->lock, flag);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(iommu_cmdbuf);
+
void amd_iommu_debugfs_setup(void)
{
struct amd_iommu *iommu;
@@ -178,5 +203,7 @@ void amd_iommu_debugfs_setup(void)
&iommu_capability_fops);
debugfs_create_file("capability_dump", 0444, iommu->debugfs,
iommu, &iommu_capability_dump_fops);
+ debugfs_create_file("cmdbuf", 0444, iommu->debugfs, iommu,
+ &iommu_cmdbuf_fops);
}
}
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 8364cd6fa47d..efc2d1ddd7cc 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -63,13 +63,6 @@ static const struct iommu_dirty_ops amd_dirty_ops;
int amd_iommu_max_glx_val = -1;
-/*
- * general struct to manage commands send to an IOMMU
- */
-struct iommu_cmd {
- u32 data[4];
-};
-
struct kmem_cache *amd_iommu_irq_cache;
static void detach_device(struct device *dev);
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 5/8] iommu/amd: Add support for device id user input
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
` (3 preceding siblings ...)
2024-10-08 10:40 ` [PATCH 4/8] iommu/amd: Add debugfs support to dump IOMMU command buffer Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 6/8] iommu/amd: Add debugfs support to dump device table Dheeraj Kumar Srivastava
` (2 subsequent siblings)
7 siblings, 0 replies; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
Device id user input is needed for dumping IOMMU data structures like
device table, IRT etc in AMD IOMMU debugfs.
eg.
1. # echo 0000:01:00.0 > /sys/kernel/debug/iommu/amd/devid
# cat /sys/kernel/debug/iommu/amd/devid
Output : 0000:01:00.0
2. # echo 01:00.0 > /sys/kernel/debug/iommu/amd/devid
# cat /sys/kernel/debug/iommu/amd/devid
Output : 0000:01:00.0
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
drivers/iommu/amd/debugfs.c | 86 +++++++++++++++++++++++++++++++++++++
1 file changed, 86 insertions(+)
diff --git a/drivers/iommu/amd/debugfs.c b/drivers/iommu/amd/debugfs.c
index 69d2b01bc0e4..0d11250c1a00 100644
--- a/drivers/iommu/amd/debugfs.c
+++ b/drivers/iommu/amd/debugfs.c
@@ -16,9 +16,11 @@ static struct dentry *amd_iommu_debugfs;
#define MAX_NAME_LEN 20
#define OFS_IN_SZ 8
+#define DEVID_IN_SZ 16
static int mmio_offset = -1;
static int cap_offset = -1;
+static int sbdf = -1;
static ssize_t iommu_mmio_write(struct file *filp, const char __user *ubuf,
size_t cnt, loff_t *ppos)
@@ -184,6 +186,88 @@ static int iommu_cmdbuf_show(struct seq_file *m, void *unused)
}
DEFINE_SHOW_ATTRIBUTE(iommu_cmdbuf);
+static ssize_t devid_write(struct file *filp, const char __user *ubuf,
+ size_t cnt, loff_t *ppos)
+{
+ struct amd_iommu_pci_seg *pci_seg;
+ int seg, bus, slot, func;
+ struct amd_iommu *iommu;
+ char *srcid_ptr;
+ int i, ret = cnt;
+ u16 devid;
+
+ if (cnt >= DEVID_IN_SZ) {
+ ret = -EINVAL;
+ goto err;
+ }
+
+ srcid_ptr = memdup_user_nul(ubuf, cnt);
+ if (IS_ERR(srcid_ptr)) {
+ ret = PTR_ERR(srcid_ptr);
+ goto err;
+ }
+
+ i = sscanf(srcid_ptr, "%x:%x:%x.%x", &seg, &bus, &slot, &func);
+ if (i != 4) {
+ i = sscanf(srcid_ptr, "%x:%x.%x", &bus, &slot, &func);
+ if (i != 3) {
+ ret = -EINVAL;
+ goto free;
+ }
+ seg = 0;
+ }
+
+ devid = PCI_DEVID(bus, PCI_DEVFN(slot, func));
+
+ /* Check if user device id input is a valid input */
+ for_each_pci_segment(pci_seg) {
+ if (pci_seg->id != seg)
+ continue;
+ if (devid > pci_seg->last_bdf) {
+ ret = -EINVAL;
+ goto free;
+ }
+ iommu = pci_seg->rlookup_table[devid];
+ if (!iommu) {
+ ret = -ENODEV;
+ goto free;
+ }
+ break;
+ }
+
+ if (pci_seg->id != seg) {
+ ret = -EINVAL;
+ goto free;
+ }
+
+ sbdf = PCI_SEG_DEVID_TO_SBDF(seg, devid);
+
+ kfree(srcid_ptr);
+ *ppos += cnt;
+ return ret;
+
+free:
+ kfree(srcid_ptr);
+err:
+ sbdf = -1;
+ return ret;
+}
+
+static int devid_show(struct seq_file *m, void *unused)
+{
+ u16 devid;
+
+ if (sbdf >= 0) {
+ devid = PCI_SBDF_TO_DEVID(sbdf);
+ seq_printf(m, "%04x:%02x:%02x:%x\n", PCI_SBDF_TO_SEGID(sbdf),
+ PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid));
+ } else
+ seq_puts(m, "No or Invalid input provided\n");
+
+ return 0;
+}
+DEFINE_SHOW_STORE_ATTRIBUTE(devid);
+
void amd_iommu_debugfs_setup(void)
{
struct amd_iommu *iommu;
@@ -206,4 +290,6 @@ void amd_iommu_debugfs_setup(void)
debugfs_create_file("cmdbuf", 0444, iommu->debugfs, iommu,
&iommu_cmdbuf_fops);
}
+ debugfs_create_file("devid", 0644, amd_iommu_debugfs, NULL,
+ &devid_fops);
}
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 6/8] iommu/amd: Add debugfs support to dump device table
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
` (4 preceding siblings ...)
2024-10-08 10:40 ` [PATCH 5/8] iommu/amd: Add support for device id user input Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 7/8] iommu/amd: Add debugfs support to dump IRT Table Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 8/8] iommu/amd: Add documentation for AMD IOMMU debugfs support Dheeraj Kumar Srivastava
7 siblings, 0 replies; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
IOMMU uses device table data structure to get per-device information for
DMA remapping, interrupt remapping, and other functionalities. It's a
valuable data structure to visualize for debugging issues related to
IOMMU.
eg.
To dump device table entry for a particular device
#echo 0000:c4:00.0 > /sys/kernel/debug/iommu/amd/devid
#cat /sys/kernel/debug/iommu/amd/devtbl
or
#echo c4:00.0 > /sys/kernel/debug/iommu/amd/devid
#cat /sys/kernel/debug/iommu/amd/devtbl
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
drivers/iommu/amd/debugfs.c | 49 +++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/drivers/iommu/amd/debugfs.c b/drivers/iommu/amd/debugfs.c
index 0d11250c1a00..61a541019432 100644
--- a/drivers/iommu/amd/debugfs.c
+++ b/drivers/iommu/amd/debugfs.c
@@ -268,6 +268,53 @@ static int devid_show(struct seq_file *m, void *unused)
}
DEFINE_SHOW_STORE_ATTRIBUTE(devid);
+static void dump_dte(struct seq_file *m, struct amd_iommu_pci_seg *pci_seg, u16 devid)
+{
+ struct dev_table_entry *dev_table;
+ struct amd_iommu *iommu;
+
+ iommu = pci_seg->rlookup_table[devid];
+ if (!iommu)
+ return;
+
+ dev_table = get_dev_table(iommu);
+ if (!dev_table) {
+ seq_puts(m, "Device table not found");
+ return;
+ }
+
+ seq_printf(m, "%-12s %16s %16s %16s %16s iommu\n", "DeviceId",
+ "QWORD[3]", "QWORD[2]", "QWORD[1]", "QWORD[0]");
+ seq_printf(m, "%04x:%02x:%02x:%x ", pci_seg->id, PCI_BUS_NUM(devid),
+ PCI_SLOT(devid), PCI_FUNC(devid));
+ for (int i = 3; i >= 0; --i)
+ seq_printf(m, "%016llx ", dev_table[devid].data[i]);
+ seq_printf(m, "iommu%d\n", iommu->index);
+}
+
+static int iommu_devtbl_show(struct seq_file *m, void *unused)
+{
+ struct amd_iommu_pci_seg *pci_seg;
+ u16 seg, devid;
+
+ if (sbdf < 0) {
+ seq_puts(m, "Please provide valid device id input\n");
+ return 0;
+ }
+ seg = PCI_SBDF_TO_SEGID(sbdf);
+ devid = PCI_SBDF_TO_DEVID(sbdf);
+
+ for_each_pci_segment(pci_seg) {
+ if (pci_seg->id != seg)
+ continue;
+ dump_dte(m, pci_seg, devid);
+ break;
+ }
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(iommu_devtbl);
+
void amd_iommu_debugfs_setup(void)
{
struct amd_iommu *iommu;
@@ -292,4 +339,6 @@ void amd_iommu_debugfs_setup(void)
}
debugfs_create_file("devid", 0644, amd_iommu_debugfs, NULL,
&devid_fops);
+ debugfs_create_file("devtbl", 0444, amd_iommu_debugfs, NULL,
+ &iommu_devtbl_fops);
}
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 7/8] iommu/amd: Add debugfs support to dump IRT Table
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
` (5 preceding siblings ...)
2024-10-08 10:40 ` [PATCH 6/8] iommu/amd: Add debugfs support to dump device table Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-08 10:40 ` [PATCH 8/8] iommu/amd: Add documentation for AMD IOMMU debugfs support Dheeraj Kumar Srivastava
7 siblings, 0 replies; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
In cases where we have an issue in the device interrupt path with IOMMU
interrupt remapping enabled, dumping valid IRT table entries for the device
is very useful and good input for debugging the issue.
eg.
To dump irte entries for a particular device
#echo "c4:00.0" > /sys/kernel/debug/iommu/amd/devid
#cat /sys/kernel/debug/iommu/amd/irqtbl | less
or
#echo "0000:c4:00.0" > /sys/kernel/debug/iommu/amd/devid
#cat /sys/kernel/debug/iommu/amd/irqtbl | less
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
drivers/iommu/amd/debugfs.c | 89 +++++++++++++++++++++++++++++++++++++
1 file changed, 89 insertions(+)
diff --git a/drivers/iommu/amd/debugfs.c b/drivers/iommu/amd/debugfs.c
index 61a541019432..c61daf921441 100644
--- a/drivers/iommu/amd/debugfs.c
+++ b/drivers/iommu/amd/debugfs.c
@@ -11,6 +11,7 @@
#include <linux/pci.h>
#include "amd_iommu.h"
+#include "../irq_remapping.h"
static struct dentry *amd_iommu_debugfs;
@@ -315,6 +316,92 @@ static int iommu_devtbl_show(struct seq_file *m, void *unused)
}
DEFINE_SHOW_ATTRIBUTE(iommu_devtbl);
+static void dump_128_irte(struct seq_file *m, struct irq_remap_table *table)
+{
+ struct irte_ga *ptr, *irte;
+ int index;
+
+ for (index = 0; index < MAX_IRQS_PER_TABLE; index++) {
+ ptr = (struct irte_ga *)table->table;
+ irte = &ptr[index];
+
+ if (AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir) &&
+ !irte->lo.fields_vapic.valid)
+ continue;
+ else if (!irte->lo.fields_remap.valid)
+ continue;
+ seq_printf(m, "IRT[%04d] %016llx%016llx\n", index, irte->hi.val, irte->lo.val);
+ }
+}
+
+static void dump_32_irte(struct seq_file *m, struct irq_remap_table *table)
+{
+ union irte *ptr, *irte;
+ int index;
+
+ for (index = 0; index < MAX_IRQS_PER_TABLE; index++) {
+ ptr = (union irte *)table->table;
+ irte = &ptr[index];
+
+ if (!irte->fields.valid)
+ continue;
+ seq_printf(m, "IRT[%04d] %08x\n", index, irte->val);
+ }
+}
+
+static void dump_irte(struct seq_file *m, u16 devid, struct amd_iommu_pci_seg *pci_seg)
+{
+ struct irq_remap_table *table;
+ unsigned long flags;
+
+ table = pci_seg->irq_lookup_table[devid];
+ if (!table) {
+ seq_printf(m, "IRQ lookup table not set for %04x:%02x:%02x:%x\n",
+ pci_seg->id, PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid));
+ return;
+ }
+
+ seq_printf(m, "DeviceId %04x:%02x:%02x:%x\n", pci_seg->id, PCI_BUS_NUM(devid),
+ PCI_SLOT(devid), PCI_FUNC(devid));
+
+ raw_spin_lock_irqsave(&table->lock, flags);
+ if (AMD_IOMMU_GUEST_IR_GA(amd_iommu_guest_ir))
+ dump_128_irte(m, table);
+ else
+ dump_32_irte(m, table);
+ seq_puts(m, "\n");
+ raw_spin_unlock_irqrestore(&table->lock, flags);
+}
+
+static int iommu_irqtbl_show(struct seq_file *m, void *unused)
+{
+ struct amd_iommu_pci_seg *pci_seg;
+ u16 devid, seg;
+
+ if (!irq_remapping_enabled) {
+ seq_puts(m, "Interrupt remapping is disabled\n");
+ return 0;
+ }
+
+ if (sbdf < 0) {
+ seq_puts(m, "Please provide valid device id input\n");
+ return 0;
+ }
+
+ seg = PCI_SBDF_TO_SEGID(sbdf);
+ devid = PCI_SBDF_TO_DEVID(sbdf);
+
+ for_each_pci_segment(pci_seg) {
+ if (pci_seg->id != seg)
+ continue;
+ dump_irte(m, devid, pci_seg);
+ break;
+ }
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(iommu_irqtbl);
+
void amd_iommu_debugfs_setup(void)
{
struct amd_iommu *iommu;
@@ -341,4 +428,6 @@ void amd_iommu_debugfs_setup(void)
&devid_fops);
debugfs_create_file("devtbl", 0444, amd_iommu_debugfs, NULL,
&iommu_devtbl_fops);
+ debugfs_create_file("irqtbl", 0444, amd_iommu_debugfs, NULL,
+ &iommu_irqtbl_fops);
}
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 8/8] iommu/amd: Add documentation for AMD IOMMU debugfs support
2024-10-08 10:40 [PATCH 0/8] Introduce debugfs support in IOMMU Dheeraj Kumar Srivastava
` (6 preceding siblings ...)
2024-10-08 10:40 ` [PATCH 7/8] iommu/amd: Add debugfs support to dump IRT Table Dheeraj Kumar Srivastava
@ 2024-10-08 10:40 ` Dheeraj Kumar Srivastava
2024-10-09 1:32 ` kernel test robot
7 siblings, 1 reply; 10+ messages in thread
From: Dheeraj Kumar Srivastava @ 2024-10-08 10:40 UTC (permalink / raw)
To: joro, suravee.suthikulpanit, will, robin.murphy,
dheerajkumar.srivastava, linux-kernel, iommu
Add documentation describing how to use AMD IOMMU debugfs support to
dump IOMMU data structures - IRT table, Device table, Registers (MMIO and
Capability) and command buffer.
Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
Documentation/ABI/testing/debugfs-amd-iommu | 140 ++++++++++++++++++++
1 file changed, 140 insertions(+)
create mode 100644 Documentation/ABI/testing/debugfs-amd-iommu
diff --git a/Documentation/ABI/testing/debugfs-amd-iommu b/Documentation/ABI/testing/debugfs-amd-iommu
new file mode 100644
index 000000000000..7acdb38bdf7b
--- /dev/null
+++ b/Documentation/ABI/testing/debugfs-amd-iommu
@@ -0,0 +1,140 @@
+What: /sys/kernel/debug/iommu/amd/iommu<x>/mmio
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This is an input read/write access file. In this file, the user input
+ mmio register offset for iommu<x> to print corresponding mmio register
+ of iommu<x>.
+
+ Example:
+ $ echo "0x18" > /sys/kernel/debug/iommu/amd/iommu00/mmio
+ $ cat /sys/kernel/debug/iommu/amd/iommu00/mmio
+
+ Output:
+ 0x18
+
+What: /sys/kernel/debug/iommu/amd/iommu<x>/mmio_dump
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This is an output read only file that contains mmio register
+ value at offset "/sys/kernel/debug/iommu/amd/iommu<x>/mmio".
+
+ Example:
+ $ cat /sys/kernel/debug/iommu/amd/iommu00/mmio_dump
+
+ Output:
+ 0x0003f48d
+
+What: /sys/kernel/debug/iommu/amd/iommu<x>/capability
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This is an input read/write access file. In this file, the user input
+ capability register offset for iommu<x> to print corresponding capability
+ register of iommu<x>.
+
+ Example:
+ $ echo "0x10" > /sys/kernel/debug/iommu/amd/iommu00/capability
+ $ cat /sys/kernel/debug/iommu/amd/iommu00/capability
+
+ Output:
+ 0x10
+
+What: /sys/kernel/debug/iommu/amd/iommu<x>/capability_dump
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This is an output read only file that contains capability register
+ value at offset "/sys/kernel/debug/iommu/amd/iommu<x>/capability".
+
+ Example:
+ $ cat /sys/kernel/debug/iommu/amd/iommu00/capability_dump
+
+ Output:
+ 0x00203040
+
+What: /sys/kernel/debug/iommu/amd/iommu<x>/cmdbuf
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This file is an output read only file that contains iommu<x> command
+ buffer entries.
+
+ Examples:
+ $ cat /sys/kernel/debug/iommu/amd/iommu<x>/cmdbuf
+
+ Output:
+ CMD Buffer Head pointer register:0x00000b90 Tail pointer register:0x00000b90
+ 0: 008350011000000100003c0000000000
+ 1: 0000000030000005fffff0037fffffff
+ 2: 008350011000000100003c0100000000
+ 3: 0000000030000005fffff0037fffffff
+ 4: 008350011000000100003c0200000000
+ 5: 0000000030000005fffff0037fffffff
+ 6: 008350011000000100003c0300000000
+ 7: 0000000030000005fffff0037fffffff
+ 8: 008350011000000100003c0400000000
+ 9: 0000000030000005fffff0037fffffff
+ 10: 008350011000000100003c0500000000
+ 11: 0000000030000005fffff0037fffffff
+ [...]
+
+What: /sys/kernel/debug/iommu/amd/devid
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This is an input read/write file that takes device id user input.
+ This input can be used for dumping iommu data structures like
+ interrupt remapping table, device table etc.
+
+ Example:
+ 1.
+ $ echo 0000:01:00.0 > /sys/kernel/debug/iommu/amd/devid
+ $ cat /sys/kernel/debug/iommu/amd/devid
+
+ Output:
+ 0000:01:00.0
+
+ 2.
+ $ echo 01:00.0 > /sys/kernel/debug/iommu/amd/devid
+ $ cat /sys/kernel/debug/iommu/amd/devid
+
+ Output:
+ 0000:01:00.0
+
+What: /sys/kernel/debug/iommu/devtbl
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This is an output read only file that contains device table entry for
+ the device id input given in /sys/kernel/debug/iommu/amd/devid.
+
+ Example:
+ $ cat /sys/kernel/debug/iommu/amd/devtbl
+
+ Output:
+ DeviceId QWORD[3] QWORD[2] QWORD[1] QWORD[0] iommu
+ 0000:01:00.0 0000000000000000 20000001373b8013 0000000000000038 6000000114d7b603 iommu3
+
+What: /sys/kernel/debug/iommu/devtbl
+Date: September 2024
+Contact: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
+Description:
+ This is an output read only file that contains IRT table valid entries
+ for the device id input given in /sys/kernel/debug/iommu/amd/devid.
+
+ Example:
+ $ cat /sys/kernel/debug/iommu/amd/irqtbl
+
+ Output:
+ DeviceId 0000:01:00.0
+ IRT[0000] 00000000000000200000000000000241
+ IRT[0001] 00000000000000200000000000000841
+ IRT[0002] 00000000000000200000000000002041
+ IRT[0003] 00000000000000200000000000008041
+ IRT[0004] 00000000000000200000000000020041
+ IRT[0005] 00000000000000200000000000080041
+ IRT[0006] 00000000000000200000000000200041
+ IRT[0007] 00000000000000200000000000800041
+ [...]
--
2.25.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH 8/8] iommu/amd: Add documentation for AMD IOMMU debugfs support
2024-10-08 10:40 ` [PATCH 8/8] iommu/amd: Add documentation for AMD IOMMU debugfs support Dheeraj Kumar Srivastava
@ 2024-10-09 1:32 ` kernel test robot
0 siblings, 0 replies; 10+ messages in thread
From: kernel test robot @ 2024-10-09 1:32 UTC (permalink / raw)
To: Dheeraj Kumar Srivastava, joro, suravee.suthikulpanit, will,
robin.murphy, linux-kernel, iommu
Cc: oe-kbuild-all
Hi Dheeraj,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 8cf0b93919e13d1e8d4466eb4080a4c4d9d66d7b]
url: https://github.com/intel-lab-lkp/linux/commits/Dheeraj-Kumar-Srivastava/iommu-amd-Refactor-AMD-IOMMU-debugfs-initial-setup/20241008-184419
base: 8cf0b93919e13d1e8d4466eb4080a4c4d9d66d7b
patch link: https://lore.kernel.org/r/20241008104035.4008-9-dheerajkumar.srivastava%40amd.com
patch subject: [PATCH 8/8] iommu/amd: Add documentation for AMD IOMMU debugfs support
reproduce: (https://download.01.org/0day-ci/archive/20241009/202410090917.z8aO1bNH-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202410090917.z8aO1bNH-lkp@intel.com/
All warnings (new ones prefixed by >>):
Warning: Documentation/devicetree/bindings/regulator/siliconmitus,sm5703-regulator.yaml references a file that doesn't exist: Documentation/devicetree/bindings/mfd/siliconmitus,sm5703.yaml
Warning: Documentation/hwmon/g762.rst references a file that doesn't exist: Documentation/devicetree/bindings/hwmon/g762.txt
Warning: MAINTAINERS references a file that doesn't exist: Documentation/devicetree/bindings/reserved-memory/qcom
Warning: MAINTAINERS references a file that doesn't exist: Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt
>> Warning: /sys/kernel/debug/iommu/devtbl is defined 2 times: ./Documentation/ABI/testing/debugfs-amd-iommu:105 ./Documentation/ABI/testing/debugfs-amd-iommu:119
Using alabaster theme
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 10+ messages in thread