All of lore.kernel.org
 help / color / mirror / Atom feed
From: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
To: "Jingoo Han" <jingoohan1@gmail.com>,
	"Manivannan Sadhasivam" <mani@kernel.org>,
	"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
	"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
	"Rob Herring" <robh@kernel.org>,
	"Bjorn Helgaas" <bhelgaas@google.com>
Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
	Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
Subject: [PATCH] PCI: dwc: Add sysfs for local loopback interface
Date: Mon, 03 Aug 2026 15:04:15 +0530	[thread overview]
Message-ID: <20260803-loopback-v1-1-508f7f19e47a@oss.qualcomm.com> (raw)

The PCIe Base Specification (r6.0, sec. 4.2.7.10) defines a Loopback
state in the LTSSM: the Loopback Master sets the Loopback bit in TS1
Ordered Sets, and the Loopback Slave reflects received data back
bit-for-bit.  DesignWare PCIe IP also exposes a local variant, entirely
within the controller and not requiring a connected endpoint, via
PCIE_PIPE_LOOPBACK_CONTROL and PORT_LINK_LOOPBACK_EN.

This patch focuses on local loopback only.  Remote (link-level)
loopback requires the link to be up while the test runs, which means
any client/endpoint drivers using the link would need to be told to
stop transfers beforehand -- up to and including a driver remove -- and
the controller/link would need to be reinitiated afterward.  That
coordination needs further discussion, so it is left for a follow-up.

Add a sysfs attribute group 'loopback/' on the platform device with:

  run      (RW) -- write "local" to run a synchronous, blocking
                   data-integrity test; reads back "idle" or "busy" to
                   indicate test state.  Refused with -EBUSY while the
                   link is up, so it cannot disrupt an active link or
                   other devices behind the same root port.

  buf_size (RW) -- transfer size in bytes, rounded up to the next
                   power-of-two and clamped to a minimum of 4 KiB and a
                   maximum of 1 MiB (default 4 KiB); cannot be changed
                   during a test

The test allocates a coherent DMA buffer, programs an inbound iATU
window to redirect PCIe writes to it, writes random data through the
PCIe window, and compares the result byte-by-byte.

On completion the loopback/link-control DBI bits are restored directly
rather than resetting the whole controller, and the LTSSM is polled for
DETECT_QUIET.  On some platforms it has been observed (and reproduced
on Qualcomm Eliza hardware) to land elsewhere instead and leave the
controller unable to service further runs; in that case the controller
is recovered via the same ops->deinit() + dw_pcie_resume_noirq()
sequence used across a system suspend/resume cycle, which has been
validated to reliably restore the controller across repeated
back-to-back test runs.

The sysfs group is added/removed via device_add_group()/
device_remove_group() rather than the devm_* variant, with removal
called explicitly at the start of dw_pcie_host_deinit() before the
rest of the bridge is torn down, since device_remove_group() blocks
until any in-flight run_store()/run_show() call returns -- closing a
race where driver unbind/remove could otherwise run concurrently with
a sysfs write.

Signed-off-by: Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>
---
 .../testing/sysfs-driver-pcie-designware-loopback  |  69 +++++
 MAINTAINERS                                        |   1 +
 drivers/pci/controller/dwc/pcie-designware-host.c  | 298 +++++++++++++++++++++
 drivers/pci/controller/dwc/pcie-designware.h       |  17 ++
 4 files changed, 385 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-driver-pcie-designware-loopback b/Documentation/ABI/testing/sysfs-driver-pcie-designware-loopback
new file mode 100644
index 000000000000..096da7800549
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-driver-pcie-designware-loopback
@@ -0,0 +1,69 @@
+What:		/sys/devices/.../loopback/run
+Date:		August 2026
+KernelVersion:	7.2
+Contact:	linux-pci@vger.kernel.org
+Description:
+		(RW) Diagnostic local loopback test interface for Synopsys
+		DesignWare PCIe root-port controllers.
+
+		Reading this file returns the current test state:
+
+		  - ``idle``  - no test is running
+		  - ``busy``  - a test is in progress
+
+		Writing ``local`` triggers a synchronous, blocking
+		data-integrity test performed entirely within the local
+		controller, without requiring a connected endpoint.  The
+		write does not return until the test completes, and logs
+		``PASSED`` or ``FAILED`` via dev_info().
+
+		The test allocates a coherent DMA buffer, configures an
+		inbound iATU window to redirect PCIe writes to that buffer,
+		writes random data through the PCIe window, and compares
+		the result byte-by-byte against what was written.
+
+		The test only runs while the PCIe link is down; writing
+		``local`` while the link is up returns -EBUSY.  This avoids
+		disrupting an active link and any devices behind it.
+
+		Only one test may run at a time.  Concurrent writes
+		return -EBUSY.
+
+		Example::
+
+		  # Run a local loopback test
+		  $ echo local  > /sys/devices/.../loopback/run
+
+		  # Check whether a test is in progress
+		  $ cat /sys/devices/.../loopback/run
+		  idle
+
+What:		/sys/devices/.../loopback/buf_size
+Date:		August 2026
+KernelVersion:	7.2
+Contact:	linux-pci@vger.kernel.org
+Description:
+		(RW) Size in bytes of the data buffer used by the
+		loopback test.
+
+		Reading this file returns the current buffer size as a
+		decimal integer.
+
+		Writing a decimal integer sets a new buffer size.  The
+		value is rounded up to the next power of two and clamped
+		to a minimum of 4096 (4 KiB) and a maximum of 1048576
+		(1 MiB); values above the maximum are rejected with
+		-EINVAL.  The new size takes effect on the next test run;
+		it cannot be changed while a test is in progress (returns
+		-EBUSY).
+
+		The default value is 4096.
+
+		Example::
+
+		  # Set the test buffer to 64 KiB
+		  $ echo 65536 > /sys/devices/.../loopback/buf_size
+
+		  # Read back the (rounded) value
+		  $ cat /sys/devices/.../loopback/buf_size
+		  65536
diff --git a/MAINTAINERS b/MAINTAINERS
index 0d7987278c07..44a11de7cc8d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -20761,6 +20761,7 @@ M:	Jingoo Han <jingoohan1@gmail.com>
 M:	Manivannan Sadhasivam <mani@kernel.org>
 L:	linux-pci@vger.kernel.org
 S:	Maintained
+F:	Documentation/ABI/testing/sysfs-driver-pcie-designware-loopback
 F:	Documentation/devicetree/bindings/pci/snps,dw-pcie-ep.yaml
 F:	Documentation/devicetree/bindings/pci/snps,dw-pcie.yaml
 F:	drivers/pci/controller/dwc/*designware*
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 06722259d2e3..f0cfe9e88520 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -19,6 +19,8 @@
 #include <linux/pci.h>
 #include <linux/pci_regs.h>
 #include <linux/platform_device.h>
+#include <linux/random.h>
+#include <linux/sizes.h>
 
 #include "../pci-host-common.h"
 #include "../../pci.h"
@@ -567,6 +569,297 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 	return 0;
 }
 
+#define DW_PCIE_LB_BUF_SIZE_MIN	SZ_4K
+#define DW_PCIE_LB_BUF_SIZE_MAX	SZ_1M
+#define DW_PCIE_LB_SETTLE_MS		100
+#define DW_PCIE_LB_LTSSM_SETTLE_US	(DW_PCIE_LB_SETTLE_MS * USEC_PER_MSEC)
+#define DW_PCIE_LB_LTSSM_POLL_US	1000
+
+static int dw_pcie_loopback_run(struct dw_pcie_rp *pp, size_t buf_size)
+{
+	struct resource lb_res = { .name = "pcie-loopback",
+				   .flags = IORESOURCE_MEM };
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	enum dw_pcie_ltssm ltssm;
+	void __iomem *src_base;
+	u32 plc, gen3, pipe_lb;
+	dma_addr_t dst_dma;
+	void *dst_virt;
+	int ret, rret;
+	void *tx_buf;
+	int ib_index;
+
+	ib_index = pci->num_ib_windows - 1;
+
+	ret = pci_bus_alloc_resource(pp->bridge->bus, &lb_res,
+				     buf_size, buf_size,
+				     PCIBIOS_MIN_MEM, 0,
+				     pcibios_align_resource,
+				     &pp->bridge->dev);
+	if (ret) {
+		dev_err(pci->dev, "loopback: failed to alloc PCIe MEM resource: %d\n", ret);
+		return ret;
+	}
+
+	src_base = ioremap(lb_res.start, buf_size);
+	if (!src_base) {
+		dev_err(pci->dev, "loopback: ioremap of PCIe source window failed\n");
+		ret = -ENOMEM;
+		goto err_release_res;
+	}
+
+	dst_virt = dma_alloc_coherent(pci->dev, buf_size, &dst_dma, GFP_KERNEL);
+	if (!dst_virt) {
+		ret = -ENOMEM;
+		goto err_iounmap;
+	}
+
+	ret = dw_pcie_prog_inbound_atu(pci, ib_index, PCIE_TLP_TYPE_MEM_RDWR,
+				       dst_dma, lb_res.start, buf_size);
+	if (ret) {
+		dev_err(pci->dev, "loopback: inbound iATU programming failed: %d\n", ret);
+		goto err_free_dma;
+	}
+
+	plc = dw_pcie_readl_dbi(pci, PCIE_PORT_LINK_CONTROL);
+
+	gen3 = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF);
+	dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF,
+			   gen3 | GEN3_RELATED_OFF_GEN3_EQ_DISABLE);
+
+	pipe_lb = dw_pcie_readl_dbi(pci, PCIE_PIPE_LOOPBACK_CONTROL);
+	dw_pcie_writel_dbi(pci, PCIE_PIPE_LOOPBACK_CONTROL,
+			   pipe_lb | PCIE_PIPE_LOOPBACK_EN);
+
+	dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, plc | PORT_LINK_LOOPBACK_EN);
+
+	msleep(DW_PCIE_LB_SETTLE_MS);
+
+	tx_buf = kmalloc(buf_size, GFP_KERNEL);
+	if (!tx_buf) {
+		ret = -ENOMEM;
+		goto err_restore_link;
+	}
+
+	get_random_bytes(tx_buf, buf_size);
+	memcpy_toio(src_base, tx_buf, buf_size);
+	msleep(DW_PCIE_LB_SETTLE_MS);
+
+	if (memcmp(tx_buf, dst_virt, buf_size))
+		ret = -EIO;
+
+	dev_info(pci->dev, "PCIe local loopback test %s\n", ret ? "FAILED" : "PASSED");
+
+	kfree(tx_buf);
+
+err_restore_link:
+	dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, plc);
+	dw_pcie_writel_dbi(pci, PCIE_PIPE_LOOPBACK_CONTROL, pipe_lb);
+	dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, gen3);
+
+	if (read_poll_timeout(dw_pcie_get_ltssm, ltssm,
+			      ltssm == DW_PCIE_LTSSM_DETECT_QUIET,
+			      DW_PCIE_LB_LTSSM_POLL_US, DW_PCIE_LB_LTSSM_SETTLE_US,
+			      false, pci)) {
+		/*
+		 * LTSSM has been observed stuck outside DETECT_QUIET on some
+		 * platforms; only a full controller reinit (mirroring system
+		 * suspend/resume) reliably recovers it.
+		 */
+		dev_warn(pci->dev,
+			 "loopback: LTSSM did not settle in DETECT_QUIET (in %s), reinitializing controller\n",
+			 dw_pcie_ltssm_status_string(ltssm));
+
+		if (pp->ops->deinit)
+			pp->ops->deinit(pp);
+
+		pci->suspended = true;
+
+		rret = dw_pcie_resume_noirq(pci);
+		if (rret) {
+			dev_err(pci->dev, "loopback: controller reinit failed: %d\n", rret);
+			if (!ret)
+				ret = rret;
+		}
+
+		ltssm = dw_pcie_get_ltssm(pci);
+	} else {
+		dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, ib_index);
+	}
+
+	dev_info(pci->dev, "PCIe LTSSM state after loopback exit: %s\n",
+		 dw_pcie_ltssm_status_string(ltssm));
+
+err_free_dma:
+	dma_free_coherent(pci->dev, buf_size, dst_virt, dst_dma);
+err_iounmap:
+	iounmap(src_base);
+err_release_res:
+	release_resource(&lb_res);
+
+	return ret;
+}
+
+static ssize_t buf_size_show(struct device *dev,
+			     struct device_attribute *attr, char *buf)
+{
+	struct dw_pcie_loopback *lb =
+		container_of(attr, struct dw_pcie_loopback, attr_buf_size);
+
+	return sysfs_emit(buf, "%zu\n", lb->buf_size);
+}
+
+static ssize_t buf_size_store(struct device *dev,
+			      struct device_attribute *attr,
+			      const char *buf, size_t count)
+{
+	struct dw_pcie_loopback *lb =
+		container_of(attr, struct dw_pcie_loopback, attr_buf_size);
+	struct dw_pcie *pci = to_dw_pcie_from_pp(lb->pp);
+	unsigned long req;
+	size_t new_size;
+	int ret;
+
+	ret = kstrtoul(buf, 0, &req);
+	if (ret)
+		return ret;
+
+	if (!req)
+		return -EINVAL;
+
+	if (req > DW_PCIE_LB_BUF_SIZE_MAX)
+		return -EINVAL;
+
+	/* Round up to next power-of-two, then enforce the 4 KiB minimum */
+	new_size = max_t(size_t, DW_PCIE_LB_BUF_SIZE_MIN,
+			 roundup_pow_of_two((size_t)req));
+
+	if (mutex_lock_interruptible(&lb->lock))
+		return -ERESTARTSYS;
+
+	if (lb->busy) {
+		mutex_unlock(&lb->lock);
+		return -EBUSY;
+	}
+
+	lb->buf_size = new_size;
+	mutex_unlock(&lb->lock);
+
+	dev_dbg(pci->dev, "loopback: buf_size set to %zu bytes\n", new_size);
+
+	return count;
+}
+
+static ssize_t run_show(struct device *dev,
+			struct device_attribute *attr, char *buf)
+{
+	struct dw_pcie_loopback *lb =
+		container_of(attr, struct dw_pcie_loopback, attr_run);
+
+	return sysfs_emit(buf, "%s\n", lb->busy ? "busy" : "idle");
+}
+
+static ssize_t run_store(struct device *dev,
+			 struct device_attribute *attr,
+			 const char *buf, size_t count)
+{
+	struct dw_pcie_loopback *lb =
+		container_of(attr, struct dw_pcie_loopback, attr_run);
+	struct dw_pcie_rp *pp = lb->pp;
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	size_t buf_size;
+	int ret;
+
+	if (!sysfs_streq(buf, "local"))
+		return -EINVAL;
+
+	if (dw_pcie_link_up(pci)) {
+		dev_err(pci->dev,
+			"loopback: refusing to run while the link is up\n");
+		return -EBUSY;
+	}
+
+	if (mutex_lock_interruptible(&lb->lock))
+		return -ERESTARTSYS;
+
+	if (lb->busy) {
+		dev_warn(pci->dev, "loopback: test already in progress\n");
+		mutex_unlock(&lb->lock);
+		return -EBUSY;
+	}
+
+	lb->busy  = true;
+	buf_size  = lb->buf_size;
+	mutex_unlock(&lb->lock);
+
+	ret = dw_pcie_loopback_run(pp, buf_size);
+
+	mutex_lock(&lb->lock);
+	lb->busy = false;
+	mutex_unlock(&lb->lock);
+
+	return ret ? ret : count;
+}
+
+static int dw_pcie_loopback_sysfs_init(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct dw_pcie_loopback *lb;
+	int ret;
+
+	lb = devm_kzalloc(pci->dev, sizeof(*lb), GFP_KERNEL);
+	if (!lb)
+		return -ENOMEM;
+
+	lb->pp = pp;
+
+	mutex_init(&lb->lock);
+	lb->buf_size = DW_PCIE_LB_BUF_SIZE_MIN;
+
+	lb->attr_run      = (struct device_attribute)
+		__ATTR(run, 0644, run_show, run_store);
+	lb->attr_buf_size = (struct device_attribute)
+		__ATTR(buf_size, 0644, buf_size_show, buf_size_store);
+
+	sysfs_attr_init(&lb->attr_run.attr);
+	sysfs_attr_init(&lb->attr_buf_size.attr);
+
+	lb->attrs[0] = &lb->attr_run.attr;
+	lb->attrs[1] = &lb->attr_buf_size.attr;
+	lb->attrs[2] = NULL;
+	lb->attr_group.name  = "loopback";
+	lb->attr_group.attrs = lb->attrs;
+
+	pp->loopback = lb;
+
+	ret = device_add_group(pci->dev, &lb->attr_group);
+	if (ret) {
+		dev_err(pci->dev, "loopback: device_add_group failed: %d\n", ret);
+		pp->loopback = NULL;
+		return ret;
+	}
+
+	return 0;
+}
+
+static void dw_pcie_loopback_sysfs_deinit(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct dw_pcie_loopback *lb = pp->loopback;
+
+	if (!lb)
+		return;
+
+	/*
+	 * device_remove_group() blocks until any in-flight run_store()/
+	 * run_show() has returned, so pp/pci/bridge are guaranteed not to be
+	 * accessed by the loopback code once this returns. Must run before
+	 * the rest of dw_pcie_host_deinit() tears down pp->bridge.
+	 */
+	device_remove_group(pci->dev, &lb->attr_group);
+	pp->loopback = NULL;
+}
+
 int dw_pcie_host_init(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -675,6 +968,9 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 
 	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
 
+	if (dw_pcie_loopback_sysfs_init(pp))
+		dev_warn(dev, "failed to create loopback sysfs entry\n");
+
 	return 0;
 
 err_stop_link:
@@ -703,6 +999,8 @@ void dw_pcie_host_deinit(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
 
+	dw_pcie_loopback_sysfs_deinit(pp);
+
 	dwc_pcie_debugfs_deinit(pci);
 
 	pci_lock_rescan_remove();
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index de4b245b1758..dfcbcdf822da 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -88,6 +88,7 @@
 #define PORT_AFR_L1_ENTRANCE_LAT_MASK	GENMASK(29, 27)
 
 #define PCIE_PORT_LINK_CONTROL		0x710
+#define PORT_LINK_LOOPBACK_EN		BIT(2)
 #define PORT_LINK_DLL_LINK_EN		BIT(5)
 #define PORT_LINK_FAST_LINK_MODE	BIT(7)
 #define PORT_LINK_MODE_MASK		GENMASK(21, 16)
@@ -173,6 +174,9 @@
 #define COHERENCY_CONTROL_2_OFF			0x8E4
 #define COHERENCY_CONTROL_3_OFF			0x8E8
 
+#define PCIE_PIPE_LOOPBACK_CONTROL	0x8B8
+#define PCIE_PIPE_LOOPBACK_EN		BIT(31)
+
 #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
 #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
 
@@ -471,6 +475,19 @@ struct dw_pcie_rp {
 	bool			native_ecam;
 	bool                    skip_l23_ready;
 	bool			skip_pwrctrl_off;
+	struct dw_pcie_loopback	*loopback;
+};
+
+struct dw_pcie_loopback {
+	struct dw_pcie_rp	*pp;
+	/* Protects busy and buf_size against concurrent sysfs access */
+	struct mutex		lock;
+	bool			busy;
+	size_t			buf_size;
+	struct device_attribute	attr_run;
+	struct device_attribute	attr_buf_size;
+	struct attribute	*attrs[3];	/* run, buf_size, NULL */
+	struct attribute_group	attr_group;
 };
 
 struct dw_pcie_ep_ops {

---
base-commit: 8ba098e6b6ff0db8edf28528d1552be261af30d4
change-id: 20250130-loopback-60e4cd7bbfe7

Best regards,
--  
Krishna Chaitanya Chundru <krishna.chundru@oss.qualcomm.com>


             reply	other threads:[~2026-08-03  9:35 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-03  9:34 Krishna Chaitanya Chundru [this message]
2026-08-03  9:47 ` [PATCH] PCI: dwc: Add sysfs for local loopback interface sashiko-bot
2026-08-04 16:11   ` Krishna Chaitanya Chundru

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260803-loopback-v1-1-508f7f19e47a@oss.qualcomm.com \
    --to=krishna.chundru@oss.qualcomm.com \
    --cc=bhelgaas@google.com \
    --cc=jingoohan1@gmail.com \
    --cc=kwilczynski@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lpieralisi@kernel.org \
    --cc=mani@kernel.org \
    --cc=robh@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.