[PATCH 0/2] spi: Implement a atomic transfer as a last resort

[PATCH 0/2] spi: Implement a atomic transfer as a last resort

[Heiko Stuebner]

SPI devices at times are used as PMICs including a power-controller
functionality that should turn off a system.

This happens in an atomic context very late during the shutdown, where
no interrupts are available anymore and conflicts with the asynchronous
nature of the SPI subsystem.


====== History ======
For the RK806 spi-connected PMIC this has already caused interesting
artifacts.

A first change, tried to "fix" the issue by using the
SYS_OFF_MODE_POWER_OFF_PREPARE step of poweroff. While this hides the
issue, on some devices this now turns the pmic off too early before
other important steps are done.

To mitigate that issue for "some" devices, a change [0] tried to use
different poweroff phases for i2c and spi devices, where Lee rightfully
said, that with this no proper solution would be done - and Lee was right,
as that issue now is open since 2024.

The newest change is a simple revert back the old way [1] because of
those issues caused on other devices, but that now breaks the SPI side
again and also did not take into account that long history.


====== Now =======
The I2C framework has an atomic transfer callback for exactly this late
stage transfers since
commit 63b96983a5dd ("i2c: core: introduce callbacks for atomic transfers")
and during the discussion it was suggested implementing something like
this for SPI too.


As this did not seem to have happened yet, I thought I'd try my luck.

Similar to I2C this is meant only for that latest stage so I stole the
gating from there too.

I assume I did not take some SPI specifics into account, but I guess
having some code to discuss is a starting point at least :-)



[0] https://lore.kernel.org/lkml/20240730180903.81688-1-sebastian.reichel@collabora.com/
[1] https://lore.kernel.org/all/20260908-veyron-shutdown-v1-1-066ecb80f5f2@gmail.com/


Cc: Heiko Stuebner <heiko@sntech.de>
Cc: linux-spi@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-rockchip@lists.infradead.org
Cc: quentin.schulz@cherry.de
Cc: Lee Jones <lee@kernel.org>
Cc: Sebastian Reichel <sebastian.reichel@collabora.com>


Heiko Stuebner (2):
  spi: Add infrastructure for atomic sync transfers
  spi: rockchip: add an atomic transfer function

 drivers/spi/spi-rockchip.c | 88 ++++++++++++++++++++++++++++++++++++++
 drivers/spi/spi.c          | 66 ++++++++++++++++++++++++++--
 include/linux/spi/spi.h    | 11 +++++
 3 files changed, 161 insertions(+), 4 deletions(-)

-- 
2.47.3

[PATCH 1/2] spi: Add infrastructure for atomic sync transfers

[Heiko Stuebner]

From: Heiko Stuebner <heiko.stuebner@cherry.de>

There are cases where SPI transfers are needed during times without
interrupts like shutdown when a PMIC should turn off the system.

I2C already has atomic transfers for exactly that case, so introduce a
similar concept for SPI. Similar to I2C, this is and should stay a
corner-case, so is only allowed during the shutdown stages of the kernel,
when interrupts are already turned off.

So add a in-atomic check stolen from I2C and an optional transfer_atomic
callback drivers can implement to do immediate uninterrupted transfers
and let spi_sync() determine if the state is late enough to use it.

spi_async() does not require a sleeping context for itself, so while it
could in theory also be included, its called waiting for the completion
can sleep, so would still cause the splat about it.

Signed-off-by: Heiko Stuebner <heiko.stuebner@cherry.de>
---
 drivers/spi/spi.c       | 66 ++++++++++++++++++++++++++++++++++++++---
 include/linux/spi/spi.h | 11 +++++++
 2 files changed, 73 insertions(+), 4 deletions(-)

diff --git a/drivers/spi/spi.c b/drivers/spi/spi.c
index 40cda0037ea5..b85ac20e63f6 100644
--- a/drivers/spi/spi.c
+++ b/drivers/spi/spi.c
@@ -5016,6 +5016,49 @@ static int __spi_sync(struct spi_device *spi, struct spi_message *message)
 	return status;
 }
 
+static int __spi_sync_atomic(struct spi_device *spi, struct spi_message *message)
+{
+	struct spi_controller *ctlr = spi->controller;
+	unsigned long flags;
+	int status;
+
+	if (!ctlr->transfer_atomic) {
+		dev_err(&ctlr->dev, "No atomic transfer handler defined\n");
+		return -EOPNOTSUPP;
+	}
+
+	if (__spi_check_suspended(ctlr)) {
+		dev_warn_once(&spi->dev, "Attempted to sync while suspend\n");
+		return -ESHUTDOWN;
+	}
+
+	status = spi_maybe_optimize_message(spi, message);
+	if (status)
+		return status;
+
+	spin_lock_irqsave(&ctlr->bus_lock_spinlock, flags);
+	status = ctlr->transfer_atomic(spi, message);
+	spin_unlock_irqrestore(&ctlr->bus_lock_spinlock, flags);
+
+	spi_maybe_unoptimize_message(message);
+
+	if (status == 0)
+		status = message->status;
+
+	return status;
+}
+
+/*
+ * We only allow atomic transfers for very late communication, e.g. to access a
+ * PMIC when powering down. Atomic transfers are a corner case and not for
+ * generic use!
+ */
+static inline bool spi_in_atomic_sync_mode(void)
+{
+	return system_state > SYSTEM_RUNNING &&
+	       (IS_ENABLED(CONFIG_PREEMPT_COUNT) ? !preemptible() : irqs_disabled());
+}
+
 /**
  * spi_sync - blocking/synchronous SPI data transfers
  * @spi: device with which data will be exchanged
@@ -5026,6 +5069,10 @@ static int __spi_sync(struct spi_device *spi, struct spi_message *message)
  * is non-interruptible, and has no timeout.  Low-overhead controller
  * drivers may DMA directly into and out of the message buffers.
  *
+ * During the time when the system is being shutdown and a transfer_atomic
+ * function is set, this call may also be used from a context that does
+ * not allow sleeping, i.e. with disabled interrupts.
+ *
  * Note that the SPI device's chip select is active during the message,
  * and then is normally disabled between messages.  Drivers for some
  * frequently-used devices may want to minimize costs of selecting a chip,
@@ -5041,9 +5088,13 @@ int spi_sync(struct spi_device *spi, struct spi_message *message)
 {
 	int ret;
 
-	mutex_lock(&spi->controller->bus_lock_mutex);
-	ret = __spi_sync(spi, message);
-	mutex_unlock(&spi->controller->bus_lock_mutex);
+	if (spi_in_atomic_sync_mode()) {
+		ret = __spi_sync_atomic(spi, message);
+	} else {
+		mutex_lock(&spi->controller->bus_lock_mutex);
+		ret = __spi_sync(spi, message);
+		mutex_unlock(&spi->controller->bus_lock_mutex);
+	}
 
 	return ret;
 }
@@ -5059,6 +5110,10 @@ EXPORT_SYMBOL_GPL(spi_sync);
  * is non-interruptible, and has no timeout.  Low-overhead controller
  * drivers may DMA directly into and out of the message buffers.
  *
+ * During the time when the system is being shutdown and a transfer_atomic
+ * function is set, this call may also be used from a context that does
+ * not allow sleeping, i.e. with disabled interrupts.
+ *
  * This call should be used by drivers that require exclusive access to the
  * SPI bus. It has to be preceded by a spi_bus_lock call. The SPI bus must
  * be released by a spi_bus_unlock call when the exclusive access is over.
@@ -5067,7 +5122,10 @@ EXPORT_SYMBOL_GPL(spi_sync);
  */
 int spi_sync_locked(struct spi_device *spi, struct spi_message *message)
 {
-	return __spi_sync(spi, message);
+	if (spi_in_atomic_sync_mode())
+		return __spi_sync_atomic(spi, message);
+	else
+		return __spi_sync(spi, message);
 }
 EXPORT_SYMBOL_GPL(spi_sync_locked);
 
diff --git a/include/linux/spi/spi.h b/include/linux/spi/spi.h
index fb4baa0d3398..2b179b5d598c 100644
--- a/include/linux/spi/spi.h
+++ b/include/linux/spi/spi.h
@@ -459,6 +459,8 @@ extern struct spi_device *devm_spi_new_ancillary_device(struct spi_device *spi,
  * controller for configuring specific CS setup time, hold time and inactive
  * delay in terms of clock counts
  * @transfer: adds a message to the controller's transfer queue.
+ * @transfer_atomic: transfer a message immediately from a late-stage
+ *	synchronous transfer
  * @cleanup: frees controller-specific state
  * @can_dma: determine whether this controller supports DMA
  * @dma_map_dev: device which can be used for DMA mapping
@@ -719,6 +721,15 @@ struct spi_controller {
 	int			(*transfer)(struct spi_device *spi,
 						struct spi_message *mesg);
 
+	/*
+	 * Bidirectional bulk atomic transfers
+	 *
+	 * Special function that allows transfers in contexts where
+	 * no interrupts nor DMA are available.
+	 */
+	int			(*transfer_atomic)(struct spi_device *spi,
+						   struct spi_message *mesg);
+
 	/* Called on deregistration to free memory provided by spi_controller */
 	void			(*cleanup)(struct spi_device *spi);
 
-- 
2.47.3

[PATCH 2/2] spi: rockchip: add an atomic transfer function

[Heiko Stuebner]

From: Heiko Stuebner <heiko.stuebner@cherry.de>

Rockchip SPI controllers are often used to handle SPI connected PMICs that
also handle system power-off events, like the RK806.

When the kernel signals the power-off, all the interrupts are already
disabled, so regular SPI with its mutexes and DMA transfers does not
work anymore.

Implement an atomic transfer function that does SPI transfers polling the
controller in a loop without any DMA or IRQ.

This can successfully handle the power-off command to RK806, but also
read back the 2 chip-id registers just fine before that, as a test.

Additionally, setting ctrl->transfer to this function also let it work
just fine as the standard transfer function, so I'm hopefully not abusing
the SPI framework too much.

Signed-off-by: Heiko Stuebner <heiko.stuebner@cherry.de>
---
 drivers/spi/spi-rockchip.c | 88 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)

diff --git a/drivers/spi/spi-rockchip.c b/drivers/spi/spi-rockchip.c
index 1a81fd94b439..96582c7d5c03 100644
--- a/drivers/spi/spi-rockchip.c
+++ b/drivers/spi/spi-rockchip.c
@@ -5,6 +5,7 @@
  */
 
 #include <linux/clk.h>
+#include <linux/delay.h>
 #include <linux/dmaengine.h>
 #include <linux/interrupt.h>
 #include <linux/module.h>
@@ -13,6 +14,7 @@
 #include <linux/platform_device.h>
 #include <linux/spi/spi.h>
 #include <linux/pm_runtime.h>
+#include <linux/reboot.h>
 #include <linux/scatterlist.h>
 
 #define DRIVER_NAME "rockchip-spi"
@@ -724,6 +726,74 @@ static bool rockchip_spi_can_dma(struct spi_controller *ctlr,
 	return xfer->len / bytes_per_word >= rs->fifo_len;
 }
 
+static int rockchip_spi_transfer_atomic(struct spi_device *spi, struct spi_message *msg)
+{
+	struct spi_controller *ctlr = spi->controller;
+	struct rockchip_spi *rs = spi_controller_get_devdata(ctlr);
+	bool cs_en = (spi->mode & SPI_CS_HIGH) ? 1 : 0;
+	struct spi_transfer *xfer;
+	int ret = 0;
+
+	pm_runtime_get_sync(rs->dev);
+	rockchip_spi_set_cs(spi, cs_en);
+
+	/* make sure all interrupts are masked and status cleared */
+	writel_relaxed(0, rs->regs + ROCKCHIP_SPI_IMR);
+	writel_relaxed(0xffffffff, rs->regs + ROCKCHIP_SPI_ICR);
+
+	list_for_each_entry(xfer, &msg->transfers, transfer_list) {
+		WARN_ON(readl_relaxed(rs->regs + ROCKCHIP_SPI_SSIENR) &&
+			(readl_relaxed(rs->regs + ROCKCHIP_SPI_SR) & SR_BUSY));
+
+		rs->n_bytes = xfer->bits_per_word <= 8 ? 1 : 2;
+
+		ret = rockchip_spi_config(rs, spi, xfer, false, false);
+		if (ret < 0)
+			goto out_disable;
+
+		writel_relaxed(0, rs->regs + ROCKCHIP_SPI_RXFTLR);
+
+		rs->tx = xfer->tx_buf;
+		rs->rx = xfer->rx_buf;
+		rs->tx_left = rs->tx ? xfer->len / rs->n_bytes : 0;
+		rs->rx_left = xfer->len / rs->n_bytes;
+
+		spi_enable_chip(rs, true);
+
+		while (rs->tx_left || rs->rx_left) {
+			u32 status = readl_relaxed(rs->regs + ROCKCHIP_SPI_SR);
+
+			if (rs->tx_left && !(status & SR_TF_FULL))
+				rockchip_spi_pio_writer(rs);
+
+			if (rs->rx_left && !(status & SR_RF_EMPTY))
+				rockchip_spi_pio_reader(rs);
+		}
+
+		wait_for_tx_idle(rs, false);
+
+		spi_enable_chip(rs, false);
+
+		if (xfer->cs_change && !list_is_last(&xfer->transfer_list, &msg->transfers)) {
+			rockchip_spi_set_cs(spi, !cs_en);
+			ndelay(10 * NSEC_PER_USEC);
+			rockchip_spi_set_cs(spi, cs_en);
+		}
+	}
+
+out_disable:
+	rockchip_spi_set_cs(spi, !cs_en);
+	ndelay(10 * NSEC_PER_USEC);
+	msg->status = ret;
+
+	if (msg->complete)
+		msg->complete(msg->context);
+
+	pm_runtime_put(rs->dev);
+
+	return ret;
+}
+
 static int rockchip_spi_setup(struct spi_device *spi)
 {
 	struct rockchip_spi *rs = spi_controller_get_devdata(spi->controller);
@@ -752,6 +822,14 @@ static int rockchip_spi_setup(struct spi_device *spi)
 	return 0;
 }
 
+static int rockchip_spi_power_off(struct sys_off_data *data)
+{
+	struct rockchip_spi *rs = data->cb_data;
+
+	/* enable controller for possible calls from shutdown handlers */
+	return pm_runtime_get_sync(rs->dev);
+}
+
 static int rockchip_spi_probe(struct platform_device *pdev)
 {
 	struct device_node *np = pdev->dev.of_node;
@@ -832,6 +910,14 @@ static int rockchip_spi_probe(struct platform_device *pdev)
 	pm_runtime_set_active(&pdev->dev);
 	pm_runtime_enable(&pdev->dev);
 
+	ret = devm_register_sys_off_handler(&pdev->dev,
+					    SYS_OFF_MODE_POWER_OFF_PREPARE,
+					    SYS_OFF_PRIO_DEFAULT,
+					    &rockchip_spi_power_off, rs);
+	if (ret)
+		return dev_err_probe(&pdev->dev, ret,
+				     "failed to register poweroff handler\n");
+
 	ctlr->auto_runtime_pm = true;
 	ctlr->bus_num = pdev->id;
 	ctlr->mode_bits = SPI_CPOL | SPI_CPHA | SPI_LOOP | SPI_LSB_FIRST;
@@ -860,6 +946,8 @@ static int rockchip_spi_probe(struct platform_device *pdev)
 	ctlr->max_transfer_size = rockchip_spi_max_transfer_size;
 	ctlr->handle_err = rockchip_spi_handle_err;
 
+	ctlr->transfer_atomic = rockchip_spi_transfer_atomic;
+
 	ctlr->dma_tx = dma_request_chan(rs->dev, "tx");
 	if (IS_ERR(ctlr->dma_tx)) {
 		/* Check tx to see if we need to defer driver probing */
-- 
2.47.3

Re: [PATCH 1/2] spi: Add infrastructure for atomic sync transfers

[Mark Brown]

> There are cases where SPI transfers are needed during times without
> interrupts like shutdown when a PMIC should turn off the system.

> I2C already has atomic transfers for exactly that case, so introduce a
> similar concept for SPI. Similar to I2C, this is and should stay a
> corner-case, so is only allowed during the shutdown stages of the kernel,
> when interrupts are already turned off.

> So add a in-atomic check stolen from I2C and an optional transfer_atomic
> callback drivers can implement to do immediate uninterrupted transfers
> and let spi_sync() determine if the state is late enough to use it.

> spi_async() does not require a sleeping context for itself, so while it
> could in theory also be included, its called waiting for the completion
> can sleep, so would still cause the splat about it.

> +static int __spi_sync_atomic(struct spi_device *spi, struct spi_message *message)
> +{
> +	struct spi_controller *ctlr = spi->controller;
> +	unsigned long flags;
> +	int status;
> +
> +	if (!ctlr->transfer_atomic) {

We do have some existing users that appear to rely on using spi_sync()
to power off like SC27xx.  I suspect they are tolerating the splats, or
just don't work, but either way we should try to preserve the current
behaviour for them.

> +	status = spi_maybe_optimize_message(spi, message);

Message optimisation may try to do things like allocations that are a
bad idea in atomic context - it seems safer to just use the message as
is in this context.

-- 
Mark Brown <broonie@kernel.org>

Re: [PATCH 2/2] spi: rockchip: add an atomic transfer function

[Mark Brown]

> Rockchip SPI controllers are often used to handle SPI connected PMICs that
> also handle system power-off events, like the RK806.

> +static int rockchip_spi_transfer_atomic(struct spi_device *spi, struct spi_message *msg)
> +{
> +	struct spi_controller *ctlr = spi->controller;
> +	struct rockchip_spi *rs = spi_controller_get_devdata(ctlr);
> +	bool cs_en = (spi->mode & SPI_CS_HIGH) ? 1 : 0;
> +	struct spi_transfer *xfer;
> +	int ret = 0;
> +
> +	pm_runtime_get_sync(rs->dev);

The check in spi_in_atomic_sync_mode() is true for SYSTEM_RESTART and
SYSTEM_SUSPEND which would get rather messy if something ends up with a
device that tries to do something in there.  Not sure if that's worth
worrying about or not.

> +	rockchip_spi_set_cs(spi, cs_en);

This won't DTRT with GPIO chip selects, that's normally handled by the
core.

> +		while (rs->tx_left || rs->rx_left) {
> +			u32 status = readl_relaxed(rs->regs + ROCKCHIP_SPI_SR);
> +
> +			if (rs->tx_left && !(status & SR_TF_FULL))
> +				rockchip_spi_pio_writer(rs);
> +
> +			if (rs->rx_left && !(status & SR_RF_EMPTY))
> +				rockchip_spi_pio_reader(rs);
> +		}
> +
> +		wait_for_tx_idle(rs, false);

This uses jiffies which don't advance in poweroff, and there's no
overall timeout here.

> +		if (xfer->cs_change && !list_is_last(&xfer->transfer_list, &msg->transfers)) {
> +			rockchip_spi_set_cs(spi, !cs_en);
> +			ndelay(10 * NSEC_PER_USEC);

udelay().

> +static int rockchip_spi_power_off(struct sys_off_data *data)
> +{
> +	struct rockchip_spi *rs = data->cb_data;
> +
> +	/* enable controller for possible calls from shutdown handlers */
> +	return pm_runtime_get_sync(rs->dev);

Would it not be safer to just eat the error here rather than refuse
poweroff?  Also pm_runtime_resume_and_get() is the more modern thing.

> +	ctlr->transfer_atomic = rockchip_spi_transfer_atomic;

The driver supports target mode as well, probably best to only register
this for host mode.

-- 
Mark Brown <broonie@kernel.org>