* RE: [PATCH v9 2/2] i3c: master: Add driver for AMD AXI I3C master controller
From: Guntupalli, Manikanta @ 2026-06-24 6:06 UTC (permalink / raw)
To: Patil, Shubham Sanjay, git (AMD-Xilinx), Simek, Michal,
alexandre.belloni@bootlin.com, Frank.Li@nxp.com, robh@kernel.org,
krzk+dt@kernel.org, conor+dt@kernel.org, pgaj@cadence.com,
wsa+renesas@sang-engineering.com,
tommaso.merciai.xr@bp.renesas.com, arnd@arndb.de,
quic_msavaliy@quicinc.com, S-k, Shyam-sundar,
sakari.ailus@linux.intel.com, billy_tsai@aspeedtech.com,
kees@kernel.org, gustavoars@kernel.org,
jarkko.nikula@linux.intel.com, jorge.marques@analog.com,
linux-i3c@lists.infradead.org, devicetree@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org,
linux-hardening@vger.kernel.org
Cc: Pandey, Radhey Shyam, Goud, Srinivas, Datta, Shubhrajyoti,
Patil, Shubham Sanjay
In-Reply-To: <20260623114417.2578189-3-shubhamsanjay.patil@amd.com>
AMD General
Hi,
> -----Original Message-----
> From: Shubham Patil <shubhamsanjay.patil@amd.com>
> Sent: Tuesday, June 23, 2026 5:14 PM
> To: git (AMD-Xilinx) <git@amd.com>; Simek, Michal <michal.simek@amd.com>;
> alexandre.belloni@bootlin.com; Frank.Li@nxp.com; robh@kernel.org;
> krzk+dt@kernel.org; conor+dt@kernel.org; pgaj@cadence.com;
> wsa+renesas@sang-engineering.com; tommaso.merciai.xr@bp.renesas.com;
> arnd@arndb.de; quic_msavaliy@quicinc.com; S-k, Shyam-sundar <Shyam-
> sundar.S-k@amd.com>; sakari.ailus@linux.intel.com; billy_tsai@aspeedtech.com;
> kees@kernel.org; gustavoars@kernel.org; jarkko.nikula@linux.intel.com;
> jorge.marques@analog.com; linux-i3c@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org; linux-
> arch@vger.kernel.org; linux-hardening@vger.kernel.org
> Cc: Pandey, Radhey Shyam <radhey.shyam.pandey@amd.com>; Goud, Srinivas
> <srinivas.goud@amd.com>; Datta, Shubhrajyoti <shubhrajyoti.datta@amd.com>;
> Patil, Shubham Sanjay <ShubhamSanjay.Patil@amd.com>; Guntupalli, Manikanta
> <manikanta.guntupalli@amd.com>
> Subject: [PATCH v9 2/2] i3c: master: Add driver for AMD AXI I3C master controller
>
> From: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
>
> Add an I3C master driver and maintainers fragment for the AMD I3C bus controller.
>
> The driver currently supports the I3C bus operating in SDR mode, with features
> including Dynamic Address Assignment, private data transfers, and CCC transfers in
> both broadcast and direct modes. It also supports operation in I2C mode.
>
> The controller's data FIFOs are accessed big-endian; the driver performs this
> conversion locally using ioread32be()/iowrite32be() with the helpers, so it does not
> depend on any core FIFO-endianness helpers.
>
> Signed-off-by: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
> Co-developed-by: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
> Signed-off-by: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
> Co-developed-by: Shubham Patil <shubhamsanjay.patil@amd.com>
> Signed-off-by: Shubham Patil <shubhamsanjay.patil@amd.com>
> ---
> Changes for V9:
> Updated commit description to note that the driver performs big-endian FIFO
> accesses locally (the v8 core-helper patches were dropped).
> Dropped the big-endian MMIO infrastructure patches from the series
> ("asm-generic/io.h: Add big-endian MMIO accessors", "i3c: fix big-endian FIFO
> transfers", and "i3c: master: Add endianness support for
> i3c_readl_fifo()/i3c_writel_fifo()"). The driver now performs big-endian FIFO
> accesses locally using ioread32be()/iowrite32be() with
> get_unaligned()/put_unaligned(), so the series is self-contained and no longer
> includes internals.h.
> Replaced the async completion/transfer-queue machinery with a simple
> synchronous transfer path under the existing mutex.
> Reworked response handling: added enum i3c_error_code to struct xi3c_cmd,
> named the response codes, return -ENODEV/-EIO as appropriate and set err =
> I3C_ERROR_M2/M0 so the i3c core and callers can tell a NACK apart from a bus
> error; propagate err to CCC commands and to each priv xfer (including actual_len).
> Switched from .priv_xfers to the new .i3c_xfers op; reject non-SDR modes with -
> EOPNOTSUPP and report actual_len.
> Reworked DAA: assign addresses incrementally, bound the device count (-
> ENOSPC), detect end-of-enumeration via -ENODEV, zero-initialize the PID buffers,
> and check i3c_master_add_i3c_dev_locked().
> Avoid busy-spinning: sleep with usleep_range() in the FIFO drain/fill loops.
> Use FIELD_PREP() with named command-FIFO field masks instead of open-coded
> shifts, and convert the register-accessor macros to inline functions.
> Split the overloaded timeout macro into XI3C_RESP_TIMEOUT_US and
> XI3C_XFER_TIMEOUT_MS with documented units, and add
> XI3C_POLL_INTERVAL_US.
> xi3c_clk_cfg(): use NSEC_PER_SEC and named timing constants, guard against
> unsigned underflow, and handle I3C_BUS_MODE_MIXED_SLOW.
> Dropped ENTHDR from supports_ccc_cmd() (SDR-only), and dispatch CCCs using
> the I3C_CCC_DIRECT bit.
> Use const for TX buffers and drop the related casts; use parity8() for the DAA parity
> bit.
> Updated MODULE_DESCRIPTION and authors, the copyright year, renamed the
> Kconfig symbol to AMD_AXI_I3C_MASTER, and fixed the MAINTAINERS entry
> (title, mailing list, and the correct binding filename).
>
> Changes for V8:
> Used time_left instead of timeout.
> Used __free(kfree) for xfer to simplify err path in multiple places.
>
> Changes for V7:
> Updated timeout macro name.
> Updated xi3c_master_wr_to_tx_fifo() and xi3c_master_rd_from_rx_fifo() to use
> i3c_writel_fifo() and i3c_readl_fifo().
>
> Changes for V6:
> Removed typecast for xi3c_getrevisionnumber(), xi3c_wrfifolevel(), and
> xi3c_rdfifolevel().
> Replaced dynamic allocation with a static variable for pid_bcr_dcr.
> Fixed sparse warning in do_daa by typecasting the address parity value to u8.
> Fixed sparse warning in xi3c_master_bus_init by typecasting the pid value to u64 in
> info.pid calculation.
>
> Changes for V5:
> Used GENMASK_ULL for PID mask as it's 64bit mask.
>
> Changes for V4:
> Updated timeout macros.
> Removed type casting for xi3c_is_resp_available() macro.
> Used ioread32() and iowrite32() instead of readl() and writel() to keep consistency.
> Read XI3C_RESET_OFFSET reg before udelay().
> Removed xi3c_master_free_xfer() and directly used kfree().
> Skipped checking return value of i3c_master_add_i3c_dev_locked().
> Used devm_mutex_init() instead of mutex_init().
>
> Changes for V3:
> Resolved merge conflicts.
>
> Changes for V2:
> Updated commit description.
> Added mixed mode support with clock configuration.
> Converted smaller functions into inline functions.
> Used FIELD_GET() in xi3c_get_response().
> Updated xi3c_master_rd_from_rx_fifo() to use cmd->rx_buf.
> Used parity8() for address parity calculation.
> Added guards for locks.
> Dropped num_targets and updated xi3c_master_do_daa().
> Used __free(kfree) in xi3c_master_send_bdcast_ccc_cmd().
> Dropped PM runtime support.
> Updated xi3c_master_read() and xi3c_master_write() with
> xi3c_is_resp_available() check.
> Created separate functions: xi3c_master_init() and xi3c_master_reinit().
> Used xi3c_master_init() in bus initialization and xi3c_master_reinit() in error paths.
> Added DAA structure to xi3c_master structure.
> ---
> MAINTAINERS | 8 +
> drivers/i3c/master/Kconfig | 15 +
> drivers/i3c/master/Makefile | 1 +
> drivers/i3c/master/amd-i3c-master.c | 1060 +++++++++++++++++++++++++++
> 4 files changed, 1084 insertions(+)
> create mode 100644 drivers/i3c/master/amd-i3c-master.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 461a3eed6129..bfaa6999913c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1035,6 +1035,14 @@ L: linux-sound@vger.kernel.org
> S: Supported
> F: sound/soc/amd/
>
> +AMD AXI I3C MASTER DRIVER
> +M: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
> +M: Shubham Patil <shubhamsanjay.patil@amd.com>
> +L: linux-i3c@lists.infradead.org
> +S: Maintained
> +F: Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
> +F: drivers/i3c/master/amd-i3c-master.c
> +
> AMD AXI W1 DRIVER
> M: Kris Chaplin <kris.chaplin@amd.com>
> R: Thomas Delev <thomas.delev@amd.com>
> diff --git a/drivers/i3c/master/Kconfig b/drivers/i3c/master/Kconfig index
> 2609f2b18e0a..da96d2aaa399 100644
> --- a/drivers/i3c/master/Kconfig
> +++ b/drivers/i3c/master/Kconfig
> @@ -86,3 +86,18 @@ config RENESAS_I3C
>
> This driver can also be built as a module. If so, the module will be
> called renesas-i3c.
> +
> +config AMD_AXI_I3C_MASTER
> + tristate "AMD AXI I3C Master driver"
> + depends on HAS_IOMEM
> + help
> + Support for the AMD AXI I3C master controller, a soft IP used on
> + AMD (Xilinx) FPGAs and adaptive SoCs with ARM or MicroBlaze
> + processors.
> +
> + The controller currently supports Standard Data Rate (SDR) mode.
> + Features include Dynamic Address Assignment, private transfers,
> + and CCC transfers in both broadcast and direct modes.
> +
> + This driver can also be built as a module. If so, the module
> + will be called amd-i3c-master.
> diff --git a/drivers/i3c/master/Makefile b/drivers/i3c/master/Makefile index
> 816a227b6f7a..8d82196dcf83 100644
> --- a/drivers/i3c/master/Makefile
> +++ b/drivers/i3c/master/Makefile
> @@ -6,3 +6,4 @@ obj-$(CONFIG_AST2600_I3C_MASTER) += ast2600-i3c-
> master.o
> obj-$(CONFIG_SVC_I3C_MASTER) += svc-i3c-master.o
> obj-$(CONFIG_MIPI_I3C_HCI) += mipi-i3c-hci/
> obj-$(CONFIG_RENESAS_I3C) += renesas-i3c.o
> +obj-$(CONFIG_AMD_AXI_I3C_MASTER) += amd-i3c-master.o
> diff --git a/drivers/i3c/master/amd-i3c-master.c b/drivers/i3c/master/amd-i3c-master.c
> new file mode 100644
> index 000000000000..34ab1028c3ce
> --- /dev/null
> +++ b/drivers/i3c/master/amd-i3c-master.c
> @@ -0,0 +1,1060 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * I3C master driver for the AMD I3C controller.
> + *
> + * Copyright (C) 2026, Advanced Micro Devices, Inc.
> + */
> +
> +#include <linux/bitfield.h>
> +#include <linux/bitops.h>
> +#include <linux/cleanup.h>
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/err.h>
> +#include <linux/i3c/master.h>
> +#include <linux/io.h>
> +#include <linux/iopoll.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/mutex.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/time.h>
> +#include <linux/unaligned.h>
> +
> +#define XI3C_VERSION_OFFSET 0x00 /* Version Register */
> +#define XI3C_RESET_OFFSET 0x04 /* Soft Reset Register */
> +#define XI3C_CR_OFFSET 0x08 /* Control Register */
> +#define XI3C_ADDRESS_OFFSET 0x0C /* Target Address
> Register */
> +#define XI3C_SR_OFFSET 0x10 /* Status Register */
> +#define XI3C_CMD_FIFO_OFFSET 0x20 /* I3C Command
> FIFO Register */
> +#define XI3C_WR_FIFO_OFFSET 0x24 /* I3C Write Data FIFO
> Register */
> +#define XI3C_RD_FIFO_OFFSET 0x28 /* I3C Read Data FIFO
> Register */
> +#define XI3C_RESP_STATUS_FIFO_OFFSET 0x2C /* I3C Response
> status FIFO Register */
> +#define XI3C_FIFO_LVL_STATUS_OFFSET 0x30 /* CMD slots free
> | WR-FIFO free (words) */
> +#define XI3C_FIFO_LVL_STATUS_1_OFFSET 0x34 /* RESP fill | RD-
> FIFO fill level (words) */
> +#define XI3C_SCL_HIGH_TIME_OFFSET 0x38 /* I3C SCL HIGH
> Register */
> +#define XI3C_SCL_LOW_TIME_OFFSET 0x3C /* I3C SCL LOW
> Register */
> +#define XI3C_SDA_HOLD_TIME_OFFSET 0x40 /* I3C SDA
> HOLD Register */
> +#define XI3C_TSU_START_OFFSET 0x48 /* I3C START
> SETUP Register */
> +#define XI3C_THD_START_OFFSET 0x4C /* I3C START
> HOLD Register */
> +#define XI3C_TSU_STOP_OFFSET 0x50 /* I3C STOP
> Setup Register */
> +#define XI3C_OD_SCL_HIGH_TIME_OFFSET 0x54 /* I3C OD SCL
> HIGH Register */
> +#define XI3C_OD_SCL_LOW_TIME_OFFSET 0x58 /* I3C OD SCL
> LOW Register */
> +#define XI3C_PID0_OFFSET 0x6C /* LSB 4 bytes of the
> PID */
> +#define XI3C_PID1_BCR_DCR 0x70 /* MSB 2 bytes of the
> PID, BCR and DCR */
> +
> +#define XI3C_CR_EN_MASK BIT(0) /* Core Enable */
> +#define XI3C_CR_RESUME_MASK BIT(2) /* Core Resume
> */
> +#define XI3C_SR_RESP_NOT_EMPTY_MASK BIT(4) /* Resp Fifo not
> empty status mask */
> +#define XI3C_RD_FIFO_NOT_EMPTY_MASK BIT(15) /* Read
> Fifo not empty status mask */
> +
> +#define XI3C_BCR_MASK GENMASK(23, 16)
> +#define XI3C_DCR_MASK GENMASK(31, 24)
> +#define XI3C_PID_MASK GENMASK_ULL(63, 16)
> +#define XI3C_TIMING_MASK GENMASK(17, 0)
> +#define XI3C_REV_NUM_MASK GENMASK(15, 8)
> +#define XI3C_PID1_MASK GENMASK(15, 0)
> +#define XI3C_FIFO_LEVEL_MASK GENMASK(15, 0)
> +#define XI3C_RESP_CODE_MASK GENMASK(8, 5)
> +#define XI3C_RESP_CODE_SUCCESS 0 /* Transfer
> completed OK */
> +#define XI3C_RESP_CODE_NO_TARGET 2 /* 7E NACK: no
> target on bus */
> +#define XI3C_RESP_CODE_NACK 3 /* Target NACK /
> CE2 / DAA end */
> +#define XI3C_ADDR_MASK GENMASK(6, 0)
> +#define XI3C_FIFOS_RST_MASK GENMASK(4, 1)
> +
> +/* Command FIFO word layout (bit ranges encoded in the GENMASK/BIT args) */
> +#define XI3C_CMD_TYPE GENMASK(3, 0) /*
> command type */
> +#define XI3C_CMD_TERMINATE BIT(4) /* terminate (last
> cmd of xfer) */
> +#define XI3C_CMD_ADDR GENMASK(15, 8) /* target
> address << 1 | RnW */
> +#define XI3C_CMD_LEN GENMASK(27, 16) /*
> payload length in bytes */
> +#define XI3C_CMD_TID GENMASK(31, 28) /* transfer
> ID */
> +
> +#define XI3C_OD_TLOW_NS 500000
> +#define XI3C_OD_THIGH_NS 41000
> +#define XI3C_I2C_TCASMIN_NS 600000
> +#define XI3C_TCASMIN_NS 260000
> +#define XI3C_MAXDATA_LENGTH 4095
> +#define XI3C_MAX_DEVS 32
> +#define XI3C_DAA_SLAVEINFO_READ_BYTECOUNT 8
> +
> +#define XI3C_THOLD_MIN_REV0 5 /* Min SDA hold cycles,
> rev 0 IP */
> +#define XI3C_THOLD_MIN_REV1 6 /* Min SDA hold cycles,
> rev >= 1 IP */
> +#define XI3C_CYCLE_ADJUST 2 /* SCL/SDA pre-bias for
> HW pipeline */
> +#define XI3C_FIFO_RESET_DELAY_US 10 /* HW settling time after
> FIFO reset */
> +#define XI3C_POLL_INTERVAL_US 10 /*
> readl_poll_timeout() sleep slice */
> +
> +#define XI3C_I2C_MODE 0
> +#define XI3C_I2C_TID 0
> +#define XI3C_SDR_MODE 1
> +#define XI3C_SDR_TID 1
> +
> +#define XI3C_WORD_LEN 4
> +
> +/*
> + * XI3C_RESP_TIMEOUT_US is in microseconds because it is passed as the
> + * timeout_us argument of readl_poll_timeout(). XI3C_XFER_TIMEOUT_MS is
> +in
> + * milliseconds because it feeds msecs_to_jiffies(). Keep the two units
> + * distinct in the names so callers cannot mix them up.
> + */
> +#define XI3C_RESP_TIMEOUT_US 500000
> +#define XI3C_XFER_TIMEOUT_MS 1000
> +
> +struct xi3c_cmd {
> + const void *tx_buf;
> + void *rx_buf;
> + u16 tx_len;
> + u16 rx_len;
> + u8 addr;
> + u8 type;
> + u8 tid;
> + bool rnw;
> + bool is_daa;
> + bool continued;
> + enum i3c_error_code err;
> +};
> +
> +struct xi3c_xfer {
> + unsigned int ncmds;
> + struct xi3c_cmd cmds[] __counted_by(ncmds); };
> +
> +/**
> + * struct xi3c_master - I3C master controller state.
> + * @base: I3C master controller embedded by the framework.
> + * @dev: Pointer to the backing device structure.
> + * @membase: Memory base of the HW registers.
> + * @pclk: Input clock driving the controller.
> + * @lock: Serializes transfers and CCC submission.
> + * @daa: ENTDAA enumeration state.
> + * @daa.addrs: Dynamic addresses assigned in enumeration order.
> + * @daa.index: Number of responders enumerated so far.
> + */
> +struct xi3c_master {
> + struct i3c_master_controller base;
> + struct device *dev;
> + void __iomem *membase;
> + struct clk *pclk;
> + struct mutex lock; /* serializes transfers and CCC submission */
> + struct {
> + u8 addrs[XI3C_MAX_DEVS];
> + u8 index;
> + } daa;
> +};
> +
> +static inline struct xi3c_master *
> +to_xi3c_master(struct i3c_master_controller *master) {
> + return container_of(master, struct xi3c_master, base); }
> +
> +static inline u8 xi3c_get_revision_number(struct xi3c_master *master) {
> + return FIELD_GET(XI3C_REV_NUM_MASK,
> + ioread32(master->membase + XI3C_VERSION_OFFSET)); }
> +
> +static inline u16 xi3c_wr_fifo_level(struct xi3c_master *master) {
> + return ioread32(master->membase + XI3C_FIFO_LVL_STATUS_OFFSET) &
> + XI3C_FIFO_LEVEL_MASK;
> +}
> +
> +static inline u16 xi3c_rd_fifo_level(struct xi3c_master *master) {
> + return ioread32(master->membase +
> XI3C_FIFO_LVL_STATUS_1_OFFSET) &
> + XI3C_FIFO_LEVEL_MASK;
> +}
> +
> +static inline bool xi3c_is_resp_available(struct xi3c_master *master) {
> + return FIELD_GET(XI3C_SR_RESP_NOT_EMPTY_MASK,
> + ioread32(master->membase + XI3C_SR_OFFSET)); }
> +
> +static int xi3c_get_response(struct xi3c_master *master, struct
> +xi3c_cmd *cmd) {
> + u32 response_data;
> + u32 resp_reg;
> + u8 code;
> + int ret;
> +
> + ret = readl_poll_timeout(master->membase + XI3C_SR_OFFSET,
> + resp_reg,
> + resp_reg & XI3C_SR_RESP_NOT_EMPTY_MASK,
> + XI3C_POLL_INTERVAL_US,
> XI3C_RESP_TIMEOUT_US);
> + if (ret) {
> + dev_err(master->dev, "XI3C response timeout\n");
> + return ret;
> + }
> +
> + response_data = ioread32(master->membase +
> XI3C_RESP_STATUS_FIFO_OFFSET);
> + code = FIELD_GET(XI3C_RESP_CODE_MASK, response_data);
> +
> + switch (code) {
> + case XI3C_RESP_CODE_SUCCESS:
> + cmd->err = I3C_ERROR_UNKNOWN;
> + return 0;
> + case XI3C_RESP_CODE_NO_TARGET:
> + case XI3C_RESP_CODE_NACK:
> + /*
> + * Target did not ACK. Record it as I3C_ERROR_M2 so callers
> + * (and the i3c core, which keys on err == I3C_ERROR_M2) can
> + * tell a NACK apart from other failures. A normal transfer
> + * surfaces this as -EIO per the i3c_xfer contract; the DAA
> + * path instead expects -ENODEV as its enumeration terminator.
> + */
> + cmd->err = I3C_ERROR_M2;
> + return cmd->is_daa ? -ENODEV : -EIO;
> + default:
> + cmd->err = I3C_ERROR_M0;
> + dev_err(master->dev, "XI3C transfer error, response code %u\n",
> + code);
> + return -EIO;
> + }
> +}
> +
> +static inline void xi3c_writesl_be(void __iomem *addr, const void *buffer,
> + unsigned int count)
> +{
> + const u32 *buf = buffer;
> +
> + while (count--)
> + iowrite32be(get_unaligned(buf++), addr); }
> +
> +static inline void xi3c_readsl_be(const void __iomem *addr, void *buffer,
> + unsigned int count)
> +{
> + u32 *buf = buffer;
> +
> + while (count--)
> + put_unaligned(ioread32be(addr), buf++); }
> +
> +static inline void xi3c_writel_fifo(void __iomem *addr, const void *buf,
> + int nbytes)
> +{
> + xi3c_writesl_be(addr, buf, nbytes / 4);
> + if (nbytes & 3) {
> + u32 tmp = 0;
> +
> + memcpy(&tmp, (const u8 *)buf + (nbytes & ~3), nbytes & 3);
> + xi3c_writesl_be(addr, &tmp, 1);
> + }
> +}
> +
> +static inline void xi3c_readl_fifo(const void __iomem *addr, void *buf,
> + int nbytes)
> +{
> + xi3c_readsl_be(addr, buf, nbytes / 4);
> + if (nbytes & 3) {
> + u32 tmp;
> +
> + xi3c_readsl_be(addr, &tmp, 1);
> + memcpy((u8 *)buf + (nbytes & ~3), &tmp, nbytes & 3);
> + }
> +}
> +
> +static void xi3c_master_write_to_cmdfifo(struct xi3c_master *master,
> + struct xi3c_cmd *cmd, u16 len)
> +{
> + u32 transfer_cmd;
> + u8 addr;
> +
> + addr = ((cmd->addr & XI3C_ADDR_MASK) << 1) | (u8)cmd->rnw;
> +
> + transfer_cmd = FIELD_PREP(XI3C_CMD_TYPE, cmd->type);
> + transfer_cmd |= FIELD_PREP(XI3C_CMD_TERMINATE, !cmd->continued);
> + transfer_cmd |= FIELD_PREP(XI3C_CMD_ADDR, addr);
> + transfer_cmd |= FIELD_PREP(XI3C_CMD_TID, cmd->tid);
> +
> + /*
> + * For dynamic addressing, an additional 1-byte length must be added
> + * to the command FIFO to account for the address present in the TX FIFO
> + */
> + if (cmd->is_daa) {
> + xi3c_writel_fifo(master->membase + XI3C_WR_FIFO_OFFSET,
> + cmd->tx_buf, cmd->tx_len);
> +
> + len++;
> + }
> +
> + transfer_cmd |= FIELD_PREP(XI3C_CMD_LEN, len);
> + iowrite32(transfer_cmd, master->membase + XI3C_CMD_FIFO_OFFSET); }
> +
> +static inline void xi3c_master_enable(struct xi3c_master *master) {
> + iowrite32(ioread32(master->membase + XI3C_CR_OFFSET) |
> XI3C_CR_EN_MASK,
> + master->membase + XI3C_CR_OFFSET);
> +}
> +
> +static inline void xi3c_master_disable(struct xi3c_master *master) {
> + iowrite32(ioread32(master->membase + XI3C_CR_OFFSET) &
> ~XI3C_CR_EN_MASK,
> + master->membase + XI3C_CR_OFFSET);
> +}
> +
> +static inline void xi3c_master_resume(struct xi3c_master *master) {
> + iowrite32(ioread32(master->membase + XI3C_CR_OFFSET) |
> + XI3C_CR_RESUME_MASK, master->membase +
> XI3C_CR_OFFSET); }
> +
> +static void xi3c_master_reset_fifos(struct xi3c_master *master) {
> + u32 data;
> +
> + /* Assert FIFO reset. */
> + data = ioread32(master->membase + XI3C_RESET_OFFSET);
> + data |= XI3C_FIFOS_RST_MASK;
> + iowrite32(data, master->membase + XI3C_RESET_OFFSET);
> + /* Read-back flushes the posted write before the settling delay below. */
> + ioread32(master->membase + XI3C_RESET_OFFSET);
> + udelay(XI3C_FIFO_RESET_DELAY_US);
> +
> + /* De-assert FIFO reset, then wait for the FIFOs to come back up. */
> + data &= ~XI3C_FIFOS_RST_MASK;
> + iowrite32(data, master->membase + XI3C_RESET_OFFSET);
> + ioread32(master->membase + XI3C_RESET_OFFSET);
> + udelay(XI3C_FIFO_RESET_DELAY_US);
> +}
> +
> +static inline void xi3c_master_init(struct xi3c_master *master) {
> + /* Reset fifos */
> + xi3c_master_reset_fifos(master);
> +
> + /* Enable controller */
> + xi3c_master_enable(master);
> +}
> +
> +static inline void xi3c_master_reinit(struct xi3c_master *master) {
> + /* Reset fifos */
> + xi3c_master_reset_fifos(master);
> +
> + /* Resume controller */
> + xi3c_master_resume(master);
> +}
> +
> +static struct xi3c_xfer *xi3c_master_alloc_xfer(unsigned int ncmds) {
> + struct xi3c_xfer *xfer;
> +
> + xfer = kzalloc(struct_size(xfer, cmds, ncmds), GFP_KERNEL);
> + if (!xfer)
> + return NULL;
> +
> + xfer->ncmds = ncmds;
> +
> + return xfer;
> +}
> +
> +static void xi3c_master_rd_from_rx_fifo(struct xi3c_master *master,
> + struct xi3c_cmd *cmd)
> +{
> + u16 rx_data_available;
> + u16 copy_len;
> + u16 len;
> +
> + rx_data_available = xi3c_rd_fifo_level(master);
> + len = rx_data_available * XI3C_WORD_LEN;
> +
> + if (!len)
> + return;
> +
> + copy_len = min_t(u16, len, cmd->rx_len);
> + xi3c_readl_fifo(master->membase + XI3C_RD_FIFO_OFFSET,
> + (u8 *)cmd->rx_buf, copy_len);
> +
> + cmd->rx_buf = (u8 *)cmd->rx_buf + copy_len;
> + cmd->rx_len -= copy_len;
> +}
> +
> +static int xi3c_master_read(struct xi3c_master *master, struct xi3c_cmd
> +*cmd) {
> + unsigned long timeout;
> + u32 status_reg;
> + int ret;
> +
> + if (!cmd->rx_buf || cmd->rx_len > XI3C_MAXDATA_LENGTH)
> + return -EINVAL;
> +
> + /* Fill command fifo */
> + xi3c_master_write_to_cmdfifo(master, cmd, cmd->rx_len);
> +
> + if (!cmd->rx_len)
> + return 0;
> +
> + ret = readl_poll_timeout(master->membase + XI3C_SR_OFFSET,
> + status_reg,
> + status_reg & (XI3C_RD_FIFO_NOT_EMPTY_MASK
> |
> + XI3C_SR_RESP_NOT_EMPTY_MASK),
> + XI3C_POLL_INTERVAL_US,
> XI3C_RESP_TIMEOUT_US);
> + if (ret) {
> + dev_err(master->dev, "XI3C read timeout\n");
> + return ret;
> + }
> +
> + if (!(status_reg & XI3C_RD_FIFO_NOT_EMPTY_MASK))
> + return 0;
> +
> + timeout = jiffies + msecs_to_jiffies(XI3C_XFER_TIMEOUT_MS);
> +
> + /* Read data from rx fifo */
> + while (cmd->rx_len > 0 && !xi3c_is_resp_available(master)) {
> + if (time_after(jiffies, timeout)) {
> + dev_err(master->dev, "XI3C read timeout\n");
> + return -EIO;
> + }
> + xi3c_master_rd_from_rx_fifo(master, cmd);
> + usleep_range(XI3C_POLL_INTERVAL_US, 2 *
> XI3C_POLL_INTERVAL_US);
> + }
> +
> + /* Read remaining data */
> + xi3c_master_rd_from_rx_fifo(master, cmd);
> +
> + return 0;
> +}
> +
> +static void xi3c_master_wr_to_tx_fifo(struct xi3c_master *master,
> + struct xi3c_cmd *cmd)
> +{
> + u16 wrfifo_space;
> + u16 len;
> +
> + wrfifo_space = xi3c_wr_fifo_level(master);
> + if (cmd->tx_len > wrfifo_space * XI3C_WORD_LEN)
> + len = wrfifo_space * XI3C_WORD_LEN;
> + else
> + len = cmd->tx_len;
> +
> + if (len) {
> + xi3c_writel_fifo(master->membase + XI3C_WR_FIFO_OFFSET,
> cmd->tx_buf,
> + len);
> +
> + cmd->tx_buf = (const u8 *)cmd->tx_buf + len;
> + cmd->tx_len -= len;
> + }
> +}
> +
> +static int xi3c_master_write(struct xi3c_master *master, struct
> +xi3c_cmd *cmd) {
> + unsigned long timeout;
> + u16 cmd_len;
> +
> + if (!cmd->tx_buf || cmd->tx_len > XI3C_MAXDATA_LENGTH)
> + return -EINVAL;
> +
> + cmd_len = cmd->tx_len;
> +
> + /* Fill Tx fifo */
> + xi3c_master_wr_to_tx_fifo(master, cmd);
> +
> + /* Write to command fifo */
> + xi3c_master_write_to_cmdfifo(master, cmd, cmd_len);
> +
> + timeout = jiffies + msecs_to_jiffies(XI3C_XFER_TIMEOUT_MS);
> + /* Fill if any remaining data to tx fifo */
> + while (cmd->tx_len > 0 && !xi3c_is_resp_available(master)) {
> + if (time_after(jiffies, timeout)) {
> + dev_err(master->dev, "XI3C write timeout\n");
> + return -EIO;
> + }
> +
> + xi3c_master_wr_to_tx_fifo(master, cmd);
> + usleep_range(XI3C_POLL_INTERVAL_US, 2 *
> XI3C_POLL_INTERVAL_US);
> + }
> +
> + return 0;
> +}
> +
> +static int xi3c_master_xfer(struct xi3c_master *master, struct xi3c_cmd
> +*cmd) {
> + int ret;
> +
> + if (cmd->rnw)
> + ret = xi3c_master_read(master, cmd);
> + else
> + ret = xi3c_master_write(master, cmd);
> +
> + if (ret)
> + goto err_xfer_out;
> +
> + ret = xi3c_get_response(master, cmd);
> + if (ret)
> + goto err_xfer_out;
> +
> + return 0;
> +
> +err_xfer_out:
> + xi3c_master_reinit(master);
> + return ret;
> +}
> +
> +static int xi3c_master_common_xfer(struct xi3c_master *master,
> + struct xi3c_xfer *xfer)
> +{
> + unsigned int i;
> + int ret;
> +
> + guard(mutex)(&master->lock);
> +
> + for (i = 0; i < xfer->ncmds; i++) {
> + ret = xi3c_master_xfer(master, &xfer->cmds[i]);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static int xi3c_master_do_daa(struct i3c_master_controller *m) {
> + u8
> pid_bufs[XI3C_MAX_DEVS][XI3C_DAA_SLAVEINFO_READ_BYTECOUNT] = {};
> + struct xi3c_master *master = to_xi3c_master(m);
> + struct xi3c_xfer *xfer __free(kfree) = NULL;
> + struct xi3c_cmd *daa_cmd;
> + int addr, ret, i;
> + u8 last_addr = 0;
> + u8 *pid_buf;
> + u8 ccc_id;
> +
> + xfer = xi3c_master_alloc_xfer(1);
> + if (!xfer)
> + return -ENOMEM;
> +
> + /* Fill ENTDAA CCC */
> + ccc_id = I3C_CCC_ENTDAA;
> + daa_cmd = &xfer->cmds[0];
> + daa_cmd->addr = I3C_BROADCAST_ADDR;
> + daa_cmd->rnw = false;
> + daa_cmd->tx_buf = &ccc_id;
> + daa_cmd->tx_len = 1;
> + daa_cmd->type = XI3C_SDR_MODE;
> + daa_cmd->tid = XI3C_SDR_TID;
> + daa_cmd->continued = true;
> +
> + ret = xi3c_master_common_xfer(master, xfer);
> + /*
> + * A NACK on the ENTDAA broadcast (I3C_ERROR_M2) means no slaves
> are
> + * present to enter DAA. Treat as a successful no-op after letting
> + * err_daa reinitialize the controller.
> + */
> + if (ret && daa_cmd->err == I3C_ERROR_M2) {
> + ret = 0;
> + goto err_daa;
> + }
> + if (ret)
> + goto err_daa;
> +
> + master->daa.index = 0;
> +
> + while (true) {
> + struct xi3c_cmd *cmd = &xfer->cmds[0];
> + u8 daa_byte;
> +
> + if (master->daa.index >= XI3C_MAX_DEVS) {
> + ret = -ENOSPC;
> + goto err_daa;
> + }
> +
> + addr = i3c_master_get_free_addr(m, last_addr + 1);
> + if (addr < 0) {
> + ret = addr;
> + goto err_daa;
> + }
> +
> + pid_buf = pid_bufs[master->daa.index];
> +
> + daa_byte = (addr << 1) | (parity8(addr) ^ 1);
> +
> + cmd->tx_buf = &daa_byte;
> + cmd->tx_len = 1;
> + cmd->addr = I3C_BROADCAST_ADDR;
> + cmd->rnw = true;
> + cmd->rx_buf = pid_buf;
> + cmd->rx_len = XI3C_DAA_SLAVEINFO_READ_BYTECOUNT;
> + cmd->is_daa = true;
> + cmd->type = XI3C_SDR_MODE;
> + cmd->tid = XI3C_SDR_TID;
> + cmd->continued = true;
> +
> + ret = xi3c_master_common_xfer(master, xfer);
> +
> + /*
> + * End of enumeration: the next responder NACK'd the
> + * dynamic-address grant, surfaced as -ENODEV.
> + * xi3c_master_xfer() has already reset the FIFOs and
> + * resumed the core for us; just exit the loop and
> + * register the responders collected so far.
> + */
> + if (ret == -ENODEV) {
> + ret = 0;
> + break;
> + }
> + if (ret)
> + goto err_daa;
> +
> + master->daa.addrs[master->daa.index] = addr;
> + last_addr = addr;
> + master->daa.index++;
> + }
> +
> + for (i = 0; i < master->daa.index; i++) {
> + u64 pid;
> +
> + ret = i3c_master_add_i3c_dev_locked(m, master->daa.addrs[i]);
> + if (ret)
> + goto err_daa;
> +
> + pid = FIELD_GET(XI3C_PID_MASK,
> + get_unaligned_be64(pid_bufs[i]));
> + dev_dbg(master->dev, "Client %d: PID: 0x%llx\n", i, pid);
> + }
> +
> + return 0;
> +
> +err_daa:
> + xi3c_master_reinit(master);
> + return ret;
> +}
> +
> +static bool
> +xi3c_master_supports_ccc_cmd(struct i3c_master_controller *master,
> + const struct i3c_ccc_cmd *cmd)
> +{
> + if (cmd->ndests > 1)
> + return false;
> +
> + switch (cmd->id) {
> + case I3C_CCC_ENEC(true):
> + case I3C_CCC_ENEC(false):
> + case I3C_CCC_DISEC(true):
> + case I3C_CCC_DISEC(false):
> + case I3C_CCC_ENTAS(0, true):
> + case I3C_CCC_ENTAS(0, false):
> + case I3C_CCC_RSTDAA(true):
> + case I3C_CCC_RSTDAA(false):
> + case I3C_CCC_ENTDAA:
> + case I3C_CCC_SETMWL(true):
> + case I3C_CCC_SETMWL(false):
> + case I3C_CCC_SETMRL(true):
> + case I3C_CCC_SETMRL(false):
> + case I3C_CCC_SETDASA:
> + case I3C_CCC_SETNEWDA:
> + case I3C_CCC_GETMWL:
> + case I3C_CCC_GETMRL:
> + case I3C_CCC_GETPID:
> + case I3C_CCC_GETBCR:
> + case I3C_CCC_GETDCR:
> + case I3C_CCC_GETSTATUS:
> + case I3C_CCC_GETMXDS:
> + return true;
> + default:
> + return false;
> + }
> +}
> +
> +static int xi3c_master_send_bdcast_ccc_cmd(struct xi3c_master *master,
> + struct i3c_ccc_cmd *ccc)
> +{
> + struct xi3c_xfer *xfer __free(kfree) = NULL;
> + u8 *buf __free(kfree) = NULL;
> + struct xi3c_cmd *cmd;
> + u16 xfer_len;
> + int ret;
> +
> + if (ccc->dests[0].payload.len >= XI3C_MAXDATA_LENGTH)
> + return -EINVAL;
> +
> + xfer_len = ccc->dests[0].payload.len + 1;
> +
> + xfer = xi3c_master_alloc_xfer(1);
> + if (!xfer)
> + return -ENOMEM;
> +
> + buf = kmalloc(xfer_len, GFP_KERNEL);
> + if (!buf)
> + return -ENOMEM;
> +
> + buf[0] = ccc->id;
> + memcpy(&buf[1], ccc->dests[0].payload.data,
> +ccc->dests[0].payload.len);
> +
> + cmd = &xfer->cmds[0];
> + cmd->addr = ccc->dests[0].addr;
> + cmd->rnw = ccc->rnw;
> + cmd->tx_buf = buf;
> + cmd->tx_len = xfer_len;
> + cmd->type = XI3C_SDR_MODE;
> + cmd->tid = XI3C_SDR_TID;
> + cmd->continued = false;
> +
> + ret = xi3c_master_common_xfer(master, xfer);
> + ccc->err = cmd->err;
> +
> + return ret;
> +}
> +
> +static int xi3c_master_send_direct_ccc_cmd(struct xi3c_master *master,
> + struct i3c_ccc_cmd *ccc)
> +{
> + struct xi3c_xfer *xfer __free(kfree) = NULL;
> + struct xi3c_cmd *cmd;
> + int ret;
> +
> + if (ccc->dests[0].payload.len > XI3C_MAXDATA_LENGTH)
> + return -EINVAL;
> +
> + xfer = xi3c_master_alloc_xfer(2);
> + if (!xfer)
> + return -ENOMEM;
> +
> + /* Broadcasted message */
> + cmd = &xfer->cmds[0];
> + cmd->addr = I3C_BROADCAST_ADDR;
> + cmd->rnw = false;
> + cmd->tx_buf = &ccc->id;
> + cmd->tx_len = 1;
> + cmd->type = XI3C_SDR_MODE;
> + cmd->tid = XI3C_SDR_TID;
> + cmd->continued = true;
> +
> + /* Directed message */
> + cmd = &xfer->cmds[1];
> + cmd->addr = ccc->dests[0].addr;
> + cmd->rnw = ccc->rnw;
> + if (cmd->rnw) {
> + cmd->rx_buf = ccc->dests[0].payload.data;
> + cmd->rx_len = ccc->dests[0].payload.len;
> + } else {
> + cmd->tx_buf = ccc->dests[0].payload.data;
> + cmd->tx_len = ccc->dests[0].payload.len;
> + }
> + cmd->type = XI3C_SDR_MODE;
> + cmd->tid = XI3C_SDR_TID;
> + cmd->continued = false;
> +
> + ret = xi3c_master_common_xfer(master, xfer);
> +
> + /*
> + * Report the broadcast command's error if it failed, otherwise the
> + * directed command's, so a NACK on either phase reaches the caller.
> + */
> + ccc->err = xfer->cmds[0].err ? xfer->cmds[0].err : xfer->cmds[1].err;
> +
> + return ret;
> +}
> +
> +static int xi3c_master_send_ccc_cmd(struct i3c_master_controller *m,
> + struct i3c_ccc_cmd *cmd)
> +{
> + struct xi3c_master *master = to_xi3c_master(m);
> +
> + if (cmd->id & I3C_CCC_DIRECT)
> + return xi3c_master_send_direct_ccc_cmd(master, cmd);
> +
> + return xi3c_master_send_bdcast_ccc_cmd(master, cmd); }
> +
> +static int xi3c_master_i3c_xfers(struct i3c_dev_desc *dev,
> + struct i3c_xfer *xfers,
> + int nxfers, enum i3c_xfer_mode mode) {
> + struct i3c_master_controller *m = i3c_dev_get_master(dev);
> + struct xi3c_master *master = to_xi3c_master(m);
> + struct xi3c_xfer *xfer __free(kfree) = NULL;
> + int i, ret;
> +
> + if (!nxfers)
> + return 0;
> +
> + if (mode != I3C_SDR)
> + return -EOPNOTSUPP;
> +
> + for (i = 0; i < nxfers; i++)
> + if (xfers[i].len > XI3C_MAXDATA_LENGTH)
> + return -EINVAL;
> +
> + xfer = xi3c_master_alloc_xfer(nxfers);
> + if (!xfer)
> + return -ENOMEM;
> +
> + for (i = 0; i < nxfers; i++) {
> + struct xi3c_cmd *cmd = &xfer->cmds[i];
> +
> + cmd->addr = dev->info.dyn_addr;
> + cmd->rnw = xfers[i].rnw;
> +
> + if (cmd->rnw) {
> + cmd->rx_buf = xfers[i].data.in;
> + cmd->rx_len = xfers[i].len;
> + } else {
> + cmd->tx_buf = xfers[i].data.out;
> + cmd->tx_len = xfers[i].len;
> + }
> +
> + cmd->type = XI3C_SDR_MODE;
> + cmd->tid = XI3C_SDR_TID;
> + cmd->continued = (i + 1) < nxfers;
> + }
> +
> + ret = xi3c_master_common_xfer(master, xfer);
> +
> + for (i = 0; i < nxfers; i++) {
> + xfers[i].err = xfer->cmds[i].err;
> + if (xfers[i].rnw)
> + xfers[i].actual_len = xfers[i].len - xfer->cmds[i].rx_len;
> + }
> +
> + return ret;
> +}
> +
> +static int xi3c_master_i2c_xfers(struct i2c_dev_desc *dev,
> + struct i2c_msg *xfers,
> + int nxfers)
> +{
> + struct i3c_master_controller *m = i2c_dev_get_master(dev);
> + struct xi3c_master *master = to_xi3c_master(m);
> + struct xi3c_xfer *xfer __free(kfree) = NULL;
> + int i;
> +
> + if (!nxfers)
> + return 0;
> +
> + for (i = 0; i < nxfers; i++)
> + if (xfers[i].len > XI3C_MAXDATA_LENGTH)
> + return -EINVAL;
> +
> + xfer = xi3c_master_alloc_xfer(nxfers);
> + if (!xfer)
> + return -ENOMEM;
> +
> + for (i = 0; i < nxfers; i++) {
> + struct xi3c_cmd *cmd = &xfer->cmds[i];
> +
> + cmd->addr = xfers[i].addr & XI3C_ADDR_MASK;
> + cmd->rnw = !!(xfers[i].flags & I2C_M_RD);
> +
> + if (cmd->rnw) {
> + cmd->rx_buf = xfers[i].buf;
> + cmd->rx_len = xfers[i].len;
> + } else {
> + cmd->tx_buf = xfers[i].buf;
> + cmd->tx_len = xfers[i].len;
> + }
> +
> + cmd->type = XI3C_I2C_MODE;
> + cmd->tid = XI3C_I2C_TID;
> + cmd->continued = (i + 1) < nxfers;
> + }
> +
> + return xi3c_master_common_xfer(master, xfer); }
> +
> +static int xi3c_clk_cfg(struct xi3c_master *master, unsigned long
> +sclhz, u8 mode) {
> + unsigned long core_rate, core_periodns;
> + u32 tcasmin, tsustart, tsustop, thdstart;
> + u32 thigh, tlow, thold;
> + u32 odthigh, odtlow;
> +
> + core_rate = clk_get_rate(master->pclk);
> + if (!core_rate)
> + return -EINVAL;
> +
> + if (!sclhz)
> + return -EINVAL;
> +
> + core_periodns = DIV_ROUND_UP(NSEC_PER_SEC, core_rate);
> +
> + thigh = DIV_ROUND_UP(core_rate, sclhz) >> 1;
> + tlow = thigh;
> +
> + if (thigh <= XI3C_CYCLE_ADJUST)
> + return -EINVAL;
> +
> + /* Hold time : 40% of tlow time */
> + thold = (tlow * 4) / 10;
> +
> + if (xi3c_get_revision_number(master) == 0)
> + thold = max_t(u32, thold, XI3C_THOLD_MIN_REV0);
> + else
> + thold = max_t(u32, thold, XI3C_THOLD_MIN_REV1);
> +
> + iowrite32((thigh - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase + XI3C_SCL_HIGH_TIME_OFFSET);
> + iowrite32((tlow - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase + XI3C_SCL_LOW_TIME_OFFSET);
> + iowrite32((thold - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase + XI3C_SDA_HOLD_TIME_OFFSET);
> +
> + if (mode == XI3C_I2C_MODE) {
> + iowrite32((thigh - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase +
> XI3C_OD_SCL_HIGH_TIME_OFFSET);
> + iowrite32((tlow - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase +
> XI3C_OD_SCL_LOW_TIME_OFFSET);
> +
> + tcasmin = DIV_ROUND_UP(XI3C_I2C_TCASMIN_NS,
> core_periodns);
> + } else {
> + odtlow = DIV_ROUND_UP(XI3C_OD_TLOW_NS, core_periodns);
> + odthigh = DIV_ROUND_UP(XI3C_OD_THIGH_NS, core_periodns);
> +
> + odtlow = max(tlow, odtlow);
> + odthigh = min(thigh, odthigh);
> +
> + if (odthigh <= XI3C_CYCLE_ADJUST)
> + return -EINVAL;
> +
> + iowrite32((odthigh - XI3C_CYCLE_ADJUST) &
> XI3C_TIMING_MASK,
> + master->membase +
> XI3C_OD_SCL_HIGH_TIME_OFFSET);
> + iowrite32((odtlow - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase +
> XI3C_OD_SCL_LOW_TIME_OFFSET);
> +
> + tcasmin = DIV_ROUND_UP(XI3C_TCASMIN_NS, core_periodns);
> + }
> +
> + thdstart = max(thigh, tcasmin);
> + tsustart = max(tlow, tcasmin);
> + tsustop = max(tlow, tcasmin);
> +
> + iowrite32((tsustart - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase + XI3C_TSU_START_OFFSET);
> + iowrite32((thdstart - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase + XI3C_THD_START_OFFSET);
> + iowrite32((tsustop - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
> + master->membase + XI3C_TSU_STOP_OFFSET);
> +
> + return 0;
> +}
> +
> +static int xi3c_master_bus_init(struct i3c_master_controller *m) {
> + struct xi3c_master *master = to_xi3c_master(m);
> + struct i3c_bus *bus = i3c_master_get_bus(m);
> + struct i3c_device_info info = {};
> + unsigned long sclhz;
> + u32 pid1_bcr_dcr;
> + u8 mode;
> + int ret;
> +
> + switch (bus->mode) {
> + case I3C_BUS_MODE_MIXED_FAST:
> + case I3C_BUS_MODE_MIXED_LIMITED:
> + case I3C_BUS_MODE_MIXED_SLOW:
> + mode = XI3C_I2C_MODE;
> + sclhz = bus->scl_rate.i2c;
> + break;
> + case I3C_BUS_MODE_PURE:
> + mode = XI3C_SDR_MODE;
> + sclhz = bus->scl_rate.i3c;
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + ret = xi3c_clk_cfg(master, sclhz, mode);
> + if (ret)
> + return ret;
> +
> + xi3c_master_init(master);
> +
> + /* Get an address for the master. */
> + ret = i3c_master_get_free_addr(m, 0);
> + if (ret < 0)
> + return ret;
> +
> + info.dyn_addr = ret;
> +
> + /* Write the dynamic address value to the address register. */
> + iowrite32(info.dyn_addr, master->membase + XI3C_ADDRESS_OFFSET);
> +
> + /* Read PID, BCR and DCR values, and assign to i3c device info. */
> + pid1_bcr_dcr = ioread32(master->membase + XI3C_PID1_BCR_DCR);
> + info.pid = ((u64)FIELD_GET(XI3C_PID1_MASK, pid1_bcr_dcr) << 32) |
> + ioread32(master->membase + XI3C_PID0_OFFSET);
> + info.bcr = FIELD_GET(XI3C_BCR_MASK, pid1_bcr_dcr);
> + info.dcr = FIELD_GET(XI3C_DCR_MASK, pid1_bcr_dcr);
> +
> + return i3c_master_set_info(&master->base, &info); }
> +
> +static void xi3c_master_bus_cleanup(struct i3c_master_controller *m) {
> + struct xi3c_master *master = to_xi3c_master(m);
> +
> + xi3c_master_disable(master);
> +}
> +
> +static const struct i3c_master_controller_ops xi3c_master_ops = {
> + .bus_init = xi3c_master_bus_init,
> + .bus_cleanup = xi3c_master_bus_cleanup,
> + .do_daa = xi3c_master_do_daa,
> + .supports_ccc_cmd = xi3c_master_supports_ccc_cmd,
> + .send_ccc_cmd = xi3c_master_send_ccc_cmd,
> + .i3c_xfers = xi3c_master_i3c_xfers,
> + .i2c_xfers = xi3c_master_i2c_xfers,
> +};
> +
> +static int xi3c_master_probe(struct platform_device *pdev) {
> + struct xi3c_master *master;
> + int ret;
> +
> + master = devm_kzalloc(&pdev->dev, sizeof(*master), GFP_KERNEL);
> + if (!master)
> + return -ENOMEM;
> +
> + master->dev = &pdev->dev;
> +
> + master->membase = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(master->membase))
> + return dev_err_probe(master->dev, PTR_ERR(master->membase),
> + "Failed to map registers\n");
> +
> + master->pclk = devm_clk_get_enabled(master->dev, NULL);
> + if (IS_ERR(master->pclk))
> + return dev_err_probe(master->dev, PTR_ERR(master->pclk),
> + "Failed to get and enable clock\n");
> +
> + ret = devm_mutex_init(master->dev, &master->lock);
> + if (ret)
> + return ret;
> +
> + platform_set_drvdata(pdev, master);
> +
> + return i3c_master_register(&master->base, master->dev,
> + &xi3c_master_ops, false);
> +}
> +
> +static void xi3c_master_remove(struct platform_device *pdev) {
> + struct xi3c_master *master = platform_get_drvdata(pdev);
> +
> + i3c_master_unregister(&master->base);
> +}
> +
> +static const struct of_device_id xi3c_master_of_ids[] = {
> + { .compatible = "xlnx,axi-i3c-1.0" },
> + { },
> +};
> +MODULE_DEVICE_TABLE(of, xi3c_master_of_ids);
> +
> +static struct platform_driver xi3c_master_driver = {
> + .probe = xi3c_master_probe,
> + .remove = xi3c_master_remove,
> + .driver = {
> + .name = "axi-i3c-master",
> + .of_match_table = xi3c_master_of_ids,
> + },
> +};
> +module_platform_driver(xi3c_master_driver);
> +
> +MODULE_AUTHOR("Manikanta Guntupalli <manikanta.guntupalli@amd.com>");
> +MODULE_AUTHOR("Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>");
> +MODULE_AUTHOR("Shubham Patil <shubhamsanjay.patil@amd.com>");
I don't agree with adding new authors in V9.
This driver is already part of the downstream kernel and is being used:
https://github.com/Xilinx/linux-xlnx/blob/master/drivers/i3c/master/amd-i3c-master.c
The main purpose of V9 is to drop the framework-level support added in recent versions. The current V9 patch is mostly aligned with the initial patch versions (without framework support changes).
Thanks,
Manikanta
> +MODULE_DESCRIPTION("AMD AXI I3C master driver");
> MODULE_LICENSE("GPL");
> --
> 2.34.1
^ permalink raw reply
* Re: [PATCH v9 2/2] i3c: master: Add driver for AMD AXI I3C master controller
From: Frank Li @ 2026-06-23 16:45 UTC (permalink / raw)
To: Shubham Patil
Cc: git, michal.simek, alexandre.belloni, Frank.Li, robh, krzk+dt,
conor+dt, pgaj, wsa+renesas, tommaso.merciai.xr, arnd,
quic_msavaliy, Shyam-sundar.S-k, sakari.ailus, billy_tsai, kees,
gustavoars, jarkko.nikula, jorge.marques, linux-i3c, devicetree,
linux-kernel, linux-arch, linux-hardening, radhey.shyam.pandey,
srinivas.goud, shubhrajyoti.datta, Manikanta Guntupalli
In-Reply-To: <20260623114417.2578189-3-shubhamsanjay.patil@amd.com>
On Tue, Jun 23, 2026 at 05:14:16PM +0530, Shubham Patil wrote:
> From: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
>
> Add an I3C master driver and maintainers fragment for the AMD I3C bus
> controller.
>
> The driver currently supports the I3C bus operating in SDR mode,
> with features including Dynamic Address Assignment, private data
> transfers, and CCC transfers in both broadcast and direct modes. It
> also supports operation in I2C mode.
>
> The controller's data FIFOs are accessed big-endian; the driver performs
> this conversion locally using ioread32be()/iowrite32be() with the
> helpers, so it does not depend on any core FIFO-endianness helpers.
>
> Signed-off-by: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
> Co-developed-by: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
> Signed-off-by: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
> Co-developed-by: Shubham Patil <shubhamsanjay.patil@amd.com>
> Signed-off-by: Shubham Patil <shubhamsanjay.patil@amd.com>
> ---
...
> +#define XI3C_REV_NUM_MASK GENMASK(15, 8)
> +#define XI3C_PID1_MASK GENMASK(15, 0)
> +#define XI3C_FIFO_LEVEL_MASK GENMASK(15, 0)
> +#define XI3C_RESP_CODE_MASK GENMASK(8, 5)
> +#define XI3C_RESP_CODE_SUCCESS 0 /* Transfer completed OK */
> +#define XI3C_RESP_CODE_NO_TARGET 2 /* 7E NACK: no target on bus */
> +#define XI3C_RESP_CODE_NACK 3 /* Target NACK / CE2 / DAA end */
> +#define XI3C_ADDR_MASK GENMASK(6, 0)
> +#define XI3C_FIFOS_RST_MASK GENMASK(4, 1)
> +
> +/* Command FIFO word layout (bit ranges encoded in the GENMASK/BIT args) */
> +#define XI3C_CMD_TYPE GENMASK(3, 0) /* command type */
> +#define XI3C_CMD_TERMINATE BIT(4) /* terminate (last cmd of xfer) */
> +#define XI3C_CMD_ADDR GENMASK(15, 8) /* target address << 1 | RnW */
> +#define XI3C_CMD_LEN GENMASK(27, 16) /* payload length in bytes */
> +#define XI3C_CMD_TID GENMASK(31, 28) /* transfer ID */
> +
> +#define XI3C_OD_TLOW_NS 500000
> +#define XI3C_OD_THIGH_NS 41000
> +#define XI3C_I2C_TCASMIN_NS 600000
> +#define XI3C_TCASMIN_NS 260000
> +#define XI3C_MAXDATA_LENGTH 4095
> +#define XI3C_MAX_DEVS 32
> +#define XI3C_DAA_SLAVEINFO_READ_BYTECOUNT 8
> +
> +#define XI3C_THOLD_MIN_REV0 5 /* Min SDA hold cycles, rev 0 IP */
> +#define XI3C_THOLD_MIN_REV1 6 /* Min SDA hold cycles, rev >= 1 IP */
> +#define XI3C_CYCLE_ADJUST 2 /* SCL/SDA pre-bias for HW pipeline */
> +#define XI3C_FIFO_RESET_DELAY_US 10 /* HW settling time after FIFO reset */
> +#define XI3C_POLL_INTERVAL_US 10 /* readl_poll_timeout() sleep slice */
Can you provide comment where these value come from, spec, datasheet ...?
> +
> +#define XI3C_I2C_MODE 0
> +#define XI3C_I2C_TID 0
> +#define XI3C_SDR_MODE 1
> +#define XI3C_SDR_TID 1
> +
> +#define XI3C_WORD_LEN 4
> +
> +/*
> + * XI3C_RESP_TIMEOUT_US is in microseconds because it is passed as the
> + * timeout_us argument of readl_poll_timeout(). XI3C_XFER_TIMEOUT_MS is in
> + * milliseconds because it feeds msecs_to_jiffies(). Keep the two units
> + * distinct in the names so callers cannot mix them up.
> + */
> +#define XI3C_RESP_TIMEOUT_US 500000
> +#define XI3C_XFER_TIMEOUT_MS 1000
the same here.
> +
> +struct xi3c_cmd {
> + const void *tx_buf;
> + void *rx_buf;
> + u16 tx_len;
> + u16 rx_len;
> + u8 addr;
> + u8 type;
> + u8 tid;
> + bool rnw;
> + bool is_daa;
> + bool continued;
> + enum i3c_error_code err;
> +};
> +
...
> +
> +static void xi3c_master_reset_fifos(struct xi3c_master *master)
> +{
> + u32 data;
> +
> + /* Assert FIFO reset. */
> + data = ioread32(master->membase + XI3C_RESET_OFFSET);
> + data |= XI3C_FIFOS_RST_MASK;
> + iowrite32(data, master->membase + XI3C_RESET_OFFSET);
> + /* Read-back flushes the posted write before the settling delay below. */
> + ioread32(master->membase + XI3C_RESET_OFFSET);
> + udelay(XI3C_FIFO_RESET_DELAY_US);
now suggest use fsleep()
> +
> + /* De-assert FIFO reset, then wait for the FIFOs to come back up. */
> + data &= ~XI3C_FIFOS_RST_MASK;
> + iowrite32(data, master->membase + XI3C_RESET_OFFSET);
> + ioread32(master->membase + XI3C_RESET_OFFSET);
> + udelay(XI3C_FIFO_RESET_DELAY_US);
> +}
> +
> +static inline void xi3c_master_init(struct xi3c_master *master)
> +{
> + /* Reset fifos */
> + xi3c_master_reset_fifos(master);
> +
> + /* Enable controller */
> + xi3c_master_enable(master);
> +}
> +
> +static inline void xi3c_master_reinit(struct xi3c_master *master)
> +{
> + /* Reset fifos */
> + xi3c_master_reset_fifos(master);
> +
> + /* Resume controller */
> + xi3c_master_resume(master);
> +}
> +
> +static struct xi3c_xfer *xi3c_master_alloc_xfer(unsigned int ncmds)
> +{
> + struct xi3c_xfer *xfer;
> +
> + xfer = kzalloc(struct_size(xfer, cmds, ncmds), GFP_KERNEL);
ues new API, kzalloc_flex()
> + if (!xfer)
> + return NULL;
> +
> + xfer->ncmds = ncmds;
> +
> + return xfer;
> +}
> +
> +static void xi3c_master_rd_from_rx_fifo(struct xi3c_master *master,
> + struct xi3c_cmd *cmd)
> +{
> + u16 rx_data_available;
> + u16 copy_len;
> + u16 len;
> +
> + rx_data_available = xi3c_rd_fifo_level(master);
> + len = rx_data_available * XI3C_WORD_LEN;
> +
> + if (!len)
> + return;
> +
> + copy_len = min_t(u16, len, cmd->rx_len);
now need't min_t, just min
> + xi3c_readl_fifo(master->membase + XI3C_RD_FIFO_OFFSET,
> + (u8 *)cmd->rx_buf, copy_len);
> +
> + cmd->rx_buf = (u8 *)cmd->rx_buf + copy_len;
> + cmd->rx_len -= copy_len;
> +}
> +
...
> +
> + timeout = jiffies + msecs_to_jiffies(XI3C_XFER_TIMEOUT_MS);
> +
> + /* Read data from rx fifo */
> + while (cmd->rx_len > 0 && !xi3c_is_resp_available(master)) {
> + if (time_after(jiffies, timeout)) {
> + dev_err(master->dev, "XI3C read timeout\n");
> + return -EIO;
> + }
> + xi3c_master_rd_from_rx_fifo(master, cmd);
> + usleep_range(XI3C_POLL_INTERVAL_US, 2 * XI3C_POLL_INTERVAL_US);
> + }
can you use read_poll_timeout macro?
> +
> + /* Read remaining data */
> + xi3c_master_rd_from_rx_fifo(master, cmd);
> +
> + return 0;
> +}
> +
...
> +
> + for (i = 0; i < master->daa.index; i++) {
> + u64 pid;
> +
> + ret = i3c_master_add_i3c_dev_locked(m, master->daa.addrs[i]);
> + if (ret)
> + goto err_daa;
https://lore.kernel.org/linux-i3c/20260608054312.10604-7-adrian.hunter@intel.com/T/#u
which defer add i3c device.
And don't check error here, because one device add failure should not impact
other following devices.
Frank
> +
> + pid = FIELD_GET(XI3C_PID_MASK,
> + get_unaligned_be64(pid_bufs[i]));
> + dev_dbg(master->dev, "Client %d: PID: 0x%llx\n", i, pid);
> + }
> +
> + return 0;
> +
> +err_daa:
> + xi3c_master_reinit(master);
> + return ret;
> +}
> +
...
> +static int xi3c_master_send_bdcast_ccc_cmd(struct xi3c_master *master,
> + struct i3c_ccc_cmd *ccc)
> +{
> + struct xi3c_xfer *xfer __free(kfree) = NULL;
> + u8 *buf __free(kfree) = NULL;
> + struct xi3c_cmd *cmd;
> + u16 xfer_len;
> + int ret;
> +
> + if (ccc->dests[0].payload.len >= XI3C_MAXDATA_LENGTH)
> + return -EINVAL;
> +
> + xfer_len = ccc->dests[0].payload.len + 1;
> +
> + xfer = xi3c_master_alloc_xfer(1);
> + if (!xfer)
> + return -ENOMEM;
> +
> + buf = kmalloc(xfer_len, GFP_KERNEL);
kmalloc_obj
Frank
> + if (!buf)
> + return -ENOMEM;
> +
> + buf[0] = ccc->id;
> + memcpy(&buf[1], ccc->dests[0].payload.data, ccc->dests[0].payload.len);
> +
> + cmd = &xfer->cmds[0];
> + cmd->addr = ccc->dests[0].addr;
> + cmd->rnw = ccc->rnw;
> + cmd->tx_buf = buf;
> + cmd->tx_len = xfer_len;
> + cmd->type = XI3C_SDR_MODE;
> + cmd->tid = XI3C_SDR_TID;
> + cmd->continued = false;
> +
> + ret = xi3c_master_common_xfer(master, xfer);
> + ccc->err = cmd->err;
> +
> + return ret;
> +}
^ permalink raw reply
* Re: [PATCH 1/2] bug: Provide WARN_ON.*DEFERRED() macros for console deferred output
From: Petr Mladek @ 2026-06-23 15:49 UTC (permalink / raw)
To: Andrew Morton
Cc: Sebastian Andrzej Siewior, linux-arch, linux-kernel, sched-ext,
netdev, David S . Miller, Andrea Righi, Arnd Bergmann, Ben Segall,
Breno Leitao, Changwoo Min, David Vernet, Dietmar Eggemann,
Eric Dumazet, Ingo Molnar, Jakub Kicinski, John Ogness,
Juri Lelli, K Prateek Nayak, Paolo Abeni, Peter Zijlstra,
Sergey Senozhatsky, Simon Horman, Steven Rostedt, Tejun Heo,
Vincent Guittot, Vlad Poenaru
In-Reply-To: <20260623081258.580e034fdb5b98f4f8dba44a@linux-foundation.org>
On Tue 2026-06-23 08:12:58, Andrew Morton wrote:
> On Tue, 23 Jun 2026 16:26:49 +0200 Sebastian Andrzej Siewior <bigeasy@linutronix.de> wrote:
>
> > Provide a deferred version of the WARN_ON() macro. It will delay
> > flushing the console until a later context. It is needed in a context
> > where the caller holds locks which can lead to a deadlock content is
> > flushed to the console driver.
> > An example would from a warning from within the scheduler resulting in a
> > wake-up of a task.
> >
> > Deferring the output works by using printk_deferred_enter/ exit() around
> > the printing output. This must be used in a context where the task can't
> > migrate to another CPU. This should be the case usually, since the
> > scheduler would acquire the rq lock whith disabled interrupts, but to be
> > safe preemption is disabled to guarantee this.
> >
> > In order not to bloat the code on architectures which provide an
> > optimized __WARN_FLAGS() define BUGFLAG_DEFERRED which is handled by
> > __report_bug() and does not increase the code size.
> >
> > Provide the DEFERRED macros based on __WARN_FLAGS and __WARN_FLAGS
> > macros. Extend __report_bug() to handle the deferred case.
> >
> > ...
> >
> > --- a/include/asm-generic/bug.h
> > +++ b/include/asm-generic/bug.h
> > @@ -229,7 +230,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
> > */
> > bug->flags |= BUGFLAG_DONE;
> > }
> > -
> > + if (deferred) {
> > + preempt_disable_notrace();
> > + printk_deferred_enter();
> > + }
>
> For some reason the comment over printk_deferred_enter() says
> "Interrupts must be disabled for the deferred duration". Is that the
> case for all the printk_deferred_enter() calls which this patch adds?
Strictly speaking, "only" CPU migration must be disabled around
printk_deferred_enter()/exit() call because the state is stored
in a per-CPU variable.
It means that preempt_disable() would work.
I do not recall whether we mentioned interrupts by mistake or
on purpose. It is possible that we suggested to disable interrupts
because we did not want to deffer messages from unrelated (interrupt)
context.
Best Regards,
Petr
^ permalink raw reply
* Re: [PATCH 1/2] bug: Provide WARN_ON.*DEFERRED() macros for console deferred output
From: Andrew Morton @ 2026-06-23 15:12 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-arch, linux-kernel, sched-ext, netdev, David S . Miller,
Andrea Righi, Arnd Bergmann, Ben Segall, Breno Leitao,
Changwoo Min, David Vernet, Dietmar Eggemann, Eric Dumazet,
Ingo Molnar, Jakub Kicinski, John Ogness, Juri Lelli,
K Prateek Nayak, Paolo Abeni, Peter Zijlstra, Petr Mladek,
Sergey Senozhatsky, Simon Horman, Steven Rostedt, Tejun Heo,
Vincent Guittot, Vlad Poenaru
In-Reply-To: <20260623142650.265721-2-bigeasy@linutronix.de>
On Tue, 23 Jun 2026 16:26:49 +0200 Sebastian Andrzej Siewior <bigeasy@linutronix.de> wrote:
> Provide a deferred version of the WARN_ON() macro. It will delay
> flushing the console until a later context. It is needed in a context
> where the caller holds locks which can lead to a deadlock content is
> flushed to the console driver.
> An example would from a warning from within the scheduler resulting in a
> wake-up of a task.
>
> Deferring the output works by using printk_deferred_enter/ exit() around
> the printing output. This must be used in a context where the task can't
> migrate to another CPU. This should be the case usually, since the
> scheduler would acquire the rq lock whith disabled interrupts, but to be
> safe preemption is disabled to guarantee this.
>
> In order not to bloat the code on architectures which provide an
> optimized __WARN_FLAGS() define BUGFLAG_DEFERRED which is handled by
> __report_bug() and does not increase the code size.
>
> Provide the DEFERRED macros based on __WARN_FLAGS and __WARN_FLAGS
> macros. Extend __report_bug() to handle the deferred case.
>
> ...
>
> --- a/include/asm-generic/bug.h
> +++ b/include/asm-generic/bug.h
> @@ -229,7 +230,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
> */
> bug->flags |= BUGFLAG_DONE;
> }
> -
> + if (deferred) {
> + preempt_disable_notrace();
> + printk_deferred_enter();
> + }
For some reason the comment over printk_deferred_enter() says
"Interrupts must be disabled for the deferred duration". Is that the
case for all the printk_deferred_enter() calls which this patch adds?
^ permalink raw reply
* Re: [PATCH 1/2] bug: Provide WARN_ON.*DEFERRED() macros for console deferred output
From: K Prateek Nayak @ 2026-06-23 14:54 UTC (permalink / raw)
To: Sebastian Andrzej Siewior, linux-arch, linux-kernel, sched-ext,
netdev
Cc: David S . Miller, Andrea Righi, Andrew Morton, Arnd Bergmann,
Ben Segall, Breno Leitao, Changwoo Min, David Vernet,
Dietmar Eggemann, Eric Dumazet, Ingo Molnar, Jakub Kicinski,
John Ogness, Juri Lelli, Paolo Abeni, Peter Zijlstra, Petr Mladek,
Sergey Senozhatsky, Simon Horman, Steven Rostedt, Tejun Heo,
Vincent Guittot, Vlad Poenaru
In-Reply-To: <20260623142650.265721-2-bigeasy@linutronix.de>
Hello Sebastian,
On 6/23/2026 7:56 PM, Sebastian Andrzej Siewior wrote:
> --- a/lib/bug.c
> +++ b/lib/bug.c
> @@ -196,7 +196,7 @@ void __warn_printf(const char *fmt, struct pt_regs *regs)
>
> static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long bugaddr, struct pt_regs *regs)
> {
> - bool warning, once, done, no_cut, has_args;
> + bool warning, once, done, no_cut, has_args, deferred;
> const char *file, *fmt;
> unsigned line;
>
> @@ -219,6 +219,7 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
> done = bug->flags & BUGFLAG_DONE;
> no_cut = bug->flags & BUGFLAG_NO_CUT_HERE;
> has_args = bug->flags & BUGFLAG_ARGS;
> + deferred = bug->flags & BUGFLAG_DEFERRED;
>
> if (warning && once) {
> if (done)
> @@ -229,7 +230,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
> */
> bug->flags |= BUGFLAG_DONE;
> }
> -
> + if (deferred) {
> + preempt_disable_notrace();
> + printk_deferred_enter();
> + }
> /*
> * BUG() and WARN_ON() families don't print a custom debug message
> * before triggering the exception handler, so we must add the
> @@ -245,6 +249,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
> /* this is a WARN_ON rather than BUG/BUG_ON */
> __warn(file, line, (void *)bugaddr, BUG_GET_TAINT(bug), regs,
> NULL);
> + if (deferred) {
> + printk_deferred_exit();
> + preempt_enable_notrace();
> + }
> return BUG_TRAP_TYPE_WARN;
nit.
Instead of replicating these bits, can we replace that return with a
"goto out" ...
> }
>
> @@ -254,6 +262,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
> pr_crit("kernel BUG at %pB [verbose debug info unavailable]\n",
> (void *)bugaddr);
>
out:
> + if (deferred) {
> + printk_deferred_exit();
> + preempt_enable_notrace();
> + }
> return BUG_TRAP_TYPE_BUG;
... and replace this return with a:
return (warning) ? BUG_TRAP_TYPE_WARN : BUG_TRAP_TYPE_BUG;
Looks a tab bit cleaner to my eyes. Thoughts?
> }
>
--
Thanks and Regards,
Prateek
^ permalink raw reply
* [PATCH 2/2] sched: Use WARN_ON.*_DEFERRED()
From: Sebastian Andrzej Siewior @ 2026-06-23 14:26 UTC (permalink / raw)
To: linux-arch, linux-kernel, sched-ext, netdev
Cc: David S . Miller, Andrea Righi, Andrew Morton, Arnd Bergmann,
Ben Segall, Breno Leitao, Changwoo Min, David Vernet,
Dietmar Eggemann, Eric Dumazet, Ingo Molnar, Jakub Kicinski,
John Ogness, Juri Lelli, K Prateek Nayak, Paolo Abeni,
Peter Zijlstra, Petr Mladek, Sergey Senozhatsky, Simon Horman,
Steven Rostedt, Tejun Heo, Vincent Guittot, Vlad Poenaru,
Sebastian Andrzej Siewior
In-Reply-To: <20260623142650.265721-1-bigeasy@linutronix.de>
Vlad managed to trigger a warning in __enqueue_entity() while the rq
lock was held. He was using the netconsole in an older kernel which was
a legacy console (not nbcon). This resulted in an immediate flush which
led to sending packets and this in turn led to waking ksoftirqd. This
wake up ended up in deadlock because the scheduler tried to acquire the
already acquired rq.
This problem is not limited to the netconsole but all legacy consoles:
Should the console wake any task while holding its internal lock then
lockdep will observe and report a possible AB-BA deadlock. Also since
the warning does not happen regulary, lockdep may observe a lockchain
while acquiring the locks, leading to a recursion report while holding
locks.
More importantly after the during the console printing and once it is
finished the console semaphore is released which will lead to wakeup if
there is a waiter pending.
Replace WARNs within the scheduler with the DEFERRED variant. This will
queue an irq_work and the print will occur once the locks are dropped.
Reported-by: Vlad Poenaru <vlad.wing@gmail.com>
Closes: https://lore.kernel.org/all/20260610183621.3915271-1-vlad.wing@gmail.com
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
kernel/sched/core.c | 78 +++++++++++++-------------
kernel/sched/core_sched.c | 6 +-
kernel/sched/cpudeadline.c | 6 +-
kernel/sched/deadline.c | 62 ++++++++++-----------
kernel/sched/ext.c | 110 ++++++++++++++++++-------------------
kernel/sched/fair.c | 88 ++++++++++++++---------------
kernel/sched/rt.c | 36 ++++++------
kernel/sched/sched.h | 18 +++---
8 files changed, 202 insertions(+), 202 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b8871449d3c69..0e282457abb91 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -853,7 +853,7 @@ void update_rq_clock(struct rq *rq)
return;
if (sched_feat(WARN_DOUBLE_CLOCK))
- WARN_ON_ONCE(rq->clock_update_flags & RQCF_UPDATED);
+ WARN_ON_ONCE_DEFERRED(rq->clock_update_flags & RQCF_UPDATED);
rq->clock_update_flags |= RQCF_UPDATED;
clock = sched_clock_cpu(cpu_of(rq));
@@ -1807,7 +1807,7 @@ static inline void uclamp_rq_dec_id(struct rq *rq, struct task_struct *p,
bucket = &uc_rq->bucket[uc_se->bucket_id];
- WARN_ON_ONCE(!bucket->tasks);
+ WARN_ON_ONCE_DEFERRED(!bucket->tasks);
if (likely(bucket->tasks))
bucket->tasks--;
@@ -1827,7 +1827,7 @@ static inline void uclamp_rq_dec_id(struct rq *rq, struct task_struct *p,
* Defensive programming: this should never happen. If it happens,
* e.g. due to future modification, warn and fix up the expected value.
*/
- WARN_ON_ONCE(bucket->value > rq_clamp);
+ WARN_ON_ONCE_DEFERRED(bucket->value > rq_clamp);
if (bucket->value >= rq_clamp) {
bkt_clamp = uclamp_rq_max_value(rq, clamp_id, uc_se->value);
uclamp_rq_set(rq, clamp_id, bkt_clamp);
@@ -2210,7 +2210,7 @@ void activate_task(struct rq *rq, struct task_struct *p, int flags)
void deactivate_task(struct rq *rq, struct task_struct *p, int flags)
{
- WARN_ON_ONCE(flags & DEQUEUE_SLEEP);
+ WARN_ON_ONCE_DEFERRED(flags & DEQUEUE_SLEEP);
WRITE_ONCE(p->on_rq, TASK_ON_RQ_MIGRATING);
ASSERT_EXCLUSIVE_WRITER(p->on_rq);
@@ -2516,7 +2516,7 @@ static struct rq *move_queued_task(struct rq *rq, struct rq_flags *rf,
rq = cpu_rq(new_cpu);
rq_lock(rq, rf);
- WARN_ON_ONCE(task_cpu(p) != new_cpu);
+ WARN_ON_ONCE_DEFERRED(task_cpu(p) != new_cpu);
activate_task(rq, p, 0);
wakeup_preempt(rq, p, 0);
@@ -2602,7 +2602,7 @@ static int migration_cpu_stop(void *data)
* If we were passed a pending, then ->stop_pending was set, thus
* p->migration_pending must have remained stable.
*/
- WARN_ON_ONCE(pending && pending != p->migration_pending);
+ WARN_ON_ONCE_DEFERRED(pending && pending != p->migration_pending);
/*
* If task_rq(p) != rq, it cannot be migrated here, because we're
@@ -2661,7 +2661,7 @@ static int migration_cpu_stop(void *data)
* determine is_migration_disabled() and so have to chase after
* it.
*/
- WARN_ON_ONCE(!pending->stop_pending);
+ WARN_ON_ONCE_DEFERRED(!pending->stop_pending);
preempt_disable();
rq_unlock(rq, &rf);
raw_spin_unlock_irqrestore(&p->pi_lock, rf.flags);
@@ -3004,7 +3004,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag
*
* Either way, we really should have a @pending here.
*/
- if (WARN_ON_ONCE(!pending)) {
+ if (WARN_ON_ONCE_DEFERRED(!pending)) {
task_rq_unlock(rq, p, rf);
return -EINVAL;
}
@@ -3116,9 +3116,9 @@ static int __set_cpus_allowed_ptr_locked(struct task_struct *p,
goto out;
}
- if (WARN_ON_ONCE(p == current &&
- is_migration_disabled(p) &&
- !cpumask_test_cpu(task_cpu(p), ctx->new_mask))) {
+ if (WARN_ON_ONCE_DEFERRED(p == current &&
+ is_migration_disabled(p) &&
+ !cpumask_test_cpu(task_cpu(p), ctx->new_mask))) {
ret = -EBUSY;
goto out;
}
@@ -3267,7 +3267,7 @@ void force_compatible_cpus_allowed_ptr(struct task_struct *p)
cpumask_pr_args(override_mask));
}
- WARN_ON(set_cpus_allowed_ptr(p, override_mask));
+ WARN_ON_DEFERRED(set_cpus_allowed_ptr(p, override_mask));
out_free_mask:
cpus_read_unlock();
free_cpumask_var(new_mask);
@@ -3293,7 +3293,7 @@ void relax_compatible_cpus_allowed_ptr(struct task_struct *p)
* Cpuset masking will be done there too.
*/
ret = __sched_setaffinity(p, &ac);
- WARN_ON_ONCE(ret);
+ WARN_ON_ONCE_DEFERRED(ret);
}
#ifdef CONFIG_SMP
@@ -3306,16 +3306,16 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
* We should never call set_task_cpu() on a blocked task,
* ttwu() will sort out the placement.
*/
- WARN_ON_ONCE(state != TASK_RUNNING && state != TASK_WAKING && !p->on_rq);
+ WARN_ON_ONCE_DEFERRED(state != TASK_RUNNING && state != TASK_WAKING && !p->on_rq);
/*
* Migrating fair class task must have p->on_rq = TASK_ON_RQ_MIGRATING,
* because schedstat_wait_{start,end} rebase migrating task's wait_start
* time relying on p->on_rq.
*/
- WARN_ON_ONCE(state == TASK_RUNNING &&
- p->sched_class == &fair_sched_class &&
- (p->on_rq && !task_on_rq_migrating(p)));
+ WARN_ON_ONCE_DEFERRED(state == TASK_RUNNING &&
+ p->sched_class == &fair_sched_class &&
+ (p->on_rq && !task_on_rq_migrating(p)));
#ifdef CONFIG_LOCKDEP
/*
@@ -3328,15 +3328,15 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
* Furthermore, all task_rq users should acquire both locks, see
* task_rq_lock().
*/
- WARN_ON_ONCE(debug_locks && !(lockdep_is_held(&p->pi_lock) ||
- lockdep_is_held(__rq_lockp(task_rq(p)))));
+ WARN_ON_ONCE_DEFERRED(debug_locks && !(lockdep_is_held(&p->pi_lock) ||
+ lockdep_is_held(__rq_lockp(task_rq(p)))));
#endif
/*
* Clearly, migrating tasks to offline CPUs is a fairly daft thing.
*/
- WARN_ON_ONCE(!cpu_online(new_cpu));
+ WARN_ON_ONCE_DEFERRED(!cpu_online(new_cpu));
- WARN_ON_ONCE(is_migration_disabled(p));
+ WARN_ON_ONCE_DEFERRED(is_migration_disabled(p));
trace_sched_migrate_task(p, new_cpu);
@@ -3803,10 +3803,10 @@ void sched_ttwu_pending(void *arg)
update_rq_clock(rq);
llist_for_each_entry_safe(p, t, llist, wake_entry.llist) {
- if (WARN_ON_ONCE(p->on_cpu))
+ if (WARN_ON_ONCE_DEFERRED(p->on_cpu))
smp_cond_load_acquire(&p->on_cpu, !VAL);
- if (WARN_ON_ONCE(task_cpu(p) != cpu_of(rq)))
+ if (WARN_ON_ONCE_DEFERRED(task_cpu(p) != cpu_of(rq)))
set_task_cpu(p, cpu_of(rq));
ttwu_do_activate(rq, p, p->sched_remote_wakeup ? WF_MIGRATED : 0, &rf);
@@ -4003,8 +4003,8 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
int match;
if (IS_ENABLED(CONFIG_DEBUG_PREEMPT)) {
- WARN_ON_ONCE((state & TASK_RTLOCK_WAIT) &&
- state != TASK_RTLOCK_WAIT);
+ WARN_ON_ONCE_DEFERRED((state & TASK_RTLOCK_WAIT) &&
+ state != TASK_RTLOCK_WAIT);
}
*success = !!(match = __task_state_match(p, state));
@@ -5745,7 +5745,7 @@ static void sched_tick_remote(struct work_struct *work)
* we are always sure that there is no proxy (only a
* single task is running).
*/
- WARN_ON_ONCE(rq->curr != rq->donor);
+ WARN_ON_ONCE_DEFERRED(rq->curr != rq->donor);
update_rq_clock(rq);
if (!is_idle_task(curr)) {
@@ -5754,7 +5754,7 @@ static void sched_tick_remote(struct work_struct *work)
* reasonable amount of time.
*/
u64 delta = rq_clock_task(rq) - curr->se.exec_start;
- WARN_ON_ONCE(delta > (u64)NSEC_PER_SEC * 30);
+ WARN_ON_ONCE_DEFERRED(delta > (u64)NSEC_PER_SEC * 30);
}
curr->sched_class->task_tick(rq, curr, 0);
@@ -5769,7 +5769,7 @@ static void sched_tick_remote(struct work_struct *work)
* first update state to reflect hotplug activity if required.
*/
os = atomic_fetch_add_unless(&twork->state, -1, TICK_SCHED_REMOTE_RUNNING);
- WARN_ON_ONCE(os == TICK_SCHED_REMOTE_OFFLINE);
+ WARN_ON_ONCE_DEFERRED(os == TICK_SCHED_REMOTE_OFFLINE);
if (os == TICK_SCHED_REMOTE_RUNNING)
queue_delayed_work(system_dfl_wq, dwork, HZ);
}
@@ -6196,7 +6196,7 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
* For robustness, update the min_vruntime_fi for
* unconstrained picks as well.
*/
- WARN_ON_ONCE(fi_before);
+ WARN_ON_ONCE_DEFERRED(fi_before);
task_vruntime_update(rq, next, false);
goto out_set_next;
}
@@ -6274,7 +6274,7 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
rq->core_sched_seq = rq->core->core_pick_seq;
/* Something should have been selected for current CPU */
- WARN_ON_ONCE(!next);
+ WARN_ON_ONCE_DEFERRED(!next);
/*
* Reschedule siblings
@@ -6317,7 +6317,7 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
}
/* Did we break L1TF mitigation requirements? */
- WARN_ON_ONCE(!cookie_match(next, rq_i->core_pick));
+ WARN_ON_ONCE_DEFERRED(!cookie_match(next, rq_i->core_pick));
if (rq_i->curr == rq_i->core_pick) {
rq_i->core_pick = NULL;
@@ -6717,7 +6717,7 @@ static void proxy_migrate_task(struct rq *rq, struct rq_flags *rf,
struct rq *target_rq = cpu_rq(target_cpu);
lockdep_assert_rq_held(rq);
- WARN_ON(p == rq->curr);
+ WARN_ON_DEFERRED(p == rq->curr);
/*
* Since we are migrating a blocked donor, it could be rq->donor,
* and we want to make sure there aren't any references from this
@@ -6749,7 +6749,7 @@ static void proxy_force_return(struct rq *rq, struct rq_flags *rf,
int cpu, wake_flag = WF_TTWU;
lockdep_assert_rq_held(rq);
- WARN_ON(p == rq->curr);
+ WARN_ON_DEFERRED(p == rq->curr);
if (p == rq->donor)
proxy_resched_idle(rq);
@@ -6951,7 +6951,7 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf)
* guarantee its existence, as per ttwu_remote().
*/
}
- WARN_ON_ONCE(owner && !owner->on_rq);
+ WARN_ON_ONCE_DEFERRED(owner && !owner->on_rq);
return owner;
deactivate:
@@ -7631,8 +7631,8 @@ void rt_mutex_setprio(struct task_struct *p, struct task_struct *pi_task)
* real need to boost.
*/
if (unlikely(p == rq->idle)) {
- WARN_ON(p != rq->curr);
- WARN_ON(p->pi_blocked_on);
+ WARN_ON_DEFERRED(p != rq->curr);
+ WARN_ON_DEFERRED(p->pi_blocked_on);
goto out_unlock;
}
@@ -8463,7 +8463,7 @@ static void balance_push_set(int cpu, bool on)
rq_lock_irqsave(rq, &rf);
if (on) {
- WARN_ON_ONCE(rq->balance_callback);
+ WARN_ON_ONCE_DEFERRED(rq->balance_callback);
rq->balance_callback = &balance_push_callback;
} else if (rq->balance_callback == &balance_push_callback) {
rq->balance_callback = NULL;
@@ -11150,7 +11150,7 @@ struct sched_change_ctx *sched_change_begin(struct task_struct *p, unsigned int
* Must exclusively use matched flags since this is both dequeue and
* enqueue.
*/
- WARN_ON_ONCE(flags & 0xFFFF0000);
+ WARN_ON_ONCE_DEFERRED(flags & 0xFFFF0000);
lockdep_assert_rq_held(rq);
@@ -11198,7 +11198,7 @@ void sched_change_end(struct sched_change_ctx *ctx)
/*
* Changing class without *QUEUE_CLASS is bad.
*/
- WARN_ON_ONCE(p->sched_class != ctx->class && !(ctx->flags & ENQUEUE_CLASS));
+ WARN_ON_ONCE_DEFERRED(p->sched_class != ctx->class && !(ctx->flags & ENQUEUE_CLASS));
if ((ctx->flags & ENQUEUE_CLASS) && p->sched_class->switching_to)
p->sched_class->switching_to(rq, p);
diff --git a/kernel/sched/core_sched.c b/kernel/sched/core_sched.c
index 73b6b24269119..ec88ed7d8ee87 100644
--- a/kernel/sched/core_sched.c
+++ b/kernel/sched/core_sched.c
@@ -67,7 +67,7 @@ static unsigned long sched_core_update_cookie(struct task_struct *p,
* a cookie until after we've removed it, we must have core scheduling
* enabled here.
*/
- WARN_ON_ONCE((p->core_cookie || cookie) && !sched_core_enabled(rq));
+ WARN_ON_ONCE_DEFERRED((p->core_cookie || cookie) && !sched_core_enabled(rq));
if (sched_core_enqueued(p))
sched_core_dequeue(rq, p, DEQUEUE_SAVE);
@@ -249,7 +249,7 @@ void __sched_core_account_forceidle(struct rq *rq)
lockdep_assert_rq_held(rq);
- WARN_ON_ONCE(!rq->core->core_forceidle_count);
+ WARN_ON_ONCE_DEFERRED(!rq->core->core_forceidle_count);
if (rq->core->core_forceidle_start == 0)
return;
@@ -260,7 +260,7 @@ void __sched_core_account_forceidle(struct rq *rq)
rq->core->core_forceidle_start = now;
- if (WARN_ON_ONCE(!rq->core->core_forceidle_occupation)) {
+ if (WARN_ON_ONCE_DEFERRED(!rq->core->core_forceidle_occupation)) {
/* can't be forced idle without a running task */
} else if (rq->core->core_forceidle_count > 1 ||
rq->core->core_forceidle_occupation > 1) {
diff --git a/kernel/sched/cpudeadline.c b/kernel/sched/cpudeadline.c
index 0a2b7e30fd10c..e305a8e993e27 100644
--- a/kernel/sched/cpudeadline.c
+++ b/kernel/sched/cpudeadline.c
@@ -149,7 +149,7 @@ int cpudl_find(struct cpudl *cp, struct task_struct *p,
} else {
int best_cpu = cpudl_maximum(cp);
- WARN_ON(best_cpu != -1 && !cpu_present(best_cpu));
+ WARN_ON_DEFERRED(best_cpu != -1 && !cpu_present(best_cpu));
if (cpumask_test_cpu(best_cpu, &p->cpus_mask) &&
dl_time_before(dl_se->deadline, cp->elements[0].dl)) {
@@ -177,7 +177,7 @@ void cpudl_clear(struct cpudl *cp, int cpu, bool online)
int old_idx, new_cpu;
unsigned long flags;
- WARN_ON(!cpu_present(cpu));
+ WARN_ON_DEFERRED(!cpu_present(cpu));
raw_spin_lock_irqsave(&cp->lock, flags);
@@ -220,7 +220,7 @@ void cpudl_set(struct cpudl *cp, int cpu, u64 dl)
int old_idx;
unsigned long flags;
- WARN_ON(!cpu_present(cpu));
+ WARN_ON_DEFERRED(!cpu_present(cpu));
raw_spin_lock_irqsave(&cp->lock, flags);
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 7db4c87df83b0..863ac7509192f 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -217,8 +217,8 @@ void __add_running_bw(u64 dl_bw, struct dl_rq *dl_rq)
lockdep_assert_rq_held(rq_of_dl_rq(dl_rq));
dl_rq->running_bw += dl_bw;
- WARN_ON_ONCE(dl_rq->running_bw < old); /* overflow */
- WARN_ON_ONCE(dl_rq->running_bw > dl_rq->this_bw);
+ WARN_ON_ONCE_DEFERRED(dl_rq->running_bw < old); /* overflow */
+ WARN_ON_ONCE_DEFERRED(dl_rq->running_bw > dl_rq->this_bw);
/* kick cpufreq (see the comment in kernel/sched/sched.h). */
cpufreq_update_util(rq_of_dl_rq(dl_rq), 0);
}
@@ -230,7 +230,7 @@ void __sub_running_bw(u64 dl_bw, struct dl_rq *dl_rq)
lockdep_assert_rq_held(rq_of_dl_rq(dl_rq));
dl_rq->running_bw -= dl_bw;
- WARN_ON_ONCE(dl_rq->running_bw > old); /* underflow */
+ WARN_ON_ONCE_DEFERRED(dl_rq->running_bw > old); /* underflow */
if (dl_rq->running_bw > old)
dl_rq->running_bw = 0;
/* kick cpufreq (see the comment in kernel/sched/sched.h). */
@@ -244,7 +244,7 @@ void __add_rq_bw(u64 dl_bw, struct dl_rq *dl_rq)
lockdep_assert_rq_held(rq_of_dl_rq(dl_rq));
dl_rq->this_bw += dl_bw;
- WARN_ON_ONCE(dl_rq->this_bw < old); /* overflow */
+ WARN_ON_ONCE_DEFERRED(dl_rq->this_bw < old); /* overflow */
}
static inline
@@ -254,10 +254,10 @@ void __sub_rq_bw(u64 dl_bw, struct dl_rq *dl_rq)
lockdep_assert_rq_held(rq_of_dl_rq(dl_rq));
dl_rq->this_bw -= dl_bw;
- WARN_ON_ONCE(dl_rq->this_bw > old); /* underflow */
+ WARN_ON_ONCE_DEFERRED(dl_rq->this_bw > old); /* underflow */
if (dl_rq->this_bw > old)
dl_rq->this_bw = 0;
- WARN_ON_ONCE(dl_rq->running_bw > dl_rq->this_bw);
+ WARN_ON_ONCE_DEFERRED(dl_rq->running_bw > dl_rq->this_bw);
}
static inline
@@ -335,7 +335,7 @@ void cancel_inactive_timer(struct sched_dl_entity *dl_se)
static void dl_change_utilization(struct task_struct *p, u64 new_bw)
{
- WARN_ON_ONCE(p->dl.flags & SCHED_FLAG_SUGOV);
+ WARN_ON_ONCE_DEFERRED(p->dl.flags & SCHED_FLAG_SUGOV);
if (task_on_rq_queued(p))
return;
@@ -416,7 +416,7 @@ static void task_non_contending(struct sched_dl_entity *dl_se, bool dl_task)
if (dl_entity_is_special(dl_se))
return;
- WARN_ON(dl_se->dl_non_contending);
+ WARN_ON_DEFERRED(dl_se->dl_non_contending);
zerolag_time = dl_se->deadline -
div64_long((dl_se->runtime * dl_se->dl_period),
@@ -582,7 +582,7 @@ static void enqueue_pushable_dl_task(struct rq *rq, struct task_struct *p)
{
struct rb_node *leftmost;
- WARN_ON_ONCE(!RB_EMPTY_NODE(&p->pushable_dl_tasks));
+ WARN_ON_ONCE_DEFERRED(!RB_EMPTY_NODE(&p->pushable_dl_tasks));
leftmost = rb_add_cached(&p->pushable_dl_tasks,
&rq->dl.pushable_dl_tasks_root,
@@ -664,7 +664,7 @@ static struct rq *dl_task_offline_migration(struct rq *rq, struct task_struct *p
* Failed to find any suitable CPU.
* The task will never come back!
*/
- WARN_ON_ONCE(dl_bandwidth_enabled());
+ WARN_ON_ONCE_DEFERRED(dl_bandwidth_enabled());
/*
* If admission control is disabled we
@@ -756,8 +756,8 @@ static inline void setup_new_dl_entity(struct sched_dl_entity *dl_se)
struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
struct rq *rq = rq_of_dl_rq(dl_rq);
- WARN_ON(is_dl_boosted(dl_se));
- WARN_ON(dl_time_before(rq_clock(rq), dl_se->deadline));
+ WARN_ON_DEFERRED(is_dl_boosted(dl_se));
+ WARN_ON_DEFERRED(dl_time_before(rq_clock(rq), dl_se->deadline));
/*
* We are racing with the deadline timer. So, do nothing because
@@ -801,7 +801,7 @@ static void replenish_dl_entity(struct sched_dl_entity *dl_se)
struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
struct rq *rq = rq_of_dl_rq(dl_rq);
- WARN_ON_ONCE(pi_of(dl_se)->dl_runtime <= 0);
+ WARN_ON_ONCE_DEFERRED(pi_of(dl_se)->dl_runtime <= 0);
/*
* This could be the case for a !-dl task that is boosted.
@@ -975,7 +975,7 @@ update_dl_revised_wakeup(struct sched_dl_entity *dl_se, struct rq *rq)
*
* See update_dl_entity() comments for further details.
*/
- WARN_ON(dl_time_before(dl_se->deadline, rq_clock(rq)));
+ WARN_ON_DEFERRED(dl_time_before(dl_se->deadline, rq_clock(rq)));
dl_se->runtime = (dl_se->dl_density * laxity) >> BW_SHIFT;
}
@@ -1080,7 +1080,7 @@ static int start_dl_timer(struct sched_dl_entity *dl_se)
* (current u > U).
*/
if (dl_se->dl_defer_armed) {
- WARN_ON_ONCE(!dl_se->dl_throttled);
+ WARN_ON_ONCE_DEFERRED(!dl_se->dl_throttled);
act = ns_to_ktime(dl_se->deadline - dl_se->runtime);
} else {
/* act = deadline - rel-deadline + period */
@@ -1451,7 +1451,7 @@ static void update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64
/*
* Non-servers would never get time accounted while throttled.
*/
- WARN_ON_ONCE(!dl_server(dl_se));
+ WARN_ON_ONCE_DEFERRED(!dl_server(dl_se));
/*
* While the server is marked idle, do not push out the
@@ -1492,7 +1492,7 @@ static void update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64
* and queue right away. Otherwise nothing might queue it. That's similar
* to what enqueue_dl_entity() does on start_dl_timer==0. For now, just warn.
*/
- WARN_ON_ONCE(!start_dl_timer(dl_se));
+ WARN_ON_ONCE_DEFERRED(!start_dl_timer(dl_se));
return;
}
@@ -1801,7 +1801,7 @@ void dl_server_start(struct sched_dl_entity *dl_se)
*/
rq->donor->sched_class->update_curr(rq);
- if (WARN_ON_ONCE(!cpu_online(cpu_of(rq))))
+ if (WARN_ON_ONCE_DEFERRED(!cpu_online(cpu_of(rq))))
return;
trace_sched_dl_server_start_tp(dl_se, cpu_of(rq), dl_get_type(dl_se, rq));
@@ -2073,7 +2073,7 @@ void inc_dl_tasks(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
static inline
void dec_dl_tasks(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq)
{
- WARN_ON(!dl_rq->dl_nr_running);
+ WARN_ON_DEFERRED(!dl_rq->dl_nr_running);
dl_rq->dl_nr_running--;
if (!dl_server(dl_se))
@@ -2165,7 +2165,7 @@ static void __enqueue_dl_entity(struct sched_dl_entity *dl_se)
{
struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
- WARN_ON_ONCE(!RB_EMPTY_NODE(&dl_se->rb_node));
+ WARN_ON_ONCE_DEFERRED(!RB_EMPTY_NODE(&dl_se->rb_node));
rb_add_cached(&dl_se->rb_node, &dl_rq->root, __dl_less);
@@ -2189,7 +2189,7 @@ static void __dequeue_dl_entity(struct sched_dl_entity *dl_se)
static void
enqueue_dl_entity(struct sched_dl_entity *dl_se, int flags)
{
- WARN_ON_ONCE(on_dl_rq(dl_se));
+ WARN_ON_ONCE_DEFERRED(on_dl_rq(dl_se));
update_stats_enqueue_dl(dl_rq_of_se(dl_se), dl_se, flags);
@@ -2611,7 +2611,7 @@ static struct task_struct *__pick_task_dl(struct rq *rq, struct rq_flags *rf)
return NULL;
dl_se = pick_next_dl_entity(dl_rq);
- WARN_ON_ONCE(!dl_se);
+ WARN_ON_ONCE_DEFERRED(!dl_se);
if (dl_server(dl_se)) {
p = dl_se->server_pick_task(dl_se, rf);
@@ -2823,12 +2823,12 @@ static struct task_struct *pick_next_pushable_dl_task(struct rq *rq)
if (!p)
return NULL;
- WARN_ON_ONCE(rq->cpu != task_cpu(p));
- WARN_ON_ONCE(task_current(rq, p));
- WARN_ON_ONCE(p->nr_cpus_allowed <= 1);
+ WARN_ON_ONCE_DEFERRED(rq->cpu != task_cpu(p));
+ WARN_ON_ONCE_DEFERRED(task_current(rq, p));
+ WARN_ON_ONCE_DEFERRED(p->nr_cpus_allowed <= 1);
- WARN_ON_ONCE(!task_on_rq_queued(p));
- WARN_ON_ONCE(!dl_task(p));
+ WARN_ON_ONCE_DEFERRED(!task_on_rq_queued(p));
+ WARN_ON_ONCE_DEFERRED(!dl_task(p));
return p;
}
@@ -2944,7 +2944,7 @@ static int push_dl_task(struct rq *rq)
if (is_migration_disabled(next_task))
return 0;
- if (WARN_ON(next_task == rq->curr))
+ if (WARN_ON_DEFERRED(next_task == rq->curr))
return 0;
/* We might release rq lock */
@@ -3050,8 +3050,8 @@ static void pull_dl_task(struct rq *this_rq)
*/
if (p && dl_time_before(p->dl.deadline, dmin) &&
dl_task_is_earliest_deadline(p, this_rq)) {
- WARN_ON(p == src_rq->curr);
- WARN_ON(!task_on_rq_queued(p));
+ WARN_ON_DEFERRED(p == src_rq->curr);
+ WARN_ON_DEFERRED(!task_on_rq_queued(p));
/*
* Then we pull iff p has actually an earlier
@@ -3109,7 +3109,7 @@ static void set_cpus_allowed_dl(struct task_struct *p,
{
struct rq *rq;
- WARN_ON_ONCE(!dl_task(p));
+ WARN_ON_ONCE_DEFERRED(!dl_task(p));
rq = task_rq(p);
/*
diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 5d2d19473a82e..47d3a4c16455a 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -512,12 +512,12 @@ do { \
* So if kf_tasks[] is set, @p's scheduler-protected fields are stable.
*
* kf_tasks[] can not stack, so task-based SCX ops must not nest. The
- * WARN_ON_ONCE() in each macro catches a re-entry of any of the three variants
- * while a previous one is still in progress.
+ * WARN_ON_ONCE_DEFERRED() in each macro catches a re-entry of any of the three
+ * variants while a previous one is still in progress.
*/
#define SCX_CALL_OP_TASK(sch, op, locked_rq, task, args...) \
do { \
- WARN_ON_ONCE(current->scx.kf_tasks[0]); \
+ WARN_ON_ONCE_DEFERRED(current->scx.kf_tasks[0]); \
current->scx.kf_tasks[0] = task; \
SCX_CALL_OP((sch), op, locked_rq, task, ##args); \
current->scx.kf_tasks[0] = NULL; \
@@ -526,7 +526,7 @@ do { \
#define SCX_CALL_OP_TASK_RET(sch, op, locked_rq, task, args...) \
({ \
__typeof__((sch)->ops.op(task, ##args)) __ret; \
- WARN_ON_ONCE(current->scx.kf_tasks[0]); \
+ WARN_ON_ONCE_DEFERRED(current->scx.kf_tasks[0]); \
current->scx.kf_tasks[0] = task; \
__ret = SCX_CALL_OP_RET((sch), op, locked_rq, task, ##args); \
current->scx.kf_tasks[0] = NULL; \
@@ -536,7 +536,7 @@ do { \
#define SCX_CALL_OP_2TASKS_RET(sch, op, locked_rq, task0, task1, args...) \
({ \
__typeof__((sch)->ops.op(task0, task1, ##args)) __ret; \
- WARN_ON_ONCE(current->scx.kf_tasks[0]); \
+ WARN_ON_ONCE_DEFERRED(current->scx.kf_tasks[0]); \
current->scx.kf_tasks[0] = task0; \
current->scx.kf_tasks[1] = task1; \
__ret = SCX_CALL_OP_RET((sch), op, locked_rq, task0, task1, ##args); \
@@ -687,7 +687,7 @@ static bool nldsq_cursor_lost_task(struct scx_dsq_list_node *cursor,
return true;
/* if @p has stayed on @dsq, its rq couldn't have changed */
- if (WARN_ON_ONCE(rq != task_rq(p)))
+ if (WARN_ON_ONCE_DEFERRED(rq != task_rq(p)))
return true;
return false;
@@ -1282,7 +1282,7 @@ static void schedule_reenq_local(struct rq *rq, u64 reenq_flags)
{
struct scx_sched *root = rcu_dereference_sched(scx_root);
- if (WARN_ON_ONCE(!root))
+ if (WARN_ON_ONCE_DEFERRED(!root))
return;
schedule_dsq_reenq(root, &rq->scx.local_dsq, reenq_flags, rq);
@@ -1379,7 +1379,7 @@ static void dsq_inc_nr(struct scx_dispatch_q *dsq, struct task_struct *p, u64 en
*/
if (enq_flags & SCX_ENQ_IMMED) {
if (unlikely(dsq->id != SCX_DSQ_LOCAL)) {
- WARN_ON_ONCE(!(enq_flags & SCX_ENQ_GDSQ_FALLBACK));
+ WARN_ON_ONCE_DEFERRED(!(enq_flags & SCX_ENQ_GDSQ_FALLBACK));
return;
}
p->scx.flags |= SCX_TASK_IMMED;
@@ -1388,7 +1388,7 @@ static void dsq_inc_nr(struct scx_dispatch_q *dsq, struct task_struct *p, u64 en
if (p->scx.flags & SCX_TASK_IMMED) {
struct rq *rq = container_of(dsq, struct rq, scx.local_dsq);
- if (WARN_ON_ONCE(dsq->id != SCX_DSQ_LOCAL))
+ if (WARN_ON_ONCE_DEFERRED(dsq->id != SCX_DSQ_LOCAL))
return;
rq->scx.nr_immed++;
@@ -1410,8 +1410,8 @@ static void dsq_dec_nr(struct scx_dispatch_q *dsq, struct task_struct *p)
if (p->scx.flags & SCX_TASK_IMMED) {
struct rq *rq = container_of(dsq, struct rq, scx.local_dsq);
- if (WARN_ON_ONCE(dsq->id != SCX_DSQ_LOCAL) ||
- WARN_ON_ONCE(rq->scx.nr_immed <= 0))
+ if (WARN_ON_ONCE_DEFERRED(dsq->id != SCX_DSQ_LOCAL) ||
+ WARN_ON_ONCE_DEFERRED(rq->scx.nr_immed <= 0))
return;
rq->scx.nr_immed--;
@@ -1521,9 +1521,9 @@ static void dispatch_enqueue(struct scx_sched *sch, struct rq *rq,
{
bool is_local = dsq->id == SCX_DSQ_LOCAL;
- WARN_ON_ONCE(p->scx.dsq || !list_empty(&p->scx.dsq_list.node));
- WARN_ON_ONCE((p->scx.dsq_flags & SCX_TASK_DSQ_ON_PRIQ) ||
- !RB_EMPTY_NODE(&p->scx.dsq_priq));
+ WARN_ON_ONCE_DEFERRED(p->scx.dsq || !list_empty(&p->scx.dsq_list.node));
+ WARN_ON_ONCE_DEFERRED((p->scx.dsq_flags & SCX_TASK_DSQ_ON_PRIQ) ||
+ !RB_EMPTY_NODE(&p->scx.dsq_priq));
if (!is_local) {
raw_spin_lock_nested(&dsq->lock,
@@ -1646,7 +1646,7 @@ static void dispatch_enqueue(struct scx_sched *sch, struct rq *rq,
static void task_unlink_from_dsq(struct task_struct *p,
struct scx_dispatch_q *dsq)
{
- WARN_ON_ONCE(list_empty(&p->scx.dsq_list.node));
+ WARN_ON_ONCE_DEFERRED(list_empty(&p->scx.dsq_list.node));
if (p->scx.dsq_flags & SCX_TASK_DSQ_ON_PRIQ) {
rb_erase(&p->scx.dsq_priq, &dsq->priq);
@@ -1709,7 +1709,7 @@ static void dispatch_dequeue(struct rq *rq, struct task_struct *p)
* holding_cpu which tells dispatch_to_local_dsq() that it lost
* the race.
*/
- WARN_ON_ONCE(!list_empty(&p->scx.dsq_list.node));
+ WARN_ON_ONCE_DEFERRED(!list_empty(&p->scx.dsq_list.node));
p->scx.holding_cpu = -1;
}
p->scx.dsq = NULL;
@@ -1787,8 +1787,8 @@ static void mark_direct_dispatch(struct scx_sched *sch,
return;
}
- WARN_ON_ONCE(p->scx.ddsp_dsq_id != SCX_DSQ_INVALID);
- WARN_ON_ONCE(p->scx.ddsp_enq_flags);
+ WARN_ON_ONCE_DEFERRED(p->scx.ddsp_dsq_id != SCX_DSQ_INVALID);
+ WARN_ON_ONCE_DEFERRED(p->scx.ddsp_enq_flags);
p->scx.ddsp_dsq_id = dsq_id;
p->scx.ddsp_enq_flags = enq_flags;
@@ -1855,7 +1855,7 @@ static void direct_dispatch(struct scx_sched *sch, struct task_struct *p,
break;
}
- WARN_ON_ONCE(p->scx.dsq || !list_empty(&p->scx.dsq_list.node));
+ WARN_ON_ONCE_DEFERRED(p->scx.dsq || !list_empty(&p->scx.dsq_list.node));
list_add_tail(&p->scx.dsq_list.node,
&rq->scx.ddsp_deferred_locals);
schedule_deferred_locked(rq);
@@ -1888,7 +1888,7 @@ static void do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags,
struct scx_dispatch_q *dsq;
unsigned long qseq;
- WARN_ON_ONCE(!(p->scx.flags & SCX_TASK_QUEUED));
+ WARN_ON_ONCE_DEFERRED(!(p->scx.flags & SCX_TASK_QUEUED));
/* internal movements - rq migration / RESTORE */
if (sticky_cpu == cpu_of(rq))
@@ -1938,11 +1938,11 @@ static void do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags,
/* DSQ bypass didn't trigger, enqueue on the BPF scheduler */
qseq = rq->scx.ops_qseq++ << SCX_OPSS_QSEQ_SHIFT;
- WARN_ON_ONCE(atomic_long_read(&p->scx.ops_state) != SCX_OPSS_NONE);
+ WARN_ON_ONCE_DEFERRED(atomic_long_read(&p->scx.ops_state) != SCX_OPSS_NONE);
atomic_long_set(&p->scx.ops_state, SCX_OPSS_QUEUEING | qseq);
ddsp_taskp = this_cpu_ptr(&direct_dispatch_task);
- WARN_ON_ONCE(*ddsp_taskp);
+ WARN_ON_ONCE_DEFERRED(*ddsp_taskp);
*ddsp_taskp = p;
SCX_CALL_OP_TASK(sch, enqueue, rq, p, enq_flags);
@@ -2039,7 +2039,7 @@ static void enqueue_task_scx(struct rq *rq, struct task_struct *p, int core_enq_
sticky_cpu = cpu_of(rq);
if (p->scx.flags & SCX_TASK_QUEUED) {
- WARN_ON_ONCE(!task_runnable(p));
+ WARN_ON_ONCE_DEFERRED(!task_runnable(p));
goto out;
}
@@ -2159,7 +2159,7 @@ static bool dequeue_task_scx(struct rq *rq, struct task_struct *p, int core_deq_
deq_flags |= SCX_DEQ_SCHED_CHANGE;
if (!(p->scx.flags & SCX_TASK_QUEUED)) {
- WARN_ON_ONCE(task_runnable(p));
+ WARN_ON_ONCE_DEFERRED(task_runnable(p));
return true;
}
@@ -2256,7 +2256,7 @@ static void move_local_task_to_local_dsq(struct scx_sched *sch,
lockdep_assert_held(&src_dsq->lock);
lockdep_assert_rq_held(dst_rq);
- WARN_ON_ONCE(p->scx.holding_cpu >= 0);
+ WARN_ON_ONCE_DEFERRED(p->scx.holding_cpu >= 0);
if (enq_flags & (SCX_ENQ_HEAD | SCX_ENQ_PREEMPT))
list_add(&p->scx.dsq_list.node, &dst_dsq->list);
@@ -2299,8 +2299,8 @@ static void move_remote_task_to_local_dsq(struct task_struct *p, u64 enq_flags,
* truncate the upper 32 bit. As we own @rq, we can pass them through
* @rq->scx.extra_enq_flags instead.
*/
- WARN_ON_ONCE(!cpumask_test_cpu(cpu_of(dst_rq), p->cpus_ptr));
- WARN_ON_ONCE(dst_rq->scx.extra_enq_flags);
+ WARN_ON_ONCE_DEFERRED(!cpumask_test_cpu(cpu_of(dst_rq), p->cpus_ptr));
+ WARN_ON_ONCE_DEFERRED(dst_rq->scx.extra_enq_flags);
dst_rq->scx.extra_enq_flags = enq_flags;
activate_task(dst_rq, p, 0);
dst_rq->scx.extra_enq_flags = 0;
@@ -2331,7 +2331,7 @@ static bool task_can_run_on_remote_rq(struct scx_sched *sch,
{
s32 cpu = cpu_of(rq);
- WARN_ON_ONCE(task_cpu(p) == cpu);
+ WARN_ON_ONCE_DEFERRED(task_cpu(p) == cpu);
/*
* If @p has migration disabled, @p->cpus_ptr is updated to contain only
@@ -2411,7 +2411,7 @@ static bool unlink_dsq_and_lock_src_rq(struct task_struct *p,
lockdep_assert_held(&dsq->lock);
- WARN_ON_ONCE(p->scx.holding_cpu >= 0);
+ WARN_ON_ONCE_DEFERRED(p->scx.holding_cpu >= 0);
task_unlink_from_dsq(p, dsq);
p->scx.holding_cpu = cpu;
@@ -2420,7 +2420,7 @@ static bool unlink_dsq_and_lock_src_rq(struct task_struct *p,
/* task_rq couldn't have changed if we're still the holding cpu */
return likely(p->scx.holding_cpu == cpu) &&
- !WARN_ON_ONCE(src_rq != task_rq(p));
+ !WARN_ON_ONCE_DEFERRED(src_rq != task_rq(p));
}
static bool consume_remote_task(struct rq *this_rq,
@@ -2630,7 +2630,7 @@ static void dispatch_to_local_dsq(struct scx_sched *sch, struct rq *rq,
/* task_rq couldn't have changed if we're still the holding cpu */
if (likely(p->scx.holding_cpu == raw_smp_processor_id()) &&
- !WARN_ON_ONCE(src_rq != task_rq(p))) {
+ !WARN_ON_ONCE_DEFERRED(src_rq != task_rq(p))) {
/*
* If @p is staying on the same rq, there's no need to go
* through the full deactivate/activate cycle. Optimize by
@@ -3099,7 +3099,7 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
* which should trigger an explicit follow-up scheduling event.
*/
if (next && sched_class_above(&ext_sched_class, next->sched_class)) {
- WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
+ WARN_ON_ONCE_DEFERRED(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
do_enqueue_task(rq, p, SCX_ENQ_LAST, -1);
} else {
do_enqueue_task(rq, p, 0, -1);
@@ -3201,7 +3201,7 @@ do_pick_task_scx(struct rq *rq, struct rq_flags *rf, bool force_scx)
keep_prev = rq->scx.flags & SCX_RQ_BAL_KEEP;
if (unlikely(keep_prev &&
prev->sched_class != &ext_sched_class)) {
- WARN_ON_ONCE(scx_enable_state() == SCX_ENABLED);
+ WARN_ON_ONCE_DEFERRED(scx_enable_state() == SCX_ENABLED);
keep_prev = false;
}
@@ -3332,7 +3332,7 @@ static int select_task_rq_scx(struct task_struct *p, int prev_cpu, int wake_flag
struct task_struct **ddsp_taskp;
ddsp_taskp = this_cpu_ptr(&direct_dispatch_task);
- WARN_ON_ONCE(*ddsp_taskp);
+ WARN_ON_ONCE_DEFERRED(*ddsp_taskp);
*ddsp_taskp = p;
this_rq()->scx.in_select_cpu = true;
@@ -3620,7 +3620,7 @@ static void __scx_enable_task(struct scx_sched *sch, struct task_struct *p)
* transitions are consistent, the flag should always be clear
* here.
*/
- WARN_ON_ONCE(p->scx.flags & SCX_TASK_IN_CUSTODY);
+ WARN_ON_ONCE_DEFERRED(p->scx.flags & SCX_TASK_IN_CUSTODY);
/*
* Set the weight before calling ops.enable() so that the scheduler
@@ -3651,7 +3651,7 @@ static void scx_disable_task(struct scx_sched *sch, struct task_struct *p)
struct rq *rq = task_rq(p);
lockdep_assert_rq_held(rq);
- WARN_ON_ONCE(scx_get_task_state(p) != SCX_TASK_ENABLED);
+ WARN_ON_ONCE_DEFERRED(scx_get_task_state(p) != SCX_TASK_ENABLED);
clear_direct_dispatch(p);
@@ -3664,7 +3664,7 @@ static void scx_disable_task(struct scx_sched *sch, struct task_struct *p)
* transitions are consistent, the flag should always be clear
* here.
*/
- WARN_ON_ONCE(p->scx.flags & SCX_TASK_IN_CUSTODY);
+ WARN_ON_ONCE_DEFERRED(p->scx.flags & SCX_TASK_IN_CUSTODY);
}
static void __scx_disable_and_exit_task(struct scx_sched *sch,
@@ -3689,7 +3689,7 @@ static void __scx_disable_and_exit_task(struct scx_sched *sch,
scx_disable_task(sch, p);
break;
default:
- WARN_ON_ONCE(true);
+ WARN_ON_ONCE_DEFERRED(true);
return;
}
@@ -3726,7 +3726,7 @@ static void scx_disable_and_exit_task(struct scx_sched *sch,
* path, so it's always clear when @p arrives here in %SCX_TASK_NONE.
*/
if (p->scx.flags & SCX_TASK_SUB_INIT) {
- if (!WARN_ON_ONCE(!scx_enabling_sub_sched))
+ if (!WARN_ON_ONCE_DEFERRED(!scx_enabling_sub_sched))
scx_sub_init_cancel_task(scx_enabling_sub_sched, p);
p->scx.flags &= ~SCX_TASK_SUB_INIT;
}
@@ -3818,7 +3818,7 @@ void scx_cancel_fork(struct task_struct *p)
struct rq_flags rf;
rq = task_rq_lock(p, &rf);
- WARN_ON_ONCE(scx_get_task_state(p) >= SCX_TASK_READY);
+ WARN_ON_ONCE_DEFERRED(scx_get_task_state(p) >= SCX_TASK_READY);
scx_disable_and_exit_task(scx_task_sched(p), p);
task_rq_unlock(rq, p, &rf);
}
@@ -3986,7 +3986,7 @@ static void process_ddsp_deferred_locals(struct rq *rq)
clear_direct_dispatch(p);
dsq = find_dsq_for_dispatch(sch, rq, dsq_id, task_cpu(p));
- if (!WARN_ON_ONCE(dsq->id != SCX_DSQ_LOCAL))
+ if (!WARN_ON_ONCE_DEFERRED(dsq->id != SCX_DSQ_LOCAL))
dispatch_to_local_dsq(sch, rq, dsq, p, enq_flags);
}
}
@@ -4041,7 +4041,7 @@ static u32 reenq_local(struct scx_sched *sch, struct rq *rq, u64 reenq_flags)
lockdep_assert_rq_held(rq);
- if (WARN_ON_ONCE(reenq_flags & __SCX_REENQ_TSR_MASK))
+ if (WARN_ON_ONCE_DEFERRED(reenq_flags & __SCX_REENQ_TSR_MASK))
reenq_flags &= ~__SCX_REENQ_TSR_MASK;
if (rq_is_open(rq, 0))
reenq_flags |= SCX_REENQ_TSR_RQ_OPEN;
@@ -4078,7 +4078,7 @@ static u32 reenq_local(struct scx_sched *sch, struct rq *rq, u64 reenq_flags)
dispatch_dequeue(rq, p);
- if (WARN_ON_ONCE(p->scx.flags & SCX_TASK_REENQ_REASON_MASK))
+ if (WARN_ON_ONCE_DEFERRED(p->scx.flags & SCX_TASK_REENQ_REASON_MASK))
p->scx.flags &= ~SCX_TASK_REENQ_REASON_MASK;
p->scx.flags |= reason;
@@ -4199,7 +4199,7 @@ static void reenq_user(struct rq *rq, struct scx_dispatch_q *dsq, u64 reenq_flag
dispatch_dequeue_locked(p, dsq);
raw_spin_unlock(&dsq->lock);
- if (WARN_ON_ONCE(p->scx.flags & SCX_TASK_REENQ_REASON_MASK))
+ if (WARN_ON_ONCE_DEFERRED(p->scx.flags & SCX_TASK_REENQ_REASON_MASK))
p->scx.flags &= ~SCX_TASK_REENQ_REASON_MASK;
p->scx.flags |= reason;
@@ -4360,7 +4360,7 @@ int scx_cgroup_can_attach(struct cgroup_taskset *tset)
struct cgroup *from = tg_cgrp(task_group(p));
struct cgroup *to = tg_cgrp(css_tg(css));
- WARN_ON_ONCE(p->scx.cgrp_moving_from);
+ WARN_ON_ONCE_DEFERRED(p->scx.cgrp_moving_from);
/*
* sched_move_task() omits identity migrations. Let's match the
@@ -4617,7 +4617,7 @@ static void exit_dsq(struct scx_dispatch_q *dsq)
* There must have been a RCU grace period since the last
* insertion and @dsq should be off the deferred list by now.
*/
- if (WARN_ON_ONCE(!list_empty(&dru->node))) {
+ if (WARN_ON_ONCE_DEFERRED(!list_empty(&dru->node))) {
guard(raw_spinlock_irqsave)(&rq->scx.deferred_reenq_lock);
list_del_init(&dru->node);
}
@@ -4745,7 +4745,7 @@ static int scx_cgroup_init(struct scx_sched *sch)
tg->scx.flags |= SCX_TG_INITED;
}
- WARN_ON_ONCE(scx_cgroup_enabled);
+ WARN_ON_ONCE_DEFERRED(scx_cgroup_enabled);
scx_cgroup_enabled = true;
return 0;
@@ -4848,7 +4848,7 @@ static void scx_sched_free_rcu_work(struct work_struct *work)
* period. As that blocks new deferrals, all
* deferred_reenq_local_node's must be off-list by now.
*/
- WARN_ON_ONCE(!list_empty(&pcpu->deferred_reenq_local.node));
+ WARN_ON_ONCE_DEFERRED(!list_empty(&pcpu->deferred_reenq_local.node));
exit_dsq(bypass_dsq(sch, cpu));
}
@@ -5324,7 +5324,7 @@ static bool inc_bypass_depth(struct scx_sched *sch)
{
lockdep_assert_held(&scx_bypass_lock);
- WARN_ON_ONCE(sch->bypass_depth < 0);
+ WARN_ON_ONCE_DEFERRED(sch->bypass_depth < 0);
WRITE_ONCE(sch->bypass_depth, sch->bypass_depth + 1);
if (sch->bypass_depth != 1)
return false;
@@ -5339,7 +5339,7 @@ static bool dec_bypass_depth(struct scx_sched *sch)
{
lockdep_assert_held(&scx_bypass_lock);
- WARN_ON_ONCE(sch->bypass_depth < 1);
+ WARN_ON_ONCE_DEFERRED(sch->bypass_depth < 1);
WRITE_ONCE(sch->bypass_depth, sch->bypass_depth - 1);
if (sch->bypass_depth != 0)
return false;
@@ -5360,7 +5360,7 @@ static void enable_bypass_dsp(struct scx_sched *sch)
* @sch->bypass_depth transitioning from 0 to 1 triggers enabling.
* Shouldn't stagger.
*/
- if (WARN_ON_ONCE(test_and_set_bit(0, &sch->bypass_dsp_claim)))
+ if (WARN_ON_ONCE_DEFERRED(test_and_set_bit(0, &sch->bypass_dsp_claim)))
return;
/*
@@ -5380,11 +5380,11 @@ static void enable_bypass_dsp(struct scx_sched *sch)
* Bump enable depth on both @sch and bypass dispatch host.
*/
ret = atomic_inc_return(&sch->bypass_dsp_enable_depth);
- WARN_ON_ONCE(ret <= 0);
+ WARN_ON_ONCE_DEFERRED(ret <= 0);
if (host != sch) {
ret = atomic_inc_return(&host->bypass_dsp_enable_depth);
- WARN_ON_ONCE(ret <= 0);
+ WARN_ON_ONCE_DEFERRED(ret <= 0);
}
/*
@@ -5405,11 +5405,11 @@ static void disable_bypass_dsp(struct scx_sched *sch)
return;
ret = atomic_dec_return(&sch->bypass_dsp_enable_depth);
- WARN_ON_ONCE(ret < 0);
+ WARN_ON_ONCE_DEFERRED(ret < 0);
if (scx_parent(sch)) {
ret = atomic_dec_return(&scx_parent(sch)->bypass_dsp_enable_depth);
- WARN_ON_ONCE(ret < 0);
+ WARN_ON_ONCE_DEFERRED(ret < 0);
}
}
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 3ebec186f9823..1213e77665fe9 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -404,7 +404,7 @@ static inline void list_del_leaf_cfs_rq(struct cfs_rq *cfs_rq)
static inline void assert_list_leaf_cfs_rq(struct rq *rq)
{
- WARN_ON_ONCE(rq->tmp_alone_branch != &rq->leaf_cfs_rq_list);
+ WARN_ON_ONCE_DEFERRED(rq->tmp_alone_branch != &rq->leaf_cfs_rq_list);
}
/* Iterate through all leaf cfs_rq's on a runqueue */
@@ -689,7 +689,7 @@ __sum_w_vruntime_add(struct cfs_rq *cfs_rq, struct sched_entity *se)
s64 w_vruntime, key = entity_key(cfs_rq, se);
w_vruntime = key * weight;
- WARN_ON_ONCE((w_vruntime >> 63) != (w_vruntime >> 62));
+ WARN_ON_ONCE_DEFERRED((w_vruntime >> 63) != (w_vruntime >> 62));
cfs_rq->sum_w_vruntime += w_vruntime;
cfs_rq->sum_weight += weight;
@@ -861,7 +861,7 @@ bool update_entity_lag(struct cfs_rq *cfs_rq, struct sched_entity *se)
u64 avruntime = avg_vruntime(cfs_rq);
s64 vlag = entity_lag(cfs_rq, se, avruntime);
- WARN_ON_ONCE(!se->on_rq);
+ WARN_ON_ONCE_DEFERRED(!se->on_rq);
if (se->sched_delayed) {
/* previous vlag < 0 otherwise se would not be delayed */
@@ -1153,7 +1153,7 @@ static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq, bool protect)
if (sched_feat(PICK_BUDDY) && protect &&
cfs_rq->next && entity_eligible(cfs_rq, cfs_rq->next)) {
/* ->next will never be delayed */
- WARN_ON_ONCE(cfs_rq->next->sched_delayed);
+ WARN_ON_ONCE_DEFERRED(cfs_rq->next->sched_delayed);
return cfs_rq->next;
}
@@ -4302,9 +4302,9 @@ static inline bool load_avg_is_decayed(struct sched_avg *sa)
* Make sure that rounding and/or propagation of PELT values never
* break this.
*/
- WARN_ON_ONCE(sa->load_avg ||
- sa->util_avg ||
- sa->runnable_avg);
+ WARN_ON_ONCE_DEFERRED(sa->load_avg ||
+ sa->util_avg ||
+ sa->runnable_avg);
return true;
}
@@ -5460,7 +5460,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
weight = avg_vruntime_weight(cfs_rq, se->load.weight);
lag *= load + weight;
- if (WARN_ON_ONCE(!load))
+ if (WARN_ON_ONCE_DEFERRED(!load))
load = 1;
lag = div64_long(lag, load);
@@ -5653,7 +5653,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
clear_buddies(cfs_rq, se);
if (flags & DEQUEUE_DELAYED) {
- WARN_ON_ONCE(!se->sched_delayed);
+ WARN_ON_ONCE_DEFERRED(!se->sched_delayed);
} else {
bool delay = sleep;
/*
@@ -5663,7 +5663,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
if (flags & (DEQUEUE_SPECIAL | DEQUEUE_THROTTLE))
delay = false;
- WARN_ON_ONCE(delay && se->sched_delayed);
+ WARN_ON_ONCE_DEFERRED(delay && se->sched_delayed);
if (sched_feat(DELAY_DEQUEUE) && delay &&
!entity_eligible(cfs_rq, se)) {
@@ -5747,7 +5747,7 @@ set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, bool first)
}
update_stats_curr_start(cfs_rq, se);
- WARN_ON_ONCE(cfs_rq->curr);
+ WARN_ON_ONCE_DEFERRED(cfs_rq->curr);
cfs_rq->curr = se;
/*
@@ -5814,7 +5814,7 @@ static void put_prev_entity(struct cfs_rq *cfs_rq, struct sched_entity *prev)
/* in !on_rq case, update occurred at dequeue */
update_load_avg(cfs_rq, prev, 0);
}
- WARN_ON_ONCE(cfs_rq->curr != prev);
+ WARN_ON_ONCE_DEFERRED(cfs_rq->curr != prev);
cfs_rq->curr = NULL;
}
@@ -6015,7 +6015,7 @@ static void throttle_cfs_rq_work(struct callback_head *work)
struct cfs_rq *cfs_rq;
struct rq *rq;
- WARN_ON_ONCE(p != current);
+ WARN_ON_ONCE_DEFERRED(p != current);
p->sched_throttle_work.next = &p->sched_throttle_work;
/*
@@ -6041,7 +6041,7 @@ static void throttle_cfs_rq_work(struct callback_head *work)
return;
rq = scope.rq;
update_rq_clock(rq);
- WARN_ON_ONCE(p->throttled || !list_empty(&p->throttle_node));
+ WARN_ON_ONCE_DEFERRED(p->throttled || !list_empty(&p->throttle_node));
dequeue_task_fair(rq, p, DEQUEUE_SLEEP | DEQUEUE_THROTTLE);
list_add(&p->throttle_node, &cfs_rq->throttled_limbo_list);
/*
@@ -6072,7 +6072,7 @@ void init_cfs_throttle_work(struct task_struct *p)
static void detach_task_cfs_rq(struct task_struct *p);
static void dequeue_throttled_task(struct task_struct *p, int flags)
{
- WARN_ON_ONCE(p->se.on_rq);
+ WARN_ON_ONCE_DEFERRED(p->se.on_rq);
list_del_init(&p->throttle_node);
/* task blocked after throttled */
@@ -6094,7 +6094,7 @@ static bool enqueue_throttled_task(struct task_struct *p)
struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
/* @p should have gone through dequeue_throttled_task() first */
- WARN_ON_ONCE(!list_empty(&p->throttle_node));
+ WARN_ON_ONCE_DEFERRED(!list_empty(&p->throttle_node));
/*
* If the throttled task @p is enqueued to a throttled cfs_rq,
@@ -6162,7 +6162,7 @@ static int tg_unthrottle_up(struct task_group *tg, void *data)
cfs_rq->throttled_clock_self = 0;
- if (WARN_ON_ONCE((s64)delta < 0))
+ if (WARN_ON_ONCE_DEFERRED((s64)delta < 0))
delta = 0;
cfs_rq->throttled_clock_self_time += delta;
@@ -6231,8 +6231,8 @@ static int tg_throttle_down(struct task_group *tg, void *data)
cfs_rq->pelt_clock_throttled = 1;
}
- WARN_ON_ONCE(cfs_rq->throttled_clock_self);
- WARN_ON_ONCE(!list_empty(&cfs_rq->throttled_limbo_list));
+ WARN_ON_ONCE_DEFERRED(cfs_rq->throttled_clock_self);
+ WARN_ON_ONCE_DEFERRED(!list_empty(&cfs_rq->throttled_limbo_list));
return 0;
}
@@ -6273,7 +6273,7 @@ static bool throttle_cfs_rq(struct cfs_rq *cfs_rq)
* throttled-list. rq->lock protects completion.
*/
cfs_rq->throttled = 1;
- WARN_ON_ONCE(cfs_rq->throttled_clock);
+ WARN_ON_ONCE_DEFERRED(cfs_rq->throttled_clock);
return true;
}
@@ -6380,7 +6380,7 @@ static inline void __unthrottle_cfs_rq_async(struct cfs_rq *cfs_rq)
}
/* Already enqueued */
- if (WARN_ON_ONCE(!list_empty(&cfs_rq->throttled_csd_list)))
+ if (WARN_ON_ONCE_DEFERRED(!list_empty(&cfs_rq->throttled_csd_list)))
return;
first = list_empty(&rq->cfsb_csd_list);
@@ -6393,7 +6393,7 @@ static void unthrottle_cfs_rq_async(struct cfs_rq *cfs_rq)
{
lockdep_assert_rq_held(rq_of(cfs_rq));
- if (WARN_ON_ONCE(!cfs_rq_throttled(cfs_rq) ||
+ if (WARN_ON_ONCE_DEFERRED(!cfs_rq_throttled(cfs_rq) ||
cfs_rq->runtime_remaining <= 0))
return;
@@ -6429,7 +6429,7 @@ static bool distribute_cfs_runtime(struct cfs_bandwidth *cfs_b)
goto next;
/* By the above checks, this should never be true */
- WARN_ON_ONCE(cfs_rq->runtime_remaining > 0);
+ WARN_ON_ONCE_DEFERRED(cfs_rq->runtime_remaining > 0);
raw_spin_lock(&cfs_b->lock);
runtime = -cfs_rq->runtime_remaining + 1;
@@ -6450,7 +6450,7 @@ static bool distribute_cfs_runtime(struct cfs_bandwidth *cfs_b)
* We currently only expect to be unthrottling
* a single cfs_rq locally.
*/
- WARN_ON_ONCE(!list_empty(&local_unthrottle));
+ WARN_ON_ONCE_DEFERRED(!list_empty(&local_unthrottle));
list_add_tail(&cfs_rq->throttled_csd_list,
&local_unthrottle);
}
@@ -6475,7 +6475,7 @@ static bool distribute_cfs_runtime(struct cfs_bandwidth *cfs_b)
rq_unlock_irqrestore(rq, &rf);
}
- WARN_ON_ONCE(!list_empty(&local_unthrottle));
+ WARN_ON_ONCE_DEFERRED(!list_empty(&local_unthrottle));
rcu_read_unlock();
@@ -7048,7 +7048,7 @@ static void hrtick_start_fair(struct rq *rq, struct task_struct *p)
u64 vdelta;
u64 delta;
- WARN_ON_ONCE(task_rq(p) != rq);
+ WARN_ON_ONCE_DEFERRED(task_rq(p) != rq);
if (rq->cfs.h_nr_queued <= 1)
return;
@@ -7171,8 +7171,8 @@ requeue_delayed_entity(struct sched_entity *se)
* Because a delayed entity is one that is still on
* the runqueue competing until elegibility.
*/
- WARN_ON_ONCE(!se->sched_delayed);
- WARN_ON_ONCE(!se->on_rq);
+ WARN_ON_ONCE_DEFERRED(!se->sched_delayed);
+ WARN_ON_ONCE_DEFERRED(!se->on_rq);
if (update_entity_lag(cfs_rq, se)) {
cfs_rq->nr_queued--;
@@ -7409,8 +7409,8 @@ static int dequeue_entities(struct rq *rq, struct sched_entity *se, int flags)
rq->next_balance = jiffies;
if (p && task_delayed) {
- WARN_ON_ONCE(!task_sleep);
- WARN_ON_ONCE(p->on_rq != 1);
+ WARN_ON_ONCE_DEFERRED(!task_sleep);
+ WARN_ON_ONCE_DEFERRED(p->on_rq != 1);
/*
* Fix-up what block_task() skipped.
@@ -8976,7 +8976,7 @@ static void set_cpus_allowed_fair(struct task_struct *p, struct affinity_context
static void set_next_buddy(struct sched_entity *se)
{
for_each_sched_entity(se) {
- if (WARN_ON_ONCE(!se->on_rq))
+ if (WARN_ON_ONCE_DEFERRED(!se->on_rq))
return;
if (se_is_idle(se))
return;
@@ -9023,7 +9023,7 @@ preempt_sync(struct rq *rq, int wake_flags,
* WF_SYNC without WF_TTWU is not expected so warn if it happens even
* though it is likely harmless.
*/
- WARN_ON_ONCE(!(wake_flags & WF_TTWU));
+ WARN_ON_ONCE_DEFERRED(!(wake_flags & WF_TTWU));
threshold = sysctl_sched_migration_cost;
delta = rq_clock_task(rq) - se->exec_start;
@@ -9095,7 +9095,7 @@ static void wakeup_preempt_fair(struct rq *rq, struct task_struct *p, int wake_f
return;
find_matching_se(&se, &pse);
- WARN_ON_ONCE(!pse);
+ WARN_ON_ONCE_DEFERRED(!pse);
cse_is_idle = se_is_idle(se);
pse_is_idle = se_is_idle(pse);
@@ -9857,8 +9857,8 @@ static void detach_task(struct task_struct *p, struct lb_env *env)
schedstat_inc(p->stats.nr_forced_migrations);
}
- WARN_ON(task_current(env->src_rq, p));
- WARN_ON(task_current_donor(env->src_rq, p));
+ WARN_ON_DEFERRED(task_current(env->src_rq, p));
+ WARN_ON_DEFERRED(task_current_donor(env->src_rq, p));
deactivate_task(env->src_rq, p, DEQUEUE_NOCLOCK);
set_task_cpu(p, env->dst_cpu);
@@ -12151,7 +12151,7 @@ static int sched_balance_rq(int this_cpu, struct rq *this_rq,
goto out_balanced;
}
- WARN_ON_ONCE(busiest == env.dst_rq);
+ WARN_ON_ONCE_DEFERRED(busiest == env.dst_rq);
update_lb_imbalance_stat(&env, sd, idle);
@@ -12461,7 +12461,7 @@ static int active_load_balance_cpu_stop(void *data)
* we need to fix it. Originally reported by
* Bjorn Helgaas on a 128-CPU setup.
*/
- WARN_ON_ONCE(busiest_rq == target_rq);
+ WARN_ON_ONCE_DEFERRED(busiest_rq == target_rq);
/* Search for an sd spanning us and the target CPU. */
rcu_read_lock();
@@ -12883,7 +12883,7 @@ static void set_cpu_sd_state_busy(int cpu)
void nohz_balance_exit_idle(struct rq *rq)
{
- WARN_ON_ONCE(rq != this_rq());
+ WARN_ON_ONCE_DEFERRED(rq != this_rq());
if (likely(!rq->nohz_tick_stopped))
return;
@@ -12918,7 +12918,7 @@ void nohz_balance_enter_idle(int cpu)
{
struct rq *rq = cpu_rq(cpu);
- WARN_ON_ONCE(cpu != smp_processor_id());
+ WARN_ON_ONCE_DEFERRED(cpu != smp_processor_id());
/* If this CPU is going down, then nothing needs to be done: */
if (!cpu_active(cpu))
@@ -13000,7 +13000,7 @@ static void _nohz_idle_balance(struct rq *this_rq, unsigned int flags)
int balance_cpu;
struct rq *rq;
- WARN_ON_ONCE((flags & NOHZ_KICK_MASK) == NOHZ_BALANCE_KICK);
+ WARN_ON_ONCE_DEFERRED((flags & NOHZ_KICK_MASK) == NOHZ_BALANCE_KICK);
/*
* We assume there will be no idle load after this update and clear
@@ -13623,7 +13623,7 @@ bool cfs_prio_less(const struct task_struct *a, const struct task_struct *b,
struct cfs_rq *cfs_rqb;
s64 delta;
- WARN_ON_ONCE(task_rq(b)->core != rq->core);
+ WARN_ON_ONCE_DEFERRED(task_rq(b)->core != rq->core);
#ifdef CONFIG_FAIR_GROUP_SCHED
/*
@@ -13839,7 +13839,7 @@ static void switched_from_fair(struct rq *rq, struct task_struct *p)
static void switched_to_fair(struct rq *rq, struct task_struct *p)
{
- WARN_ON_ONCE(p->se.sched_delayed);
+ WARN_ON_ONCE_DEFERRED(p->se.sched_delayed);
attach_task_cfs_rq(p);
@@ -13872,7 +13872,7 @@ static void __set_next_task_fair(struct rq *rq, struct task_struct *p, bool firs
if (!first)
return;
- WARN_ON_ONCE(se->sched_delayed);
+ WARN_ON_ONCE_DEFERRED(se->sched_delayed);
if (hrtick_enabled_fair(rq))
hrtick_start_fair(rq, p);
@@ -14148,7 +14148,7 @@ int sched_group_set_idle(struct task_group *tg, long idle)
rq_lock_irqsave(rq, &rf);
grp_cfs_rq->idle = idle;
- if (WARN_ON_ONCE(was_idle == cfs_rq_is_idle(grp_cfs_rq)))
+ if (WARN_ON_ONCE_DEFERRED(was_idle == cfs_rq_is_idle(grp_cfs_rq)))
goto next_cpu;
idle_task_delta = grp_cfs_rq->h_nr_queued -
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 4ee8faf01441a..506d0f1afa58f 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -170,7 +170,7 @@ static void destroy_rt_bandwidth(struct rt_bandwidth *rt_b)
static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se)
{
- WARN_ON_ONCE(!rt_entity_is_task(rt_se));
+ WARN_ON_ONCE_DEFERRED(!rt_entity_is_task(rt_se));
return container_of(rt_se, struct task_struct, rt);
}
@@ -178,13 +178,13 @@ static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se)
static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq)
{
/* Cannot fold with non-CONFIG_RT_GROUP_SCHED version, layout */
- WARN_ON(!rt_group_sched_enabled() && rt_rq->tg != &root_task_group);
+ WARN_ON_DEFERRED(!rt_group_sched_enabled() && rt_rq->tg != &root_task_group);
return rt_rq->rq;
}
static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se)
{
- WARN_ON(!rt_group_sched_enabled() && rt_se->rt_rq->tg != &root_task_group);
+ WARN_ON_DEFERRED(!rt_group_sched_enabled() && rt_se->rt_rq->tg != &root_task_group);
return rt_se->rt_rq;
}
@@ -192,7 +192,7 @@ static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se)
{
struct rt_rq *rt_rq = rt_se->rt_rq;
- WARN_ON(!rt_group_sched_enabled() && rt_rq->tg != &root_task_group);
+ WARN_ON_DEFERRED(!rt_group_sched_enabled() && rt_rq->tg != &root_task_group);
return rt_rq->rq;
}
@@ -493,7 +493,7 @@ typedef struct task_group *rt_rq_iter_t;
static inline struct task_group *next_task_group(struct task_group *tg)
{
if (!rt_group_sched_enabled()) {
- WARN_ON(tg != &root_task_group);
+ WARN_ON_DEFERRED(tg != &root_task_group);
return NULL;
}
@@ -723,7 +723,7 @@ static void __disable_runtime(struct rq *rq)
* We cannot be left wanting - that would mean some runtime
* leaked out of the system.
*/
- WARN_ON_ONCE(want);
+ WARN_ON_ONCE_DEFERRED(want);
balanced:
/*
* Disable all the borrow logic by pretending we have inf
@@ -1094,7 +1094,7 @@ dec_rt_prio(struct rt_rq *rt_rq, int prio)
if (rt_rq->rt_nr_running) {
- WARN_ON(prio < prev_prio);
+ WARN_ON_DEFERRED(prio < prev_prio);
/*
* This may have been our highest task, and therefore
@@ -1131,7 +1131,7 @@ dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
if (rt_se_boosted(rt_se))
rt_rq->rt_nr_boosted--;
- WARN_ON(!rt_rq->rt_nr_running && rt_rq->rt_nr_boosted);
+ WARN_ON_DEFERRED(!rt_rq->rt_nr_running && rt_rq->rt_nr_boosted);
}
#else /* !CONFIG_RT_GROUP_SCHED: */
@@ -1176,7 +1176,7 @@ void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
{
int prio = rt_se_prio(rt_se);
- WARN_ON(!rt_prio(prio));
+ WARN_ON_DEFERRED(!rt_prio(prio));
rt_rq->rt_nr_running += rt_se_nr_running(rt_se);
rt_rq->rr_nr_running += rt_se_rr_nr_running(rt_se);
@@ -1187,8 +1187,8 @@ void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
static inline
void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq)
{
- WARN_ON(!rt_prio(rt_se_prio(rt_se)));
- WARN_ON(!rt_rq->rt_nr_running);
+ WARN_ON_DEFERRED(!rt_prio(rt_se_prio(rt_se)));
+ WARN_ON_DEFERRED(!rt_rq->rt_nr_running);
rt_rq->rt_nr_running -= rt_se_nr_running(rt_se);
rt_rq->rr_nr_running -= rt_se_rr_nr_running(rt_se);
@@ -1348,7 +1348,7 @@ static void __enqueue_rt_entity(struct sched_rt_entity *rt_se, unsigned int flag
}
if (move_entity(flags)) {
- WARN_ON_ONCE(rt_se->on_list);
+ WARN_ON_ONCE_DEFERRED(rt_se->on_list);
if (flags & ENQUEUE_HEAD)
list_add(&rt_se->run_list, queue);
else
@@ -1368,7 +1368,7 @@ static void __dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int flag
struct rt_prio_array *array = &rt_rq->active;
if (move_entity(flags)) {
- WARN_ON_ONCE(!rt_se->on_list);
+ WARN_ON_ONCE_DEFERRED(!rt_se->on_list);
__delist_rt_entity(rt_se, array);
}
rt_se->on_rq = 0;
@@ -1684,7 +1684,7 @@ static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq)
BUG_ON(idx >= MAX_RT_PRIO);
queue = array->queue + idx;
- if (WARN_ON_ONCE(list_empty(queue)))
+ if (WARN_ON_ONCE_DEFERRED(list_empty(queue)))
return NULL;
next = list_entry(queue->next, struct sched_rt_entity, run_list);
@@ -2016,7 +2016,7 @@ static int push_rt_task(struct rq *rq, bool pull)
return 0;
}
- if (WARN_ON(next_task == rq->curr))
+ if (WARN_ON_DEFERRED(next_task == rq->curr))
return 0;
/* We might release rq lock */
@@ -2316,8 +2316,8 @@ static void pull_rt_task(struct rq *this_rq)
* the to-be-scheduled task?
*/
if (p && (p->prio < this_rq->rt.highest_prio.curr)) {
- WARN_ON(p == src_rq->curr);
- WARN_ON(!task_on_rq_queued(p));
+ WARN_ON_DEFERRED(p == src_rq->curr);
+ WARN_ON_DEFERRED(!task_on_rq_queued(p));
/*
* There's a chance that p is higher in priority
@@ -2583,7 +2583,7 @@ static int task_is_throttled_rt(struct task_struct *p, int cpu)
#ifdef CONFIG_RT_GROUP_SCHED // XXX maybe add task_rt_rq(), see also sched_rt_period_rt_rq
rt_rq = task_group(p)->rt_rq[cpu];
- WARN_ON(!rt_group_sched_enabled() && rt_rq->tg != &root_task_group);
+ WARN_ON_DEFERRED(!rt_group_sched_enabled() && rt_rq->tg != &root_task_group);
#else
rt_rq = &cpu_rq(cpu)->rt;
#endif
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 9f63b15d309d1..f74f9cd44e098 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1684,7 +1684,7 @@ static inline void update_idle_core(struct rq *rq) { }
static inline struct task_struct *task_of(struct sched_entity *se)
{
- WARN_ON_ONCE(!entity_is_task(se));
+ WARN_ON_ONCE_DEFERRED(!entity_is_task(se));
return container_of(se, struct task_struct, se);
}
@@ -1766,7 +1766,7 @@ static inline void assert_clock_updated(struct rq *rq)
* The only reason for not seeing a clock update since the
* last rq_pin_lock() is if we're currently skipping updates.
*/
- WARN_ON_ONCE(rq->clock_update_flags < RQCF_ACT_SKIP);
+ WARN_ON_ONCE_DEFERRED(rq->clock_update_flags < RQCF_ACT_SKIP);
}
static inline u64 rq_clock(struct rq *rq)
@@ -1813,7 +1813,7 @@ static inline void rq_clock_cancel_skipupdate(struct rq *rq)
static inline void rq_clock_start_loop_update(struct rq *rq)
{
lockdep_assert_rq_held(rq);
- WARN_ON_ONCE(rq->clock_update_flags & RQCF_ACT_SKIP);
+ WARN_ON_ONCE_DEFERRED(rq->clock_update_flags & RQCF_ACT_SKIP);
rq->clock_update_flags |= RQCF_ACT_SKIP;
}
@@ -1870,9 +1870,9 @@ static inline void scx_rq_clock_invalidate(struct rq *rq) {}
static inline void assert_balance_callbacks_empty(struct rq *rq)
{
- WARN_ON_ONCE(IS_ENABLED(CONFIG_PROVE_LOCKING) &&
- rq->balance_callback &&
- rq->balance_callback != &balance_push_callback);
+ WARN_ON_ONCE_DEFERRED(IS_ENABLED(CONFIG_PROVE_LOCKING) &&
+ rq->balance_callback &&
+ rq->balance_callback != &balance_push_callback);
}
/*
@@ -2681,7 +2681,7 @@ struct sched_class {
static inline void put_prev_task(struct rq *rq, struct task_struct *prev)
{
- WARN_ON_ONCE(rq->donor != prev);
+ WARN_ON_ONCE_DEFERRED(rq->donor != prev);
prev->sched_class->put_prev_task(rq, prev, NULL);
}
@@ -2704,7 +2704,7 @@ static inline void put_prev_set_next_task(struct rq *rq,
struct task_struct *prev,
struct task_struct *next)
{
- WARN_ON_ONCE(rq->donor != prev);
+ WARN_ON_ONCE_DEFERRED(rq->donor != prev);
__put_prev_set_next_dl_server(rq, prev, next);
@@ -3030,7 +3030,7 @@ static inline void attach_task(struct rq *rq, struct task_struct *p)
{
lockdep_assert_rq_held(rq);
- WARN_ON_ONCE(task_rq(p) != rq);
+ WARN_ON_ONCE_DEFERRED(task_rq(p) != rq);
activate_task(rq, p, ENQUEUE_NOCLOCK);
wakeup_preempt(rq, p, 0);
}
--
2.53.0
^ permalink raw reply related
* [PATCH 0/2] sched: Introduce and use deferred WARNs in sched
From: Sebastian Andrzej Siewior @ 2026-06-23 14:26 UTC (permalink / raw)
To: linux-arch, linux-kernel, sched-ext, netdev
Cc: David S . Miller, Andrea Righi, Andrew Morton, Arnd Bergmann,
Ben Segall, Breno Leitao, Changwoo Min, David Vernet,
Dietmar Eggemann, Eric Dumazet, Ingo Molnar, Jakub Kicinski,
John Ogness, Juri Lelli, K Prateek Nayak, Paolo Abeni,
Peter Zijlstra, Petr Mladek, Sergey Senozhatsky, Simon Horman,
Steven Rostedt, Tejun Heo, Vincent Guittot, Vlad Poenaru,
Sebastian Andrzej Siewior
This is a follow-up to the netconsole lockup reported
https://lore.kernel.org/all/20260610183621.3915271-1-vlad.wing@gmail.com/
The idea is to use deferred printing for WARNs and use them in sched. I
tried to use only where it looks that the rq lock acquired instead a
plain s/WARN_ON/WARN_ON_DEFFERED which would be simpler.
This unholy deferred mess can be removed once we don't have legacy
consoles anymore _or_ force force_legacy_kthread=true.
The initial report is against v6.16 and netconsole. The reported problem
does not occur upstream since commit 7eab73b18630e ("netconsole: convert
to NBCON console infrastructure") which is v7.0-rc1.
Should this be rejected outright because the preferred sollution is to
| - stick msg in buffer (lockless)
| - print to atomic consoles (lockless)
| - use irq_work to wake console kthreads (lockless)
| - each kthread then tries to flush buffer to its own non-atomic console
| in non-atomic context."
then this means to force force_legacy_kthread=true.
The threaded legacy printer is available since v6.12-rc1. It terms of stable
fix, this could go back as of v6.12 stable and not earlier (in case we care).
I tested this on a x86 box with 8250 and warning in put_prev_entity().
After it printed the initial warning, it dead-locked shortly after
because systemd was writing to the kernel buffer it acquired the
uart_port_lock then attempted to write lockdep report which required the
same lock…
Sebastian Andrzej Siewior (2):
bug: Provide WARN_ON.*DEFERRED() macros for console deferred output
sched: Use WARN_ON.*_DEFERRED()
include/asm-generic/bug.h | 41 ++++++++++++++
kernel/sched/core.c | 78 +++++++++++++-------------
kernel/sched/core_sched.c | 6 +-
kernel/sched/cpudeadline.c | 6 +-
kernel/sched/deadline.c | 62 ++++++++++-----------
kernel/sched/ext.c | 110 ++++++++++++++++++-------------------
kernel/sched/fair.c | 88 ++++++++++++++---------------
kernel/sched/rt.c | 36 ++++++------
kernel/sched/sched.h | 18 +++---
lib/bug.c | 16 +++++-
10 files changed, 257 insertions(+), 204 deletions(-)
--
2.53.0
^ permalink raw reply
* [PATCH 1/2] bug: Provide WARN_ON.*DEFERRED() macros for console deferred output
From: Sebastian Andrzej Siewior @ 2026-06-23 14:26 UTC (permalink / raw)
To: linux-arch, linux-kernel, sched-ext, netdev
Cc: David S . Miller, Andrea Righi, Andrew Morton, Arnd Bergmann,
Ben Segall, Breno Leitao, Changwoo Min, David Vernet,
Dietmar Eggemann, Eric Dumazet, Ingo Molnar, Jakub Kicinski,
John Ogness, Juri Lelli, K Prateek Nayak, Paolo Abeni,
Peter Zijlstra, Petr Mladek, Sergey Senozhatsky, Simon Horman,
Steven Rostedt, Tejun Heo, Vincent Guittot, Vlad Poenaru,
Sebastian Andrzej Siewior
In-Reply-To: <20260623142650.265721-1-bigeasy@linutronix.de>
Provide a deferred version of the WARN_ON() macro. It will delay
flushing the console until a later context. It is needed in a context
where the caller holds locks which can lead to a deadlock content is
flushed to the console driver.
An example would from a warning from within the scheduler resulting in a
wake-up of a task.
Deferring the output works by using printk_deferred_enter/ exit() around
the printing output. This must be used in a context where the task can't
migrate to another CPU. This should be the case usually, since the
scheduler would acquire the rq lock whith disabled interrupts, but to be
safe preemption is disabled to guarantee this.
In order not to bloat the code on architectures which provide an
optimized __WARN_FLAGS() define BUGFLAG_DEFERRED which is handled by
__report_bug() and does not increase the code size.
Provide the DEFERRED macros based on __WARN_FLAGS and __WARN_FLAGS
macros. Extend __report_bug() to handle the deferred case.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/asm-generic/bug.h | 41 +++++++++++++++++++++++++++++++++++++++
lib/bug.c | 16 +++++++++++++--
2 files changed, 55 insertions(+), 2 deletions(-)
diff --git a/include/asm-generic/bug.h b/include/asm-generic/bug.h
index 09e8eccee8ed9..1e3ff00f709b8 100644
--- a/include/asm-generic/bug.h
+++ b/include/asm-generic/bug.h
@@ -14,6 +14,7 @@
#define BUGFLAG_DONE (1 << 2)
#define BUGFLAG_NO_CUT_HERE (1 << 3) /* CUT_HERE already sent */
#define BUGFLAG_ARGS (1 << 4)
+#define BUGFLAG_DEFERRED (1 << 5)
#define BUGFLAG_TAINT(taint) ((taint) << 8)
#define BUG_GET_TAINT(bug) ((bug)->flags >> 8)
#endif
@@ -115,6 +116,16 @@ extern __printf(1, 2) void __warn_printk(const char *fmt, ...);
})
#endif
+#define WARN_ON_DEFERRED(condition) ({ \
+ int __ret_warn_on = !!(condition); \
+ if (unlikely(__ret_warn_on)) { \
+ __WARN_FLAGS(#condition, \
+ BUGFLAG_DEFERRED | \
+ BUGFLAG_TAINT(TAINT_WARN)); \
+ } \
+ unlikely(__ret_warn_on); \
+})
+
#ifndef WARN_ON_ONCE
#define WARN_ON_ONCE(condition) ({ \
int __ret_warn_on = !!(condition); \
@@ -125,6 +136,16 @@ extern __printf(1, 2) void __warn_printk(const char *fmt, ...);
unlikely(__ret_warn_on); \
})
#endif
+
+#define WARN_ON_ONCE_DEFERRED(condition) ({ \
+ int __ret_warn_on = !!(condition); \
+ if (unlikely(__ret_warn_on)) { \
+ __WARN_FLAGS(#condition, \
+ BUGFLAG_ONCE | BUGFLAG_DEFERRED | \
+ BUGFLAG_TAINT(TAINT_WARN)); \
+ } \
+ unlikely(__ret_warn_on); \
+})
#endif /* __WARN_FLAGS */
#if defined(__WARN_FLAGS) && !defined(__WARN_printf)
@@ -159,6 +180,19 @@ extern __printf(1, 2) void __warn_printk(const char *fmt, ...);
})
#endif
+#ifndef WARN_ON_DEFERRED
+#define WARN_ON_DEFERRED(condition) ({ \
+ int __ret_warn_on = !!(condition); \
+ if (unlikely(__ret_warn_on)) { \
+ guard(preempt)(); \
+ printk_deferred_enter() \
+ __WARN(); \
+ printk_deferred_exit() \
+ } \
+ unlikely(__ret_warn_on); \
+})
+#endif
+
#ifndef WARN
#define WARN(condition, format...) ({ \
int __ret_warn_on = !!(condition); \
@@ -180,6 +214,11 @@ extern __printf(1, 2) void __warn_printk(const char *fmt, ...);
DO_ONCE_LITE_IF(condition, WARN_ON, 1)
#endif
+#ifndef WARN_ON_ONCE_DEFERRED
+#define WARN_ON_ONCE_DEFERRED(condition) \
+ DO_ONCE_LITE_IF(condition, WARN_ON_DEFERRED, 1)
+#endif
+
#ifndef WARN_ONCE
#define WARN_ONCE(condition, format...) \
DO_ONCE_LITE_IF(condition, WARN, 1, format)
@@ -215,7 +254,9 @@ extern __printf(1, 2) void __warn_printk(const char *fmt, ...);
})
#endif
+#define WARN_ON_DEFERRED(condition) WARN_ON(condition)
#define WARN_ON_ONCE(condition) WARN_ON(condition)
+#define WARN_ON_ONCE_DEFERRED(condition) WARN_ON(condition)
#define WARN_ONCE(condition, format...) WARN(condition, format)
#define WARN_TAINT(condition, taint, format...) WARN(condition, format)
#define WARN_TAINT_ONCE(condition, taint, format...) WARN(condition, format)
diff --git a/lib/bug.c b/lib/bug.c
index 224f4cfa4aa31..f5768f5d17b47 100644
--- a/lib/bug.c
+++ b/lib/bug.c
@@ -196,7 +196,7 @@ void __warn_printf(const char *fmt, struct pt_regs *regs)
static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long bugaddr, struct pt_regs *regs)
{
- bool warning, once, done, no_cut, has_args;
+ bool warning, once, done, no_cut, has_args, deferred;
const char *file, *fmt;
unsigned line;
@@ -219,6 +219,7 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
done = bug->flags & BUGFLAG_DONE;
no_cut = bug->flags & BUGFLAG_NO_CUT_HERE;
has_args = bug->flags & BUGFLAG_ARGS;
+ deferred = bug->flags & BUGFLAG_DEFERRED;
if (warning && once) {
if (done)
@@ -229,7 +230,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
*/
bug->flags |= BUGFLAG_DONE;
}
-
+ if (deferred) {
+ preempt_disable_notrace();
+ printk_deferred_enter();
+ }
/*
* BUG() and WARN_ON() families don't print a custom debug message
* before triggering the exception handler, so we must add the
@@ -245,6 +249,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
/* this is a WARN_ON rather than BUG/BUG_ON */
__warn(file, line, (void *)bugaddr, BUG_GET_TAINT(bug), regs,
NULL);
+ if (deferred) {
+ printk_deferred_exit();
+ preempt_enable_notrace();
+ }
return BUG_TRAP_TYPE_WARN;
}
@@ -254,6 +262,10 @@ static enum bug_trap_type __report_bug(struct bug_entry *bug, unsigned long buga
pr_crit("kernel BUG at %pB [verbose debug info unavailable]\n",
(void *)bugaddr);
+ if (deferred) {
+ printk_deferred_exit();
+ preempt_enable_notrace();
+ }
return BUG_TRAP_TYPE_BUG;
}
--
2.53.0
^ permalink raw reply related
* Re: [PATCH v9 1/2] dt-bindings: i3c: Add AMD I3C master controller support
From: Pandey, Radhey Shyam @ 2026-06-23 12:57 UTC (permalink / raw)
To: Shubham Patil, git, michal.simek, alexandre.belloni, Frank.Li,
robh, krzk+dt, conor+dt, pgaj, wsa+renesas, tommaso.merciai.xr,
arnd, quic_msavaliy, Shyam-sundar.S-k, sakari.ailus, billy_tsai,
kees, gustavoars, jarkko.nikula, jorge.marques, linux-i3c,
devicetree, linux-kernel, linux-arch, linux-hardening
Cc: radhey.shyam.pandey, srinivas.goud, shubhrajyoti.datta,
Manikanta Guntupalli
In-Reply-To: <20260623114417.2578189-2-shubhamsanjay.patil@amd.com>
> From: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
>
> Add device tree binding documentation for the AMD I3C master controller
> version 1.0.
>
> Signed-off-by: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
> Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
It's missing your SOB.
Any further SoBs (Signed-off-by:s) following the author SoB are
from people handling and transporting the patch, but were not
involved in its development.
> ---
> Changes for V9:
> None.
It's not correct - you updated maintainer section in yaml?
>
> Changes for V8:
> None.
>
> Changes for V7:
> Added i3c controller version details to commit description.
>
> Changes for V6:
> Corrected the file name for $id in yaml to fix the dtschema warning.
>
> Changes for V5:
> Renamed the xlnx,axi-i3c.yaml file into xlnx,axi-i3c-1.0.yaml.
>
> Changes for V4:
> Added h/w documentation details.
>
> Changes for V3:
> Updated commit description.
> Corrected the order of properties and removed resets property.
> Added compatible to required list.
> Added interrupts to example.
>
> Changes for V2:
> Updated commit subject and description.
> Moved allOf to after required.
> Removed xlnx,num-targets property.
> ---
> .../bindings/i3c/xlnx,axi-i3c-1.0.yaml | 56 +++++++++++++++++++
> 1 file changed, 56 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
>
> diff --git a/Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml b/Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
> new file mode 100644
> index 000000000000..75f677696f02
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
> @@ -0,0 +1,56 @@
> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/i3c/xlnx,axi-i3c-1.0.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: AMD I3C master
> +
> +maintainers:
> + - Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
> + - Shubham Patil <shubhamsanjay.patil@amd.com>
> +
> +description:
> + The AXI-I3C IP is an I3C Controller with an AXI4-Lite interface, compatible
> + with the MIPI I3C Specification v1.1.1. The design includes bidirectional I/O
> + buffers that implement open collector drivers for the SDA and SCL signals.
> + External pull-up resistors are required to properly hold the bus at a Logic-1
> + level when the drivers are released.
> +
> + For more details, please see https://docs.amd.com/r/en-US/pg439-axi-i3c
> +
> +properties:
> + compatible:
> + const: xlnx,axi-i3c-1.0
> +
> + reg:
> + maxItems: 1
> +
> + clocks:
> + maxItems: 1
> +
> + interrupts:
> + maxItems: 1
> +
> +required:
> + - compatible
> + - reg
> + - clocks
> +
> +allOf:
> + - $ref: i3c.yaml#
> +
> +unevaluatedProperties: false
> +
> +examples:
> + - |
> + i3c@80000000 {
> + compatible = "xlnx,axi-i3c-1.0";
> + reg = <0x80000000 0x10000>;
> + clocks = <&zynqmp_clk 71>;
> + interrupt-parent = <&imux>;
> + interrupts = <0 89 4>;
Nit - Don't use hard-coded numbers.
> + #address-cells = <3>;
> + #size-cells = <0>;
> + };
> +...
^ permalink raw reply
* [PATCH v9 2/2] i3c: master: Add driver for AMD AXI I3C master controller
From: Shubham Patil @ 2026-06-23 11:44 UTC (permalink / raw)
To: git, michal.simek, alexandre.belloni, Frank.Li, robh, krzk+dt,
conor+dt, pgaj, wsa+renesas, tommaso.merciai.xr, arnd,
quic_msavaliy, Shyam-sundar.S-k, sakari.ailus, billy_tsai, kees,
gustavoars, jarkko.nikula, jorge.marques, linux-i3c, devicetree,
linux-kernel, linux-arch, linux-hardening
Cc: radhey.shyam.pandey, srinivas.goud, shubhrajyoti.datta,
shubhamsanjay.patil, Manikanta Guntupalli
In-Reply-To: <20260623114417.2578189-1-shubhamsanjay.patil@amd.com>
From: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
Add an I3C master driver and maintainers fragment for the AMD I3C bus
controller.
The driver currently supports the I3C bus operating in SDR mode,
with features including Dynamic Address Assignment, private data
transfers, and CCC transfers in both broadcast and direct modes. It
also supports operation in I2C mode.
The controller's data FIFOs are accessed big-endian; the driver performs
this conversion locally using ioread32be()/iowrite32be() with the
helpers, so it does not depend on any core FIFO-endianness helpers.
Signed-off-by: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
Co-developed-by: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
Signed-off-by: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
Co-developed-by: Shubham Patil <shubhamsanjay.patil@amd.com>
Signed-off-by: Shubham Patil <shubhamsanjay.patil@amd.com>
---
Changes for V9:
Updated commit description to note that the driver performs big-endian
FIFO accesses locally (the v8 core-helper patches were dropped).
Dropped the big-endian MMIO infrastructure patches from the series
("asm-generic/io.h: Add big-endian MMIO accessors", "i3c: fix
big-endian FIFO transfers", and "i3c: master: Add endianness support
for i3c_readl_fifo()/i3c_writel_fifo()"). The driver now performs
big-endian FIFO accesses locally using ioread32be()/iowrite32be() with
get_unaligned()/put_unaligned(), so the series is self-contained and no
longer includes internals.h.
Replaced the async completion/transfer-queue machinery with a simple
synchronous transfer path under the existing mutex.
Reworked response handling: added enum i3c_error_code to struct
xi3c_cmd, named the response codes, return -ENODEV/-EIO as appropriate
and set err = I3C_ERROR_M2/M0 so the i3c core and callers can tell a
NACK apart from a bus error; propagate err to CCC commands and to each
priv xfer (including actual_len).
Switched from .priv_xfers to the new .i3c_xfers op; reject non-SDR
modes with -EOPNOTSUPP and report actual_len.
Reworked DAA: assign addresses incrementally, bound the device count
(-ENOSPC), detect end-of-enumeration via -ENODEV, zero-initialize the
PID buffers, and check i3c_master_add_i3c_dev_locked().
Avoid busy-spinning: sleep with usleep_range() in the FIFO drain/fill
loops.
Use FIELD_PREP() with named command-FIFO field masks instead of
open-coded shifts, and convert the register-accessor macros to inline
functions.
Split the overloaded timeout macro into XI3C_RESP_TIMEOUT_US and
XI3C_XFER_TIMEOUT_MS with documented units, and add
XI3C_POLL_INTERVAL_US.
xi3c_clk_cfg(): use NSEC_PER_SEC and named timing constants, guard
against unsigned underflow, and handle I3C_BUS_MODE_MIXED_SLOW.
Dropped ENTHDR from supports_ccc_cmd() (SDR-only), and dispatch CCCs
using the I3C_CCC_DIRECT bit.
Use const for TX buffers and drop the related casts; use parity8() for
the DAA parity bit.
Updated MODULE_DESCRIPTION and authors, the copyright year, renamed the
Kconfig symbol to AMD_AXI_I3C_MASTER, and fixed the MAINTAINERS entry
(title, mailing list, and the correct binding filename).
Changes for V8:
Used time_left instead of timeout.
Used __free(kfree) for xfer to simplify err path in multiple places.
Changes for V7:
Updated timeout macro name.
Updated xi3c_master_wr_to_tx_fifo() and xi3c_master_rd_from_rx_fifo()
to use i3c_writel_fifo() and i3c_readl_fifo().
Changes for V6:
Removed typecast for xi3c_getrevisionnumber(), xi3c_wrfifolevel(),
and xi3c_rdfifolevel().
Replaced dynamic allocation with a static variable for pid_bcr_dcr.
Fixed sparse warning in do_daa by typecasting the address parity value
to u8.
Fixed sparse warning in xi3c_master_bus_init by typecasting the pid value
to u64 in info.pid calculation.
Changes for V5:
Used GENMASK_ULL for PID mask as it's 64bit mask.
Changes for V4:
Updated timeout macros.
Removed type casting for xi3c_is_resp_available() macro.
Used ioread32() and iowrite32() instead of readl() and writel()
to keep consistency.
Read XI3C_RESET_OFFSET reg before udelay().
Removed xi3c_master_free_xfer() and directly used kfree().
Skipped checking return value of i3c_master_add_i3c_dev_locked().
Used devm_mutex_init() instead of mutex_init().
Changes for V3:
Resolved merge conflicts.
Changes for V2:
Updated commit description.
Added mixed mode support with clock configuration.
Converted smaller functions into inline functions.
Used FIELD_GET() in xi3c_get_response().
Updated xi3c_master_rd_from_rx_fifo() to use cmd->rx_buf.
Used parity8() for address parity calculation.
Added guards for locks.
Dropped num_targets and updated xi3c_master_do_daa().
Used __free(kfree) in xi3c_master_send_bdcast_ccc_cmd().
Dropped PM runtime support.
Updated xi3c_master_read() and xi3c_master_write() with
xi3c_is_resp_available() check.
Created separate functions: xi3c_master_init() and xi3c_master_reinit().
Used xi3c_master_init() in bus initialization and xi3c_master_reinit()
in error paths.
Added DAA structure to xi3c_master structure.
---
MAINTAINERS | 8 +
drivers/i3c/master/Kconfig | 15 +
drivers/i3c/master/Makefile | 1 +
drivers/i3c/master/amd-i3c-master.c | 1060 +++++++++++++++++++++++++++
4 files changed, 1084 insertions(+)
create mode 100644 drivers/i3c/master/amd-i3c-master.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 461a3eed6129..bfaa6999913c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1035,6 +1035,14 @@ L: linux-sound@vger.kernel.org
S: Supported
F: sound/soc/amd/
+AMD AXI I3C MASTER DRIVER
+M: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
+M: Shubham Patil <shubhamsanjay.patil@amd.com>
+L: linux-i3c@lists.infradead.org
+S: Maintained
+F: Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
+F: drivers/i3c/master/amd-i3c-master.c
+
AMD AXI W1 DRIVER
M: Kris Chaplin <kris.chaplin@amd.com>
R: Thomas Delev <thomas.delev@amd.com>
diff --git a/drivers/i3c/master/Kconfig b/drivers/i3c/master/Kconfig
index 2609f2b18e0a..da96d2aaa399 100644
--- a/drivers/i3c/master/Kconfig
+++ b/drivers/i3c/master/Kconfig
@@ -86,3 +86,18 @@ config RENESAS_I3C
This driver can also be built as a module. If so, the module will be
called renesas-i3c.
+
+config AMD_AXI_I3C_MASTER
+ tristate "AMD AXI I3C Master driver"
+ depends on HAS_IOMEM
+ help
+ Support for the AMD AXI I3C master controller, a soft IP used on
+ AMD (Xilinx) FPGAs and adaptive SoCs with ARM or MicroBlaze
+ processors.
+
+ The controller currently supports Standard Data Rate (SDR) mode.
+ Features include Dynamic Address Assignment, private transfers,
+ and CCC transfers in both broadcast and direct modes.
+
+ This driver can also be built as a module. If so, the module
+ will be called amd-i3c-master.
diff --git a/drivers/i3c/master/Makefile b/drivers/i3c/master/Makefile
index 816a227b6f7a..8d82196dcf83 100644
--- a/drivers/i3c/master/Makefile
+++ b/drivers/i3c/master/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_AST2600_I3C_MASTER) += ast2600-i3c-master.o
obj-$(CONFIG_SVC_I3C_MASTER) += svc-i3c-master.o
obj-$(CONFIG_MIPI_I3C_HCI) += mipi-i3c-hci/
obj-$(CONFIG_RENESAS_I3C) += renesas-i3c.o
+obj-$(CONFIG_AMD_AXI_I3C_MASTER) += amd-i3c-master.o
diff --git a/drivers/i3c/master/amd-i3c-master.c b/drivers/i3c/master/amd-i3c-master.c
new file mode 100644
index 000000000000..34ab1028c3ce
--- /dev/null
+++ b/drivers/i3c/master/amd-i3c-master.c
@@ -0,0 +1,1060 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * I3C master driver for the AMD I3C controller.
+ *
+ * Copyright (C) 2026, Advanced Micro Devices, Inc.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/cleanup.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/i3c/master.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/time.h>
+#include <linux/unaligned.h>
+
+#define XI3C_VERSION_OFFSET 0x00 /* Version Register */
+#define XI3C_RESET_OFFSET 0x04 /* Soft Reset Register */
+#define XI3C_CR_OFFSET 0x08 /* Control Register */
+#define XI3C_ADDRESS_OFFSET 0x0C /* Target Address Register */
+#define XI3C_SR_OFFSET 0x10 /* Status Register */
+#define XI3C_CMD_FIFO_OFFSET 0x20 /* I3C Command FIFO Register */
+#define XI3C_WR_FIFO_OFFSET 0x24 /* I3C Write Data FIFO Register */
+#define XI3C_RD_FIFO_OFFSET 0x28 /* I3C Read Data FIFO Register */
+#define XI3C_RESP_STATUS_FIFO_OFFSET 0x2C /* I3C Response status FIFO Register */
+#define XI3C_FIFO_LVL_STATUS_OFFSET 0x30 /* CMD slots free | WR-FIFO free (words) */
+#define XI3C_FIFO_LVL_STATUS_1_OFFSET 0x34 /* RESP fill | RD-FIFO fill level (words) */
+#define XI3C_SCL_HIGH_TIME_OFFSET 0x38 /* I3C SCL HIGH Register */
+#define XI3C_SCL_LOW_TIME_OFFSET 0x3C /* I3C SCL LOW Register */
+#define XI3C_SDA_HOLD_TIME_OFFSET 0x40 /* I3C SDA HOLD Register */
+#define XI3C_TSU_START_OFFSET 0x48 /* I3C START SETUP Register */
+#define XI3C_THD_START_OFFSET 0x4C /* I3C START HOLD Register */
+#define XI3C_TSU_STOP_OFFSET 0x50 /* I3C STOP Setup Register */
+#define XI3C_OD_SCL_HIGH_TIME_OFFSET 0x54 /* I3C OD SCL HIGH Register */
+#define XI3C_OD_SCL_LOW_TIME_OFFSET 0x58 /* I3C OD SCL LOW Register */
+#define XI3C_PID0_OFFSET 0x6C /* LSB 4 bytes of the PID */
+#define XI3C_PID1_BCR_DCR 0x70 /* MSB 2 bytes of the PID, BCR and DCR */
+
+#define XI3C_CR_EN_MASK BIT(0) /* Core Enable */
+#define XI3C_CR_RESUME_MASK BIT(2) /* Core Resume */
+#define XI3C_SR_RESP_NOT_EMPTY_MASK BIT(4) /* Resp Fifo not empty status mask */
+#define XI3C_RD_FIFO_NOT_EMPTY_MASK BIT(15) /* Read Fifo not empty status mask */
+
+#define XI3C_BCR_MASK GENMASK(23, 16)
+#define XI3C_DCR_MASK GENMASK(31, 24)
+#define XI3C_PID_MASK GENMASK_ULL(63, 16)
+#define XI3C_TIMING_MASK GENMASK(17, 0)
+#define XI3C_REV_NUM_MASK GENMASK(15, 8)
+#define XI3C_PID1_MASK GENMASK(15, 0)
+#define XI3C_FIFO_LEVEL_MASK GENMASK(15, 0)
+#define XI3C_RESP_CODE_MASK GENMASK(8, 5)
+#define XI3C_RESP_CODE_SUCCESS 0 /* Transfer completed OK */
+#define XI3C_RESP_CODE_NO_TARGET 2 /* 7E NACK: no target on bus */
+#define XI3C_RESP_CODE_NACK 3 /* Target NACK / CE2 / DAA end */
+#define XI3C_ADDR_MASK GENMASK(6, 0)
+#define XI3C_FIFOS_RST_MASK GENMASK(4, 1)
+
+/* Command FIFO word layout (bit ranges encoded in the GENMASK/BIT args) */
+#define XI3C_CMD_TYPE GENMASK(3, 0) /* command type */
+#define XI3C_CMD_TERMINATE BIT(4) /* terminate (last cmd of xfer) */
+#define XI3C_CMD_ADDR GENMASK(15, 8) /* target address << 1 | RnW */
+#define XI3C_CMD_LEN GENMASK(27, 16) /* payload length in bytes */
+#define XI3C_CMD_TID GENMASK(31, 28) /* transfer ID */
+
+#define XI3C_OD_TLOW_NS 500000
+#define XI3C_OD_THIGH_NS 41000
+#define XI3C_I2C_TCASMIN_NS 600000
+#define XI3C_TCASMIN_NS 260000
+#define XI3C_MAXDATA_LENGTH 4095
+#define XI3C_MAX_DEVS 32
+#define XI3C_DAA_SLAVEINFO_READ_BYTECOUNT 8
+
+#define XI3C_THOLD_MIN_REV0 5 /* Min SDA hold cycles, rev 0 IP */
+#define XI3C_THOLD_MIN_REV1 6 /* Min SDA hold cycles, rev >= 1 IP */
+#define XI3C_CYCLE_ADJUST 2 /* SCL/SDA pre-bias for HW pipeline */
+#define XI3C_FIFO_RESET_DELAY_US 10 /* HW settling time after FIFO reset */
+#define XI3C_POLL_INTERVAL_US 10 /* readl_poll_timeout() sleep slice */
+
+#define XI3C_I2C_MODE 0
+#define XI3C_I2C_TID 0
+#define XI3C_SDR_MODE 1
+#define XI3C_SDR_TID 1
+
+#define XI3C_WORD_LEN 4
+
+/*
+ * XI3C_RESP_TIMEOUT_US is in microseconds because it is passed as the
+ * timeout_us argument of readl_poll_timeout(). XI3C_XFER_TIMEOUT_MS is in
+ * milliseconds because it feeds msecs_to_jiffies(). Keep the two units
+ * distinct in the names so callers cannot mix them up.
+ */
+#define XI3C_RESP_TIMEOUT_US 500000
+#define XI3C_XFER_TIMEOUT_MS 1000
+
+struct xi3c_cmd {
+ const void *tx_buf;
+ void *rx_buf;
+ u16 tx_len;
+ u16 rx_len;
+ u8 addr;
+ u8 type;
+ u8 tid;
+ bool rnw;
+ bool is_daa;
+ bool continued;
+ enum i3c_error_code err;
+};
+
+struct xi3c_xfer {
+ unsigned int ncmds;
+ struct xi3c_cmd cmds[] __counted_by(ncmds);
+};
+
+/**
+ * struct xi3c_master - I3C master controller state.
+ * @base: I3C master controller embedded by the framework.
+ * @dev: Pointer to the backing device structure.
+ * @membase: Memory base of the HW registers.
+ * @pclk: Input clock driving the controller.
+ * @lock: Serializes transfers and CCC submission.
+ * @daa: ENTDAA enumeration state.
+ * @daa.addrs: Dynamic addresses assigned in enumeration order.
+ * @daa.index: Number of responders enumerated so far.
+ */
+struct xi3c_master {
+ struct i3c_master_controller base;
+ struct device *dev;
+ void __iomem *membase;
+ struct clk *pclk;
+ struct mutex lock; /* serializes transfers and CCC submission */
+ struct {
+ u8 addrs[XI3C_MAX_DEVS];
+ u8 index;
+ } daa;
+};
+
+static inline struct xi3c_master *
+to_xi3c_master(struct i3c_master_controller *master)
+{
+ return container_of(master, struct xi3c_master, base);
+}
+
+static inline u8 xi3c_get_revision_number(struct xi3c_master *master)
+{
+ return FIELD_GET(XI3C_REV_NUM_MASK,
+ ioread32(master->membase + XI3C_VERSION_OFFSET));
+}
+
+static inline u16 xi3c_wr_fifo_level(struct xi3c_master *master)
+{
+ return ioread32(master->membase + XI3C_FIFO_LVL_STATUS_OFFSET) &
+ XI3C_FIFO_LEVEL_MASK;
+}
+
+static inline u16 xi3c_rd_fifo_level(struct xi3c_master *master)
+{
+ return ioread32(master->membase + XI3C_FIFO_LVL_STATUS_1_OFFSET) &
+ XI3C_FIFO_LEVEL_MASK;
+}
+
+static inline bool xi3c_is_resp_available(struct xi3c_master *master)
+{
+ return FIELD_GET(XI3C_SR_RESP_NOT_EMPTY_MASK,
+ ioread32(master->membase + XI3C_SR_OFFSET));
+}
+
+static int xi3c_get_response(struct xi3c_master *master, struct xi3c_cmd *cmd)
+{
+ u32 response_data;
+ u32 resp_reg;
+ u8 code;
+ int ret;
+
+ ret = readl_poll_timeout(master->membase + XI3C_SR_OFFSET,
+ resp_reg,
+ resp_reg & XI3C_SR_RESP_NOT_EMPTY_MASK,
+ XI3C_POLL_INTERVAL_US, XI3C_RESP_TIMEOUT_US);
+ if (ret) {
+ dev_err(master->dev, "XI3C response timeout\n");
+ return ret;
+ }
+
+ response_data = ioread32(master->membase + XI3C_RESP_STATUS_FIFO_OFFSET);
+ code = FIELD_GET(XI3C_RESP_CODE_MASK, response_data);
+
+ switch (code) {
+ case XI3C_RESP_CODE_SUCCESS:
+ cmd->err = I3C_ERROR_UNKNOWN;
+ return 0;
+ case XI3C_RESP_CODE_NO_TARGET:
+ case XI3C_RESP_CODE_NACK:
+ /*
+ * Target did not ACK. Record it as I3C_ERROR_M2 so callers
+ * (and the i3c core, which keys on err == I3C_ERROR_M2) can
+ * tell a NACK apart from other failures. A normal transfer
+ * surfaces this as -EIO per the i3c_xfer contract; the DAA
+ * path instead expects -ENODEV as its enumeration terminator.
+ */
+ cmd->err = I3C_ERROR_M2;
+ return cmd->is_daa ? -ENODEV : -EIO;
+ default:
+ cmd->err = I3C_ERROR_M0;
+ dev_err(master->dev, "XI3C transfer error, response code %u\n",
+ code);
+ return -EIO;
+ }
+}
+
+static inline void xi3c_writesl_be(void __iomem *addr, const void *buffer,
+ unsigned int count)
+{
+ const u32 *buf = buffer;
+
+ while (count--)
+ iowrite32be(get_unaligned(buf++), addr);
+}
+
+static inline void xi3c_readsl_be(const void __iomem *addr, void *buffer,
+ unsigned int count)
+{
+ u32 *buf = buffer;
+
+ while (count--)
+ put_unaligned(ioread32be(addr), buf++);
+}
+
+static inline void xi3c_writel_fifo(void __iomem *addr, const void *buf,
+ int nbytes)
+{
+ xi3c_writesl_be(addr, buf, nbytes / 4);
+ if (nbytes & 3) {
+ u32 tmp = 0;
+
+ memcpy(&tmp, (const u8 *)buf + (nbytes & ~3), nbytes & 3);
+ xi3c_writesl_be(addr, &tmp, 1);
+ }
+}
+
+static inline void xi3c_readl_fifo(const void __iomem *addr, void *buf,
+ int nbytes)
+{
+ xi3c_readsl_be(addr, buf, nbytes / 4);
+ if (nbytes & 3) {
+ u32 tmp;
+
+ xi3c_readsl_be(addr, &tmp, 1);
+ memcpy((u8 *)buf + (nbytes & ~3), &tmp, nbytes & 3);
+ }
+}
+
+static void xi3c_master_write_to_cmdfifo(struct xi3c_master *master,
+ struct xi3c_cmd *cmd, u16 len)
+{
+ u32 transfer_cmd;
+ u8 addr;
+
+ addr = ((cmd->addr & XI3C_ADDR_MASK) << 1) | (u8)cmd->rnw;
+
+ transfer_cmd = FIELD_PREP(XI3C_CMD_TYPE, cmd->type);
+ transfer_cmd |= FIELD_PREP(XI3C_CMD_TERMINATE, !cmd->continued);
+ transfer_cmd |= FIELD_PREP(XI3C_CMD_ADDR, addr);
+ transfer_cmd |= FIELD_PREP(XI3C_CMD_TID, cmd->tid);
+
+ /*
+ * For dynamic addressing, an additional 1-byte length must be added
+ * to the command FIFO to account for the address present in the TX FIFO
+ */
+ if (cmd->is_daa) {
+ xi3c_writel_fifo(master->membase + XI3C_WR_FIFO_OFFSET,
+ cmd->tx_buf, cmd->tx_len);
+
+ len++;
+ }
+
+ transfer_cmd |= FIELD_PREP(XI3C_CMD_LEN, len);
+ iowrite32(transfer_cmd, master->membase + XI3C_CMD_FIFO_OFFSET);
+}
+
+static inline void xi3c_master_enable(struct xi3c_master *master)
+{
+ iowrite32(ioread32(master->membase + XI3C_CR_OFFSET) | XI3C_CR_EN_MASK,
+ master->membase + XI3C_CR_OFFSET);
+}
+
+static inline void xi3c_master_disable(struct xi3c_master *master)
+{
+ iowrite32(ioread32(master->membase + XI3C_CR_OFFSET) & ~XI3C_CR_EN_MASK,
+ master->membase + XI3C_CR_OFFSET);
+}
+
+static inline void xi3c_master_resume(struct xi3c_master *master)
+{
+ iowrite32(ioread32(master->membase + XI3C_CR_OFFSET) |
+ XI3C_CR_RESUME_MASK, master->membase + XI3C_CR_OFFSET);
+}
+
+static void xi3c_master_reset_fifos(struct xi3c_master *master)
+{
+ u32 data;
+
+ /* Assert FIFO reset. */
+ data = ioread32(master->membase + XI3C_RESET_OFFSET);
+ data |= XI3C_FIFOS_RST_MASK;
+ iowrite32(data, master->membase + XI3C_RESET_OFFSET);
+ /* Read-back flushes the posted write before the settling delay below. */
+ ioread32(master->membase + XI3C_RESET_OFFSET);
+ udelay(XI3C_FIFO_RESET_DELAY_US);
+
+ /* De-assert FIFO reset, then wait for the FIFOs to come back up. */
+ data &= ~XI3C_FIFOS_RST_MASK;
+ iowrite32(data, master->membase + XI3C_RESET_OFFSET);
+ ioread32(master->membase + XI3C_RESET_OFFSET);
+ udelay(XI3C_FIFO_RESET_DELAY_US);
+}
+
+static inline void xi3c_master_init(struct xi3c_master *master)
+{
+ /* Reset fifos */
+ xi3c_master_reset_fifos(master);
+
+ /* Enable controller */
+ xi3c_master_enable(master);
+}
+
+static inline void xi3c_master_reinit(struct xi3c_master *master)
+{
+ /* Reset fifos */
+ xi3c_master_reset_fifos(master);
+
+ /* Resume controller */
+ xi3c_master_resume(master);
+}
+
+static struct xi3c_xfer *xi3c_master_alloc_xfer(unsigned int ncmds)
+{
+ struct xi3c_xfer *xfer;
+
+ xfer = kzalloc(struct_size(xfer, cmds, ncmds), GFP_KERNEL);
+ if (!xfer)
+ return NULL;
+
+ xfer->ncmds = ncmds;
+
+ return xfer;
+}
+
+static void xi3c_master_rd_from_rx_fifo(struct xi3c_master *master,
+ struct xi3c_cmd *cmd)
+{
+ u16 rx_data_available;
+ u16 copy_len;
+ u16 len;
+
+ rx_data_available = xi3c_rd_fifo_level(master);
+ len = rx_data_available * XI3C_WORD_LEN;
+
+ if (!len)
+ return;
+
+ copy_len = min_t(u16, len, cmd->rx_len);
+ xi3c_readl_fifo(master->membase + XI3C_RD_FIFO_OFFSET,
+ (u8 *)cmd->rx_buf, copy_len);
+
+ cmd->rx_buf = (u8 *)cmd->rx_buf + copy_len;
+ cmd->rx_len -= copy_len;
+}
+
+static int xi3c_master_read(struct xi3c_master *master, struct xi3c_cmd *cmd)
+{
+ unsigned long timeout;
+ u32 status_reg;
+ int ret;
+
+ if (!cmd->rx_buf || cmd->rx_len > XI3C_MAXDATA_LENGTH)
+ return -EINVAL;
+
+ /* Fill command fifo */
+ xi3c_master_write_to_cmdfifo(master, cmd, cmd->rx_len);
+
+ if (!cmd->rx_len)
+ return 0;
+
+ ret = readl_poll_timeout(master->membase + XI3C_SR_OFFSET,
+ status_reg,
+ status_reg & (XI3C_RD_FIFO_NOT_EMPTY_MASK |
+ XI3C_SR_RESP_NOT_EMPTY_MASK),
+ XI3C_POLL_INTERVAL_US, XI3C_RESP_TIMEOUT_US);
+ if (ret) {
+ dev_err(master->dev, "XI3C read timeout\n");
+ return ret;
+ }
+
+ if (!(status_reg & XI3C_RD_FIFO_NOT_EMPTY_MASK))
+ return 0;
+
+ timeout = jiffies + msecs_to_jiffies(XI3C_XFER_TIMEOUT_MS);
+
+ /* Read data from rx fifo */
+ while (cmd->rx_len > 0 && !xi3c_is_resp_available(master)) {
+ if (time_after(jiffies, timeout)) {
+ dev_err(master->dev, "XI3C read timeout\n");
+ return -EIO;
+ }
+ xi3c_master_rd_from_rx_fifo(master, cmd);
+ usleep_range(XI3C_POLL_INTERVAL_US, 2 * XI3C_POLL_INTERVAL_US);
+ }
+
+ /* Read remaining data */
+ xi3c_master_rd_from_rx_fifo(master, cmd);
+
+ return 0;
+}
+
+static void xi3c_master_wr_to_tx_fifo(struct xi3c_master *master,
+ struct xi3c_cmd *cmd)
+{
+ u16 wrfifo_space;
+ u16 len;
+
+ wrfifo_space = xi3c_wr_fifo_level(master);
+ if (cmd->tx_len > wrfifo_space * XI3C_WORD_LEN)
+ len = wrfifo_space * XI3C_WORD_LEN;
+ else
+ len = cmd->tx_len;
+
+ if (len) {
+ xi3c_writel_fifo(master->membase + XI3C_WR_FIFO_OFFSET, cmd->tx_buf,
+ len);
+
+ cmd->tx_buf = (const u8 *)cmd->tx_buf + len;
+ cmd->tx_len -= len;
+ }
+}
+
+static int xi3c_master_write(struct xi3c_master *master, struct xi3c_cmd *cmd)
+{
+ unsigned long timeout;
+ u16 cmd_len;
+
+ if (!cmd->tx_buf || cmd->tx_len > XI3C_MAXDATA_LENGTH)
+ return -EINVAL;
+
+ cmd_len = cmd->tx_len;
+
+ /* Fill Tx fifo */
+ xi3c_master_wr_to_tx_fifo(master, cmd);
+
+ /* Write to command fifo */
+ xi3c_master_write_to_cmdfifo(master, cmd, cmd_len);
+
+ timeout = jiffies + msecs_to_jiffies(XI3C_XFER_TIMEOUT_MS);
+ /* Fill if any remaining data to tx fifo */
+ while (cmd->tx_len > 0 && !xi3c_is_resp_available(master)) {
+ if (time_after(jiffies, timeout)) {
+ dev_err(master->dev, "XI3C write timeout\n");
+ return -EIO;
+ }
+
+ xi3c_master_wr_to_tx_fifo(master, cmd);
+ usleep_range(XI3C_POLL_INTERVAL_US, 2 * XI3C_POLL_INTERVAL_US);
+ }
+
+ return 0;
+}
+
+static int xi3c_master_xfer(struct xi3c_master *master, struct xi3c_cmd *cmd)
+{
+ int ret;
+
+ if (cmd->rnw)
+ ret = xi3c_master_read(master, cmd);
+ else
+ ret = xi3c_master_write(master, cmd);
+
+ if (ret)
+ goto err_xfer_out;
+
+ ret = xi3c_get_response(master, cmd);
+ if (ret)
+ goto err_xfer_out;
+
+ return 0;
+
+err_xfer_out:
+ xi3c_master_reinit(master);
+ return ret;
+}
+
+static int xi3c_master_common_xfer(struct xi3c_master *master,
+ struct xi3c_xfer *xfer)
+{
+ unsigned int i;
+ int ret;
+
+ guard(mutex)(&master->lock);
+
+ for (i = 0; i < xfer->ncmds; i++) {
+ ret = xi3c_master_xfer(master, &xfer->cmds[i]);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int xi3c_master_do_daa(struct i3c_master_controller *m)
+{
+ u8 pid_bufs[XI3C_MAX_DEVS][XI3C_DAA_SLAVEINFO_READ_BYTECOUNT] = {};
+ struct xi3c_master *master = to_xi3c_master(m);
+ struct xi3c_xfer *xfer __free(kfree) = NULL;
+ struct xi3c_cmd *daa_cmd;
+ int addr, ret, i;
+ u8 last_addr = 0;
+ u8 *pid_buf;
+ u8 ccc_id;
+
+ xfer = xi3c_master_alloc_xfer(1);
+ if (!xfer)
+ return -ENOMEM;
+
+ /* Fill ENTDAA CCC */
+ ccc_id = I3C_CCC_ENTDAA;
+ daa_cmd = &xfer->cmds[0];
+ daa_cmd->addr = I3C_BROADCAST_ADDR;
+ daa_cmd->rnw = false;
+ daa_cmd->tx_buf = &ccc_id;
+ daa_cmd->tx_len = 1;
+ daa_cmd->type = XI3C_SDR_MODE;
+ daa_cmd->tid = XI3C_SDR_TID;
+ daa_cmd->continued = true;
+
+ ret = xi3c_master_common_xfer(master, xfer);
+ /*
+ * A NACK on the ENTDAA broadcast (I3C_ERROR_M2) means no slaves are
+ * present to enter DAA. Treat as a successful no-op after letting
+ * err_daa reinitialize the controller.
+ */
+ if (ret && daa_cmd->err == I3C_ERROR_M2) {
+ ret = 0;
+ goto err_daa;
+ }
+ if (ret)
+ goto err_daa;
+
+ master->daa.index = 0;
+
+ while (true) {
+ struct xi3c_cmd *cmd = &xfer->cmds[0];
+ u8 daa_byte;
+
+ if (master->daa.index >= XI3C_MAX_DEVS) {
+ ret = -ENOSPC;
+ goto err_daa;
+ }
+
+ addr = i3c_master_get_free_addr(m, last_addr + 1);
+ if (addr < 0) {
+ ret = addr;
+ goto err_daa;
+ }
+
+ pid_buf = pid_bufs[master->daa.index];
+
+ daa_byte = (addr << 1) | (parity8(addr) ^ 1);
+
+ cmd->tx_buf = &daa_byte;
+ cmd->tx_len = 1;
+ cmd->addr = I3C_BROADCAST_ADDR;
+ cmd->rnw = true;
+ cmd->rx_buf = pid_buf;
+ cmd->rx_len = XI3C_DAA_SLAVEINFO_READ_BYTECOUNT;
+ cmd->is_daa = true;
+ cmd->type = XI3C_SDR_MODE;
+ cmd->tid = XI3C_SDR_TID;
+ cmd->continued = true;
+
+ ret = xi3c_master_common_xfer(master, xfer);
+
+ /*
+ * End of enumeration: the next responder NACK'd the
+ * dynamic-address grant, surfaced as -ENODEV.
+ * xi3c_master_xfer() has already reset the FIFOs and
+ * resumed the core for us; just exit the loop and
+ * register the responders collected so far.
+ */
+ if (ret == -ENODEV) {
+ ret = 0;
+ break;
+ }
+ if (ret)
+ goto err_daa;
+
+ master->daa.addrs[master->daa.index] = addr;
+ last_addr = addr;
+ master->daa.index++;
+ }
+
+ for (i = 0; i < master->daa.index; i++) {
+ u64 pid;
+
+ ret = i3c_master_add_i3c_dev_locked(m, master->daa.addrs[i]);
+ if (ret)
+ goto err_daa;
+
+ pid = FIELD_GET(XI3C_PID_MASK,
+ get_unaligned_be64(pid_bufs[i]));
+ dev_dbg(master->dev, "Client %d: PID: 0x%llx\n", i, pid);
+ }
+
+ return 0;
+
+err_daa:
+ xi3c_master_reinit(master);
+ return ret;
+}
+
+static bool
+xi3c_master_supports_ccc_cmd(struct i3c_master_controller *master,
+ const struct i3c_ccc_cmd *cmd)
+{
+ if (cmd->ndests > 1)
+ return false;
+
+ switch (cmd->id) {
+ case I3C_CCC_ENEC(true):
+ case I3C_CCC_ENEC(false):
+ case I3C_CCC_DISEC(true):
+ case I3C_CCC_DISEC(false):
+ case I3C_CCC_ENTAS(0, true):
+ case I3C_CCC_ENTAS(0, false):
+ case I3C_CCC_RSTDAA(true):
+ case I3C_CCC_RSTDAA(false):
+ case I3C_CCC_ENTDAA:
+ case I3C_CCC_SETMWL(true):
+ case I3C_CCC_SETMWL(false):
+ case I3C_CCC_SETMRL(true):
+ case I3C_CCC_SETMRL(false):
+ case I3C_CCC_SETDASA:
+ case I3C_CCC_SETNEWDA:
+ case I3C_CCC_GETMWL:
+ case I3C_CCC_GETMRL:
+ case I3C_CCC_GETPID:
+ case I3C_CCC_GETBCR:
+ case I3C_CCC_GETDCR:
+ case I3C_CCC_GETSTATUS:
+ case I3C_CCC_GETMXDS:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static int xi3c_master_send_bdcast_ccc_cmd(struct xi3c_master *master,
+ struct i3c_ccc_cmd *ccc)
+{
+ struct xi3c_xfer *xfer __free(kfree) = NULL;
+ u8 *buf __free(kfree) = NULL;
+ struct xi3c_cmd *cmd;
+ u16 xfer_len;
+ int ret;
+
+ if (ccc->dests[0].payload.len >= XI3C_MAXDATA_LENGTH)
+ return -EINVAL;
+
+ xfer_len = ccc->dests[0].payload.len + 1;
+
+ xfer = xi3c_master_alloc_xfer(1);
+ if (!xfer)
+ return -ENOMEM;
+
+ buf = kmalloc(xfer_len, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ buf[0] = ccc->id;
+ memcpy(&buf[1], ccc->dests[0].payload.data, ccc->dests[0].payload.len);
+
+ cmd = &xfer->cmds[0];
+ cmd->addr = ccc->dests[0].addr;
+ cmd->rnw = ccc->rnw;
+ cmd->tx_buf = buf;
+ cmd->tx_len = xfer_len;
+ cmd->type = XI3C_SDR_MODE;
+ cmd->tid = XI3C_SDR_TID;
+ cmd->continued = false;
+
+ ret = xi3c_master_common_xfer(master, xfer);
+ ccc->err = cmd->err;
+
+ return ret;
+}
+
+static int xi3c_master_send_direct_ccc_cmd(struct xi3c_master *master,
+ struct i3c_ccc_cmd *ccc)
+{
+ struct xi3c_xfer *xfer __free(kfree) = NULL;
+ struct xi3c_cmd *cmd;
+ int ret;
+
+ if (ccc->dests[0].payload.len > XI3C_MAXDATA_LENGTH)
+ return -EINVAL;
+
+ xfer = xi3c_master_alloc_xfer(2);
+ if (!xfer)
+ return -ENOMEM;
+
+ /* Broadcasted message */
+ cmd = &xfer->cmds[0];
+ cmd->addr = I3C_BROADCAST_ADDR;
+ cmd->rnw = false;
+ cmd->tx_buf = &ccc->id;
+ cmd->tx_len = 1;
+ cmd->type = XI3C_SDR_MODE;
+ cmd->tid = XI3C_SDR_TID;
+ cmd->continued = true;
+
+ /* Directed message */
+ cmd = &xfer->cmds[1];
+ cmd->addr = ccc->dests[0].addr;
+ cmd->rnw = ccc->rnw;
+ if (cmd->rnw) {
+ cmd->rx_buf = ccc->dests[0].payload.data;
+ cmd->rx_len = ccc->dests[0].payload.len;
+ } else {
+ cmd->tx_buf = ccc->dests[0].payload.data;
+ cmd->tx_len = ccc->dests[0].payload.len;
+ }
+ cmd->type = XI3C_SDR_MODE;
+ cmd->tid = XI3C_SDR_TID;
+ cmd->continued = false;
+
+ ret = xi3c_master_common_xfer(master, xfer);
+
+ /*
+ * Report the broadcast command's error if it failed, otherwise the
+ * directed command's, so a NACK on either phase reaches the caller.
+ */
+ ccc->err = xfer->cmds[0].err ? xfer->cmds[0].err : xfer->cmds[1].err;
+
+ return ret;
+}
+
+static int xi3c_master_send_ccc_cmd(struct i3c_master_controller *m,
+ struct i3c_ccc_cmd *cmd)
+{
+ struct xi3c_master *master = to_xi3c_master(m);
+
+ if (cmd->id & I3C_CCC_DIRECT)
+ return xi3c_master_send_direct_ccc_cmd(master, cmd);
+
+ return xi3c_master_send_bdcast_ccc_cmd(master, cmd);
+}
+
+static int xi3c_master_i3c_xfers(struct i3c_dev_desc *dev,
+ struct i3c_xfer *xfers,
+ int nxfers, enum i3c_xfer_mode mode)
+{
+ struct i3c_master_controller *m = i3c_dev_get_master(dev);
+ struct xi3c_master *master = to_xi3c_master(m);
+ struct xi3c_xfer *xfer __free(kfree) = NULL;
+ int i, ret;
+
+ if (!nxfers)
+ return 0;
+
+ if (mode != I3C_SDR)
+ return -EOPNOTSUPP;
+
+ for (i = 0; i < nxfers; i++)
+ if (xfers[i].len > XI3C_MAXDATA_LENGTH)
+ return -EINVAL;
+
+ xfer = xi3c_master_alloc_xfer(nxfers);
+ if (!xfer)
+ return -ENOMEM;
+
+ for (i = 0; i < nxfers; i++) {
+ struct xi3c_cmd *cmd = &xfer->cmds[i];
+
+ cmd->addr = dev->info.dyn_addr;
+ cmd->rnw = xfers[i].rnw;
+
+ if (cmd->rnw) {
+ cmd->rx_buf = xfers[i].data.in;
+ cmd->rx_len = xfers[i].len;
+ } else {
+ cmd->tx_buf = xfers[i].data.out;
+ cmd->tx_len = xfers[i].len;
+ }
+
+ cmd->type = XI3C_SDR_MODE;
+ cmd->tid = XI3C_SDR_TID;
+ cmd->continued = (i + 1) < nxfers;
+ }
+
+ ret = xi3c_master_common_xfer(master, xfer);
+
+ for (i = 0; i < nxfers; i++) {
+ xfers[i].err = xfer->cmds[i].err;
+ if (xfers[i].rnw)
+ xfers[i].actual_len = xfers[i].len - xfer->cmds[i].rx_len;
+ }
+
+ return ret;
+}
+
+static int xi3c_master_i2c_xfers(struct i2c_dev_desc *dev,
+ struct i2c_msg *xfers,
+ int nxfers)
+{
+ struct i3c_master_controller *m = i2c_dev_get_master(dev);
+ struct xi3c_master *master = to_xi3c_master(m);
+ struct xi3c_xfer *xfer __free(kfree) = NULL;
+ int i;
+
+ if (!nxfers)
+ return 0;
+
+ for (i = 0; i < nxfers; i++)
+ if (xfers[i].len > XI3C_MAXDATA_LENGTH)
+ return -EINVAL;
+
+ xfer = xi3c_master_alloc_xfer(nxfers);
+ if (!xfer)
+ return -ENOMEM;
+
+ for (i = 0; i < nxfers; i++) {
+ struct xi3c_cmd *cmd = &xfer->cmds[i];
+
+ cmd->addr = xfers[i].addr & XI3C_ADDR_MASK;
+ cmd->rnw = !!(xfers[i].flags & I2C_M_RD);
+
+ if (cmd->rnw) {
+ cmd->rx_buf = xfers[i].buf;
+ cmd->rx_len = xfers[i].len;
+ } else {
+ cmd->tx_buf = xfers[i].buf;
+ cmd->tx_len = xfers[i].len;
+ }
+
+ cmd->type = XI3C_I2C_MODE;
+ cmd->tid = XI3C_I2C_TID;
+ cmd->continued = (i + 1) < nxfers;
+ }
+
+ return xi3c_master_common_xfer(master, xfer);
+}
+
+static int xi3c_clk_cfg(struct xi3c_master *master, unsigned long sclhz, u8 mode)
+{
+ unsigned long core_rate, core_periodns;
+ u32 tcasmin, tsustart, tsustop, thdstart;
+ u32 thigh, tlow, thold;
+ u32 odthigh, odtlow;
+
+ core_rate = clk_get_rate(master->pclk);
+ if (!core_rate)
+ return -EINVAL;
+
+ if (!sclhz)
+ return -EINVAL;
+
+ core_periodns = DIV_ROUND_UP(NSEC_PER_SEC, core_rate);
+
+ thigh = DIV_ROUND_UP(core_rate, sclhz) >> 1;
+ tlow = thigh;
+
+ if (thigh <= XI3C_CYCLE_ADJUST)
+ return -EINVAL;
+
+ /* Hold time : 40% of tlow time */
+ thold = (tlow * 4) / 10;
+
+ if (xi3c_get_revision_number(master) == 0)
+ thold = max_t(u32, thold, XI3C_THOLD_MIN_REV0);
+ else
+ thold = max_t(u32, thold, XI3C_THOLD_MIN_REV1);
+
+ iowrite32((thigh - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_SCL_HIGH_TIME_OFFSET);
+ iowrite32((tlow - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_SCL_LOW_TIME_OFFSET);
+ iowrite32((thold - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_SDA_HOLD_TIME_OFFSET);
+
+ if (mode == XI3C_I2C_MODE) {
+ iowrite32((thigh - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_OD_SCL_HIGH_TIME_OFFSET);
+ iowrite32((tlow - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_OD_SCL_LOW_TIME_OFFSET);
+
+ tcasmin = DIV_ROUND_UP(XI3C_I2C_TCASMIN_NS, core_periodns);
+ } else {
+ odtlow = DIV_ROUND_UP(XI3C_OD_TLOW_NS, core_periodns);
+ odthigh = DIV_ROUND_UP(XI3C_OD_THIGH_NS, core_periodns);
+
+ odtlow = max(tlow, odtlow);
+ odthigh = min(thigh, odthigh);
+
+ if (odthigh <= XI3C_CYCLE_ADJUST)
+ return -EINVAL;
+
+ iowrite32((odthigh - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_OD_SCL_HIGH_TIME_OFFSET);
+ iowrite32((odtlow - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_OD_SCL_LOW_TIME_OFFSET);
+
+ tcasmin = DIV_ROUND_UP(XI3C_TCASMIN_NS, core_periodns);
+ }
+
+ thdstart = max(thigh, tcasmin);
+ tsustart = max(tlow, tcasmin);
+ tsustop = max(tlow, tcasmin);
+
+ iowrite32((tsustart - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_TSU_START_OFFSET);
+ iowrite32((thdstart - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_THD_START_OFFSET);
+ iowrite32((tsustop - XI3C_CYCLE_ADJUST) & XI3C_TIMING_MASK,
+ master->membase + XI3C_TSU_STOP_OFFSET);
+
+ return 0;
+}
+
+static int xi3c_master_bus_init(struct i3c_master_controller *m)
+{
+ struct xi3c_master *master = to_xi3c_master(m);
+ struct i3c_bus *bus = i3c_master_get_bus(m);
+ struct i3c_device_info info = {};
+ unsigned long sclhz;
+ u32 pid1_bcr_dcr;
+ u8 mode;
+ int ret;
+
+ switch (bus->mode) {
+ case I3C_BUS_MODE_MIXED_FAST:
+ case I3C_BUS_MODE_MIXED_LIMITED:
+ case I3C_BUS_MODE_MIXED_SLOW:
+ mode = XI3C_I2C_MODE;
+ sclhz = bus->scl_rate.i2c;
+ break;
+ case I3C_BUS_MODE_PURE:
+ mode = XI3C_SDR_MODE;
+ sclhz = bus->scl_rate.i3c;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ret = xi3c_clk_cfg(master, sclhz, mode);
+ if (ret)
+ return ret;
+
+ xi3c_master_init(master);
+
+ /* Get an address for the master. */
+ ret = i3c_master_get_free_addr(m, 0);
+ if (ret < 0)
+ return ret;
+
+ info.dyn_addr = ret;
+
+ /* Write the dynamic address value to the address register. */
+ iowrite32(info.dyn_addr, master->membase + XI3C_ADDRESS_OFFSET);
+
+ /* Read PID, BCR and DCR values, and assign to i3c device info. */
+ pid1_bcr_dcr = ioread32(master->membase + XI3C_PID1_BCR_DCR);
+ info.pid = ((u64)FIELD_GET(XI3C_PID1_MASK, pid1_bcr_dcr) << 32) |
+ ioread32(master->membase + XI3C_PID0_OFFSET);
+ info.bcr = FIELD_GET(XI3C_BCR_MASK, pid1_bcr_dcr);
+ info.dcr = FIELD_GET(XI3C_DCR_MASK, pid1_bcr_dcr);
+
+ return i3c_master_set_info(&master->base, &info);
+}
+
+static void xi3c_master_bus_cleanup(struct i3c_master_controller *m)
+{
+ struct xi3c_master *master = to_xi3c_master(m);
+
+ xi3c_master_disable(master);
+}
+
+static const struct i3c_master_controller_ops xi3c_master_ops = {
+ .bus_init = xi3c_master_bus_init,
+ .bus_cleanup = xi3c_master_bus_cleanup,
+ .do_daa = xi3c_master_do_daa,
+ .supports_ccc_cmd = xi3c_master_supports_ccc_cmd,
+ .send_ccc_cmd = xi3c_master_send_ccc_cmd,
+ .i3c_xfers = xi3c_master_i3c_xfers,
+ .i2c_xfers = xi3c_master_i2c_xfers,
+};
+
+static int xi3c_master_probe(struct platform_device *pdev)
+{
+ struct xi3c_master *master;
+ int ret;
+
+ master = devm_kzalloc(&pdev->dev, sizeof(*master), GFP_KERNEL);
+ if (!master)
+ return -ENOMEM;
+
+ master->dev = &pdev->dev;
+
+ master->membase = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(master->membase))
+ return dev_err_probe(master->dev, PTR_ERR(master->membase),
+ "Failed to map registers\n");
+
+ master->pclk = devm_clk_get_enabled(master->dev, NULL);
+ if (IS_ERR(master->pclk))
+ return dev_err_probe(master->dev, PTR_ERR(master->pclk),
+ "Failed to get and enable clock\n");
+
+ ret = devm_mutex_init(master->dev, &master->lock);
+ if (ret)
+ return ret;
+
+ platform_set_drvdata(pdev, master);
+
+ return i3c_master_register(&master->base, master->dev,
+ &xi3c_master_ops, false);
+}
+
+static void xi3c_master_remove(struct platform_device *pdev)
+{
+ struct xi3c_master *master = platform_get_drvdata(pdev);
+
+ i3c_master_unregister(&master->base);
+}
+
+static const struct of_device_id xi3c_master_of_ids[] = {
+ { .compatible = "xlnx,axi-i3c-1.0" },
+ { },
+};
+MODULE_DEVICE_TABLE(of, xi3c_master_of_ids);
+
+static struct platform_driver xi3c_master_driver = {
+ .probe = xi3c_master_probe,
+ .remove = xi3c_master_remove,
+ .driver = {
+ .name = "axi-i3c-master",
+ .of_match_table = xi3c_master_of_ids,
+ },
+};
+module_platform_driver(xi3c_master_driver);
+
+MODULE_AUTHOR("Manikanta Guntupalli <manikanta.guntupalli@amd.com>");
+MODULE_AUTHOR("Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>");
+MODULE_AUTHOR("Shubham Patil <shubhamsanjay.patil@amd.com>");
+MODULE_DESCRIPTION("AMD AXI I3C master driver");
+MODULE_LICENSE("GPL");
--
2.34.1
^ permalink raw reply related
* [PATCH v9 1/2] dt-bindings: i3c: Add AMD I3C master controller support
From: Shubham Patil @ 2026-06-23 11:44 UTC (permalink / raw)
To: git, michal.simek, alexandre.belloni, Frank.Li, robh, krzk+dt,
conor+dt, pgaj, wsa+renesas, tommaso.merciai.xr, arnd,
quic_msavaliy, Shyam-sundar.S-k, sakari.ailus, billy_tsai, kees,
gustavoars, jarkko.nikula, jorge.marques, linux-i3c, devicetree,
linux-kernel, linux-arch, linux-hardening
Cc: radhey.shyam.pandey, srinivas.goud, shubhrajyoti.datta,
shubhamsanjay.patil, Manikanta Guntupalli
In-Reply-To: <20260623114417.2578189-1-shubhamsanjay.patil@amd.com>
From: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
Add device tree binding documentation for the AMD I3C master controller
version 1.0.
Signed-off-by: Manikanta Guntupalli <manikanta.guntupalli@amd.com>
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
---
Changes for V9:
None.
Changes for V8:
None.
Changes for V7:
Added i3c controller version details to commit description.
Changes for V6:
Corrected the file name for $id in yaml to fix the dtschema warning.
Changes for V5:
Renamed the xlnx,axi-i3c.yaml file into xlnx,axi-i3c-1.0.yaml.
Changes for V4:
Added h/w documentation details.
Changes for V3:
Updated commit description.
Corrected the order of properties and removed resets property.
Added compatible to required list.
Added interrupts to example.
Changes for V2:
Updated commit subject and description.
Moved allOf to after required.
Removed xlnx,num-targets property.
---
.../bindings/i3c/xlnx,axi-i3c-1.0.yaml | 56 +++++++++++++++++++
1 file changed, 56 insertions(+)
create mode 100644 Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
diff --git a/Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml b/Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
new file mode 100644
index 000000000000..75f677696f02
--- /dev/null
+++ b/Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/i3c/xlnx,axi-i3c-1.0.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: AMD I3C master
+
+maintainers:
+ - Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
+ - Shubham Patil <shubhamsanjay.patil@amd.com>
+
+description:
+ The AXI-I3C IP is an I3C Controller with an AXI4-Lite interface, compatible
+ with the MIPI I3C Specification v1.1.1. The design includes bidirectional I/O
+ buffers that implement open collector drivers for the SDA and SCL signals.
+ External pull-up resistors are required to properly hold the bus at a Logic-1
+ level when the drivers are released.
+
+ For more details, please see https://docs.amd.com/r/en-US/pg439-axi-i3c
+
+properties:
+ compatible:
+ const: xlnx,axi-i3c-1.0
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - clocks
+
+allOf:
+ - $ref: i3c.yaml#
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ i3c@80000000 {
+ compatible = "xlnx,axi-i3c-1.0";
+ reg = <0x80000000 0x10000>;
+ clocks = <&zynqmp_clk 71>;
+ interrupt-parent = <&imux>;
+ interrupts = <0 89 4>;
+ #address-cells = <3>;
+ #size-cells = <0>;
+ };
+...
--
2.34.1
^ permalink raw reply related
* [PATCH v9 0/2] Add AMD I3C master controller driver and bindings
From: Shubham Patil @ 2026-06-23 11:44 UTC (permalink / raw)
To: git, michal.simek, alexandre.belloni, Frank.Li, robh, krzk+dt,
conor+dt, pgaj, wsa+renesas, tommaso.merciai.xr, arnd,
quic_msavaliy, Shyam-sundar.S-k, sakari.ailus, billy_tsai, kees,
gustavoars, jarkko.nikula, jorge.marques, linux-i3c, devicetree,
linux-kernel, linux-arch, linux-hardening
Cc: radhey.shyam.pandey, srinivas.goud, shubhrajyoti.datta,
shubhamsanjay.patil
This patch series introduces support for the AMD I3C master controller,
including the device tree binding and driver implementation.
Compared to v8, the big-endian MMIO accessor and i3c FIFO-endianness
patches have been dropped; the driver now handles big-endian FIFO
accesses internally.
Note: There was an extended gap since v8 due to the transfer of ownership
of this series from Manikanta to Shubham. This transition contributed
to the delay in releasing the v9 update addressing the v8 review comments.
We appreciate your patience.
---
Changes for V9:
Dropped the three big-endian MMIO/FIFO infrastructure patches; the
driver now handles big-endian FIFO accesses internally.
Replaced the async transfer-queue with a synchronous transfer path.
Reworked error/response handling using enum i3c_error_code and proper
-ENODEV/-EIO returns; propagate err to CCCs and priv xfers.
Switched to the new .i3c_xfers op (reject non-SDR, report actual_len).
Reworked DAA (incremental address assignment, -ENOSPC bound, -ENODEV
end-of-enumeration, checked device registration).
Avoid busy-spin with usleep_range(); use FIELD_PREP() and inline
helpers; split the timeout macros with documented units.
Dropped ENTHDR (SDR-only); updated MAINTAINERS, Kconfig symbol
(AMD_AXI_I3C_MASTER), authors and binding maintainers.
Changes for V8:
Included dependent patch "i3c: fix big-endian FIFO transfers"
to this series as [3/5].
Resolved conflicts with "i3c: fix big-endian FIFO transfers".
Updated description.
Used time_left instead of timeout.
Used __free(kfree) for xfer to simplify err path in multiple places.
Changes for V7:
Added i3c controller version details to commit description.
Added Reviewed-by tag to binding patch [1/4].
Added big-endian MMIO accessors [2/4].
Added endianness support for i3c_readl_fifo() and i3c_writel_fifo() [3/4].
Updated timeout macro name.
Updated xi3c_master_wr_to_tx_fifo() and xi3c_master_rd_from_rx_fifo()
to use i3c_writel_fifo() and i3c_readl_fifo().
Changes for V6:
Corrected the $id in the YAML file to match the filename and fix
the dtschema warning.
Removed typecast for xi3c_getrevisionnumber(), xi3c_wrfifolevel(),
and xi3c_rdfifolevel().
Replaced dynamic allocation with a static variable for pid_bcr_dcr.
Fixed sparse warning in do_daa by typecasting the address parity value
to u8.
Fixed sparse warning in xi3c_master_bus_init by typecasting the pid value
to u64 in info.pid calculation.
Changes for V5:
Renamed the xlnx,axi-i3c.yaml file into xlnx,axi-i3c-1.0.yaml.
Used GENMASK_ULL for PID mask as it's 64bit mask.
Changes for V4:
Added h/w documentation details.
Updated timeout macros.
Removed type casting for xi3c_is_resp_available() macro.
Used ioread32() and iowrite32() instead of readl() and writel()
to keep consistency.
Read XI3C_RESET_OFFSET reg before udelay().
Removed xi3c_master_free_xfer() and directly used kfree().
Skipped checking return value of i3c_master_add_i3c_dev_locked().
Used devm_mutex_init() instead of mutex_init().
Changes for V3:
Updated commit description.
Corrected the order of properties and removed resets property.
Added compatible to required list.
Added interrupts to example.
Resolved merge conflicts.
Changes for V2:
Updated commit subject and description.
Moved allOf to after required.
Removed xlnx,num-targets property.
Added mixed mode support with clock configuration.
Converted smaller functions into inline functions.
Used FIELD_GET() in xi3c_get_response().
Updated xi3c_master_rd_from_rx_fifo() to use cmd->rx_buf.
Used parity8() for address parity calculation.
Added guards for locks.
Dropped num_targets and updated xi3c_master_do_daa().
Used __free(kfree) in xi3c_master_send_bdcast_ccc_cmd().
Dropped PM runtime support.
Updated xi3c_master_read() and xi3c_master_write() with
xi3c_is_resp_available() check.
Created separate functions: xi3c_master_init() and xi3c_master_reinit().
Used xi3c_master_init() in bus initialization and xi3c_master_reinit()
in error paths.
Added DAA structure to xi3c_master structure.
---
Manikanta Guntupalli (2):
dt-bindings: i3c: Add AMD I3C master controller support
i3c: master: Add driver for AMD AXI I3C master controller
.../bindings/i3c/xlnx,axi-i3c-1.0.yaml | 56 +
MAINTAINERS | 8 +
drivers/i3c/master/Kconfig | 15 +
drivers/i3c/master/Makefile | 1 +
drivers/i3c/master/amd-i3c-master.c | 1060 +++++++++++++++++
5 files changed, 1140 insertions(+)
create mode 100644 Documentation/devicetree/bindings/i3c/xlnx,axi-i3c-1.0.yaml
create mode 100644 drivers/i3c/master/amd-i3c-master.c
--
2.34.1
^ permalink raw reply
* Re: [PATCH v4 5/8] riscv/runtime-const: Introduce runtime_const_mask_32()
From: Charlie Jenkins @ 2026-06-23 7:01 UTC (permalink / raw)
To: K Prateek Nayak
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Sebastian Andrzej Siewior, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Guo Ren, Darren Hart, Davidlohr Bueso,
André Almeida, linux-arch, linux-kernel, linux-s390,
linux-riscv, linux-arm-kernel, Alexandre Ghiti, Charlie Jenkins,
Jisheng Zhang, Charles Mirabile
In-Reply-To: <ff9678fb-4cca-4849-8ffb-7cb76db60e1a@amd.com>
On Tue, Jun 23, 2026 at 11:43:39AM +0530, K Prateek Nayak wrote:
> Hello Charlie,
>
> On 6/23/2026 10:54 AM, Charlie Jenkins wrote:
> > On Thu, 30 Apr 2026 09:47:27 +0000, K Prateek Nayak <kprateek.nayak@amd.com> wrote:
> >> Futex hash computation requires a mask operation with read-only after
> >> init data that will be converted to a runtime constant in the subsequent
> >> commit.
> >>
> >> Introduce runtime_const_mask_32 to further optimize the mask operation
> >> in the futex hash computation hot path. GCC generates a:
> >>
> >> lui a0, 0x12346 # upper; +0x800 then >>12 for correct rounding
> >> addi a0, a0, 0x678 # lower 12 bits
> >> and a1, a1, a0 # a1 = a1 & a0
> >>
> >> pattern to tackle arbitrary 32-bit masks and the same was also suggested
> >> by Claude which is implemented here. The final (__ret & val) operation
> >> is intentionally placed outside of asm block to allow compilers to
> >> further optimize it if possible.
> >
> > If the mask fits in 12 bits, we can nop the lui and the addi and just
> > patch an "andi" instruction with the 12 bits of the mask. We already do
> > this with the lui+addi block and nop the lui if val fits in 12 bits. I
> > would be happy to help draft that optimization.
> >
> > But I think the better solution would be to take the power of 2
> > assumption since that will also benefit arm. We should still only emit
> > an andi if val fits in 12 bits, but if it doesn't we can patch in
> > shifts:
> >
> > slli a0,a0,x
> > srli a0,a0,x
> >
> > Where x is the constant (arch_size - _futex_shift - 1)
>
> I can do that for the next version and use ubfx for ARM. I can just put
> in a BUG_ON() at the arch/ specific __runtime_fixup_mask() and if a
> new use case arises which hits that, we can perhaps move on the dynamic
> nop patching scheme that you mentioned earlier.
>
> Let me know if that works and I can pivot to that scheme in v5 and send
> it out post -rc1 after some testing.
That sounds like a great plan :)
- Charlie
>
> --
> Thanks and Regards,
> Prateek
>
^ permalink raw reply
* [RFC] signal: per-thread control over alternate signal stack delivery for selected signals
From: Tim Parth @ 2026-06-23 6:30 UTC (permalink / raw)
To: linux-api@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org
Hi,
I am looking for guidance on a Linux signal ABI limitation that shows up in multi-runtime processes, specifically a .NET host loading a Go c-shared library.
Disclaimer: I am reporting this from the application/runtime integration side, not as a kernel developer. I arrived here after tracing crashes in a .NET application hosting a Go shared library through several runtime-specific issues, reproductions, and analyses. My understanding of the Linux signal subsystem and ABI details is therefore limited, and I may be missing important details.
The technical summary below reflects my best understanding of the issue based on the referenced investigations. I used AI-assisted editing to help structure and clarify this report, but the observations, reproducer, and referenced analyses come from the linked investigations.
This is not a claim that the current kernel behavior violates the existing ABI. Rather, I believe the current ABI lacks a way for multiple language runtimes in the same process to compose their signal and sigaltstack requirements safely.
Observed failure
================
A .NET process loads a Go shared library built with -buildmode=c-shared and calls it via P/Invoke. Under stress, the process crashes with SIGSEGV while CoreCLR is handling SIGRTMIN for runtime activation / GC suspension.
The reproducer is here:
https://github.com/egonelbre/csharp-go-interop-issue/tree/main/dotnet-go-reproducer
Related runtime issues:
https://github.com/golang/go/issues/78883
https://github.com/dotnet/runtime/issues/127320
The .NET-side analysis shows that the crash happens inside CoreCLR's inject_activation_handler path. The kernel delivered SIGRTMIN on the thread's alternate signal stack, and CoreCLR then ran a call chain deep enough to overflow that stack. In the reported case the per-thread alternate stack installed by CoreCLR was 16 KiB. Increasing it to around 49 KiB avoids the crash in the provided stress test, but that is a runtime-specific mitigation and does not address the general ABI composition problem.
Current ABI interaction
=======================
The problematic interaction is:
1. Signal disposition, including SA_ONSTACK, is per-process.
2. sigaltstack is per-thread.
3. On signal delivery, Linux uses the alternate signal stack if the handler has SA_ONSTACK and the current thread has an alternate stack.
4. The Go runtime documents that non-Go signal handlers must use SA_ONSTACK, because Go may be running on limited stacks. For -buildmode=c-shared, when Go sees an existing signal handler it may turn on SA_ONSTACK and otherwise keep the existing handler.
5. CoreCLR has internal signals such as SIGRTMIN whose handlers may need a different stack policy or a larger stack budget than the alternate stack currently registered on that thread.
The result is that one runtime can make a process-wide SA_ONSTACK decision that affects handlers and threads owned by another runtime. The other runtime can install a larger per-thread sigaltstack, but that becomes an arms race and does not give a runtime any way to express which signals should use which stack policy on a particular thread.
Why existing mechanisms do not fully solve this
===============================================
- Raising SIGSTKSZ or MINSIGSTKSZ does not solve the general issue. The kernel can only know the signal frame requirements, not the maximum user-space stack consumption of an arbitrary signal handler and everything it calls.
- The kernel cannot automatically extend an alternate signal stack.
- Clearing SA_ONSTACK with sigaction is process-wide and can violate the requirements of another runtime, for example Go's requirement that signal handlers run on an alternate stack when Go code may be interrupted.
- SS_AUTODISARM helps with a different class of problems, such as avoiding corruption when switching away from a signal handler, but it does not let a thread express "use an alternate stack for SIGSEGV but not for this runtime-internal suspension signal", nor does it provide separate stack policies for different signals.
Possible ABI direction
======================
One possible direction would be an opt-in, per-thread signal-altstack policy, for example a prctl() or similar interface that lets a thread provide a signal mask for which SA_ONSTACK should be ignored on that thread:
PR_SET_SIGALTSTACK_EXCLUDE_MASK(sigset_t *mask, size_t sigsetsize)
The default mask would be empty, preserving current behavior. Signal delivery would then become, conceptually:
if (handler_has_SA_ONSTACK &&
thread_has_altstack &&
!signal_is_in_current_thread_altstack_exclude_mask)
deliver_on_altstack;
else
deliver_on_normal_stack;
This is only a sketch. I am not attached to this exact interface. Another shape might be preferable, such as a more general per-thread/per-signal alternate stack policy or a way to associate alternate stack requirements with particular signals.
Questions
=========
1. Is the signal maintainers' view that multi-runtime processes should solve this entirely in userspace by agreeing on one sufficiently large per-thread sigaltstack?
2. Would a per-thread/per-signal opt-in policy for alternate signal stack delivery be considered acceptable as a Linux UAPI extension?
3. If such a UAPI is plausible, is prctl() the right place, or would maintainers prefer a different interface?
4. Which subsystem/list should own this discussion? I am sending this first to linux-api and LKML because this appears to be a userspace ABI issue around signal delivery.
Environment from the reproducer report
======================================
- Architecture: x86_64
- OS: Linux
- Example distro: Ubuntu 24.04
- Go: go1.26.2 linux/amd64
- .NET: 10.0.6 and runtime main were tested in the linked report
- Signal involved in the reproducer: SIGRTMIN
- Failure mode: SIGSEGV while running CoreCLR activation handling on the
alternate signal stack
Thanks,
Tim Parth
^ permalink raw reply
* Re: [PATCH v4 5/8] riscv/runtime-const: Introduce runtime_const_mask_32()
From: K Prateek Nayak @ 2026-06-23 6:13 UTC (permalink / raw)
To: Charlie Jenkins
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Sebastian Andrzej Siewior, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Guo Ren, Darren Hart, Davidlohr Bueso,
André Almeida, linux-arch, linux-kernel, linux-s390,
linux-riscv, linux-arm-kernel, Alexandre Ghiti, Charlie Jenkins,
Jisheng Zhang, Charles Mirabile
In-Reply-To: <178219229643.10927.7189200920480581019.b4-review@b4>
Hello Charlie,
On 6/23/2026 10:54 AM, Charlie Jenkins wrote:
> On Thu, 30 Apr 2026 09:47:27 +0000, K Prateek Nayak <kprateek.nayak@amd.com> wrote:
>> Futex hash computation requires a mask operation with read-only after
>> init data that will be converted to a runtime constant in the subsequent
>> commit.
>>
>> Introduce runtime_const_mask_32 to further optimize the mask operation
>> in the futex hash computation hot path. GCC generates a:
>>
>> lui a0, 0x12346 # upper; +0x800 then >>12 for correct rounding
>> addi a0, a0, 0x678 # lower 12 bits
>> and a1, a1, a0 # a1 = a1 & a0
>>
>> pattern to tackle arbitrary 32-bit masks and the same was also suggested
>> by Claude which is implemented here. The final (__ret & val) operation
>> is intentionally placed outside of asm block to allow compilers to
>> further optimize it if possible.
>
> If the mask fits in 12 bits, we can nop the lui and the addi and just
> patch an "andi" instruction with the 12 bits of the mask. We already do
> this with the lui+addi block and nop the lui if val fits in 12 bits. I
> would be happy to help draft that optimization.
>
> But I think the better solution would be to take the power of 2
> assumption since that will also benefit arm. We should still only emit
> an andi if val fits in 12 bits, but if it doesn't we can patch in
> shifts:
>
> slli a0,a0,x
> srli a0,a0,x
>
> Where x is the constant (arch_size - _futex_shift - 1)
I can do that for the next version and use ubfx for ARM. I can just put
in a BUG_ON() at the arch/ specific __runtime_fixup_mask() and if a
new use case arises which hits that, we can perhaps move on the dynamic
nop patching scheme that you mentioned earlier.
Let me know if that works and I can pivot to that scheme in v5 and send
it out post -rc1 after some testing.
--
Thanks and Regards,
Prateek
^ permalink raw reply
* Re: [PATCH v4 5/8] riscv/runtime-const: Introduce runtime_const_mask_32()
From: Charlie Jenkins @ 2026-06-23 5:24 UTC (permalink / raw)
To: K Prateek Nayak
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Sebastian Andrzej Siewior, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Guo Ren, Darren Hart, Davidlohr Bueso,
André Almeida, linux-arch, linux-kernel, linux-s390,
linux-riscv, linux-arm-kernel, Alexandre Ghiti, Charlie Jenkins,
Jisheng Zhang, Charles Mirabile
In-Reply-To: <20260430094730.31624-6-kprateek.nayak@amd.com>
On Thu, 30 Apr 2026 09:47:27 +0000, K Prateek Nayak <kprateek.nayak@amd.com> wrote:
> Futex hash computation requires a mask operation with read-only after
> init data that will be converted to a runtime constant in the subsequent
> commit.
>
> Introduce runtime_const_mask_32 to further optimize the mask operation
> in the futex hash computation hot path. GCC generates a:
>
> lui a0, 0x12346 # upper; +0x800 then >>12 for correct rounding
> addi a0, a0, 0x678 # lower 12 bits
> and a1, a1, a0 # a1 = a1 & a0
>
> pattern to tackle arbitrary 32-bit masks and the same was also suggested
> by Claude which is implemented here. The final (__ret & val) operation
> is intentionally placed outside of asm block to allow compilers to
> further optimize it if possible.
If the mask fits in 12 bits, we can nop the lui and the addi and just
patch an "andi" instruction with the 12 bits of the mask. We already do
this with the lui+addi block and nop the lui if val fits in 12 bits. I
would be happy to help draft that optimization.
But I think the better solution would be to take the power of 2
assumption since that will also benefit arm. We should still only emit
an andi if val fits in 12 bits, but if it doesn't we can patch in
shifts:
slli a0,a0,x
srli a0,a0,x
Where x is the constant (arch_size - _futex_shift - 1)
- Charlie
--
Charlie Jenkins <thecharlesjenkins@gmail.com>
^ permalink raw reply
* Re: [PATCH v4 4/8] riscv/runtime-const: Replace open-coded placeholder with RUNTIME_MAGIC
From: Charlie Jenkins @ 2026-06-23 5:24 UTC (permalink / raw)
To: K Prateek Nayak
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Sebastian Andrzej Siewior, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Guo Ren, Darren Hart, Davidlohr Bueso,
André Almeida, linux-arch, linux-kernel, linux-s390,
linux-riscv, linux-arm-kernel, Alexandre Ghiti, Charlie Jenkins,
Jisheng Zhang, Charles Mirabile
In-Reply-To: <20260430094730.31624-5-kprateek.nayak@amd.com>
On Thu, 30 Apr 2026 09:47:26 +0000, K Prateek Nayak <kprateek.nayak@amd.com> wrote:
> Define the placeholder used for lui + addi[w] patching sequence as
> RUNTIME_MAGIC and use that instead of open coding the constants in the
> inline assembly.
>
> No functional changes intended.
>
> Suggested-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
This is a great addition, thank you!
Reviewed-by: Charlie Jenkins <thecharlesjenkins@gmail.com>
Tested-by: Charlie Jenkins <thecharlesjenkins@gmail.com>
--
Charlie Jenkins <thecharlesjenkins@gmail.com>
^ permalink raw reply
* Re: [PATCH v4 3/8] arm64/runtime-const: Introduce runtime_const_mask_32()
From: Charlie Jenkins @ 2026-06-23 5:24 UTC (permalink / raw)
To: K Prateek Nayak
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Sebastian Andrzej Siewior, Catalin Marinas, Will Deacon,
Darren Hart, Davidlohr Bueso, André Almeida, linux-arch,
linux-kernel, linux-s390, linux-riscv, linux-arm-kernel,
Jisheng Zhang
In-Reply-To: <20260430094730.31624-4-kprateek.nayak@amd.com>
On Thu, 30 Apr 2026 09:47:25 +0000, K Prateek Nayak <kprateek.nayak@amd.com> wrote:
> [...]
> is intentiaonally placed outside of asm block to allow compilers to
> further optimize it if possible.
>
> __runtime_fixup_ptr() already patches a "movz, + movk lsl #16" sequence
> which has been reused to patch the same sequence for
> __runtime_fixup_mask().
I think we should opt to do power of 2 masks instead of arbitrary masks
since that is what the usecase is and we can reduce these three
instructions into a single instruction for arm (and benefit riscv as
well). It is convenient to have this function as arbitrary masks, but
since we are going down the route of hyper optimizations already with
runtime constants here (and we can always add an arbitrary mask later if
there becomes a usecase), I feel like we should go all the way :)
The and immediate instruction for arm supports all power of 2 masks so
you can do:
and w0, w0, #0xMASK
- Charlie
--
Charlie Jenkins <thecharlesjenkins@gmail.com>
^ permalink raw reply
* [PATCH v2] module: Extend module_blacklist parameter to built-in modules
From: Aaron Tomlin @ 2026-06-22 14:02 UTC (permalink / raw)
To: arnd, mcgrof, petr.pavlu, da.gomez, samitolvanen, peterz
Cc: mhiramat, atomlin, neelx, da.anzani, sean, chjohnst, steve,
mproche, nick.lane, linux-arch, linux-modules, linux-kernel
Currently, the module_blacklist= parameter only prevents
the dynamic loading of external modules. It possesses no mechanism to
intercept or prevent the initialisation of built-in modules, as their
associated initcalls are invoked unconditionally during system boot.
This patch extends the blacklisting behaviour to encompass built-in
modules. It introduces a dedicated ".initcall.modnames" section into
the linker script, systematically mapping each initcall to its
originating module name. During the boot sequence, do_one_initcall()
interrogates this mapping; should the executing initcall belong to a
blacklisted module, its execution is explicitly bypassed.
Furthermore, to preserve the efficacy of Kernel Address Space Layout
Randomisation (KASLR) and prevent binary bloat, the mapping mechanism
rigorously adheres to CONFIG_HAVE_ARCH_PREL32_RELOCATIONS. Rather than
storing absolute function pointers, it employs 32-bit relative offsets,
successfully avoiding the generation of thousands of absolute
relocations.
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
--
Changes since v1:
- Pivoted entirely from exposing built-in initcalls and their blacklist
status via a debugfs interface to directly extending the existing
"module_blacklist=" to intercept built-in modules at boot (Petr Pavlu)
- Implemented 32-bit relative offsets (CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)
to store the mappings, preventing binary bloat and preserving KASLR
efficacy
- Linked to v1: https://lore.kernel.org/lkml/20260510061301.41341-1-atomlin@atomlin.com/
---
include/asm-generic/vmlinux.lds.h | 7 +++++++
include/linux/init.h | 31 +++++++++++++++++++++++++++++--
include/linux/module.h | 2 ++
init/main.c | 27 +++++++++++++++++++++++++++
kernel/module/main.c | 4 ++--
5 files changed, 67 insertions(+), 4 deletions(-)
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 5659f4b5a125..ac0e5f4f2893 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -957,6 +957,12 @@
#define CON_INITCALL \
BOUNDED_SECTION_POST_LABEL(.con_initcall.init, __con_initcall, _start, _end)
+#define INITCALL_MODNAMES \
+ . = ALIGN(8); \
+ __start_initcall_modnames = .; \
+ KEEP(*(.initcall.modnames)) \
+ __stop_initcall_modnames = .;
+
#define NAMED_SECTION(name) \
. = ALIGN(8); \
name : AT(ADDR(name) - LOAD_OFFSET) \
@@ -1166,6 +1172,7 @@
INIT_SETUP(initsetup_align) \
INIT_CALLS \
CON_INITCALL \
+ INITCALL_MODNAMES \
INIT_RAM_FS \
}
diff --git a/include/linux/init.h b/include/linux/init.h
index 40331923b9f4..212f64a07c73 100644
--- a/include/linux/init.h
+++ b/include/linux/init.h
@@ -125,6 +125,16 @@ static inline initcall_t initcall_from_entry(initcall_entry_t *entry)
}
#endif
+struct initcall_modname {
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+ initcall_entry_t initcall_fn;
+ int modname_offset;
+#else
+ initcall_t initcall_fn;
+ const char *modname;
+#endif
+};
+
extern initcall_entry_t __con_initcall_start[], __con_initcall_end[];
/* Used for constructor calls. */
@@ -270,9 +280,26 @@ extern struct module __this_module;
__initcall_stub(fn, __iid, id), \
__initcall_name(initcall, __iid, id), \
__initcall_section(__sec, __iid))
-
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+#define ___define_initcall(fn, id, __sec) \
+ __unique_initcall(fn, id, __sec, __initcall_id(fn)); \
+ asm(".pushsection \".initcall.modnames\", \"a\"\n" \
+ ".balign 4\n" \
+ ".long " #fn " - .\n" \
+ ".long __initcall_modstr_" #fn #id " - .\n" \
+ ".popsection\n" \
+ ".pushsection .init.rodata, \"a\"\n" \
+ "__initcall_modstr_" #fn #id ": .string \"" KBUILD_MODNAME "\"\n" \
+ ".popsection\n");
+#else
#define ___define_initcall(fn, id, __sec) \
- __unique_initcall(fn, id, __sec, __initcall_id(fn))
+ __unique_initcall(fn, id, __sec, __initcall_id(fn)); \
+ static struct initcall_modname __initcall_modname_##fn##id __used \
+ __section(".initcall.modnames") = { \
+ .initcall_fn = fn, \
+ .modname = KBUILD_MODNAME \
+ };
+#endif
#define __define_initcall(fn, id) ___define_initcall(fn, id, .initcall##id)
diff --git a/include/linux/module.h b/include/linux/module.h
index 7566815fabbe..7e25fcd61b50 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -756,6 +756,8 @@ void *dereference_module_function_descriptor(struct module *mod, void *ptr);
int register_module_notifier(struct notifier_block *nb);
int unregister_module_notifier(struct notifier_block *nb);
+extern bool module_is_blacklisted(const char *module_name);
+
extern void print_modules(void);
static inline bool module_requested_async_probing(struct module *module)
diff --git a/init/main.c b/init/main.c
index e363232b428b..fbdc42859791 100644
--- a/init/main.c
+++ b/init/main.c
@@ -1334,12 +1334,39 @@ static inline void do_trace_initcall_level(const char *level)
}
#endif /* !TRACEPOINTS_ENABLED */
+extern struct initcall_modname __start_initcall_modnames[];
+extern struct initcall_modname __stop_initcall_modnames[];
+
+static const char *initcall_get_modname(initcall_t fn)
+{
+ struct initcall_modname *p;
+
+ for (p = __start_initcall_modnames; p < __stop_initcall_modnames; p++) {
+ if (initcall_from_entry(&p->initcall_fn) == fn) {
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+ return (const char *)offset_to_ptr(&p->modname_offset);
+#else
+ return p->modname;
+#endif
+ }
+ }
+ return NULL;
+}
+
int __init_or_module do_one_initcall(initcall_t fn)
{
int count = preempt_count();
char msgbuf[64];
+ const char *modname;
int ret;
+ modname = initcall_get_modname(fn);
+ if (modname && module_is_blacklisted(modname)) {
+ pr_info("Skipping initcall for blacklisted built-in module %s\n",
+ modname);
+ return 0;
+ }
+
if (initcall_blacklisted(fn))
return -EPERM;
diff --git a/kernel/module/main.c b/kernel/module/main.c
index 46dd8d25a605..02deee5a7480 100644
--- a/kernel/module/main.c
+++ b/kernel/module/main.c
@@ -2921,7 +2921,7 @@ int __weak module_frob_arch_sections(Elf_Ehdr *hdr,
/* module_blacklist is a comma-separated list of module names */
static char *module_blacklist;
-static bool blacklisted(const char *module_name)
+bool module_is_blacklisted(const char *module_name)
{
const char *p;
size_t len;
@@ -3391,7 +3391,7 @@ static int early_mod_check(struct load_info *info, int flags)
* Now that we know we have the correct module name, check
* if it's blacklisted.
*/
- if (blacklisted(info->name)) {
+ if (module_is_blacklisted(info->name)) {
pr_err("Module %s is blacklisted\n", info->name);
return -EPERM;
}
--
2.51.0
^ permalink raw reply related
* Re: [PATCH] riscv: entry: Fixup do_trap_break from kernel side
From: Thomas Gleixner @ 2026-06-22 11:33 UTC (permalink / raw)
To: Vivian Wang, Peter Zijlstra, Guo Ren
Cc: Kees Cook, arnd, palmer, luto, conor.dooley, heiko, jszhang,
lazyparser, falcon, chenhuacai, apatel, atishp, mark.rutland,
bjorn, palmer, bjorn, daniel.thompson, linux-arch, linux-kernel,
linux-riscv, stable, Guo Ren
In-Reply-To: <2f32370b-63c1-4e8a-bf71-d40874b6bebb@iscas.ac.cn>
On Mon, Jun 22 2026 at 18:25, Vivian Wang wrote:
> On 6/22/26 16:28, Peter Zijlstra wrote:
>> I still don't understand it. This cannot fix anything. Consider:
>>
>> EBREAK
>> raw_spin_lock_irq(&your_lock)
>> EBREAK
>>
>> So now the first 'works', but the second will crash. Additionally,
>> having the EBREAK context differ so dramatically between invocations
>> seems like a very bad deal to me.
>
> To spell it out, the problem that needs fixing is:
>
> -> BUG()
> -> ebreak instruction
> -> Breakpoint exception
> -> do_trap_break()
> -> irqentry_nmi_enter()
> [ now in_nmi() / in_interrupt() ]
> -> report_bug() returns BUG_TRAP_TYPE_BUG
> -> die()
> -> make_task_dead()
> -> panic() because we're in_interrupt()
>
> As such, currently on riscv all BUG() simply completely panic() the
> entire machine, rather than just killing the one task.
>
> How do you think this should be fixed? Here are some ideas but I'm not
> familiar with generic entry stuff:
>
> * Should we irqentry_nmi_exit() before calling die() for BUG()?
> * Should we move the GENERIC_BUG trap instruction to cause illegal
> instruction exception instead, for which we can write a simpler
> handler that doesn't need to care about the probe stuff?
Look at how x86 handles UD exceptions.
^ permalink raw reply
* Re: [PATCH] riscv: entry: Fixup do_trap_break from kernel side
From: Peter Zijlstra @ 2026-06-22 11:17 UTC (permalink / raw)
To: Vivian Wang
Cc: Guo Ren, Kees Cook, arnd, palmer, tglx, luto, conor.dooley, heiko,
jszhang, lazyparser, falcon, chenhuacai, apatel, atishp,
mark.rutland, bjorn, palmer, bjorn, daniel.thompson, linux-arch,
linux-kernel, linux-riscv, stable, Guo Ren
In-Reply-To: <2f32370b-63c1-4e8a-bf71-d40874b6bebb@iscas.ac.cn>
On Mon, Jun 22, 2026 at 06:25:13PM +0800, Vivian Wang wrote:
> > I still don't understand it. This cannot fix anything. Consider:
> >
> > EBREAK
> > raw_spin_lock_irq(&your_lock)
> > EBREAK
> >
> > So now the first 'works', but the second will crash. Additionally,
> > having the EBREAK context differ so dramatically between invocations
> > seems like a very bad deal to me.
>
> To spell it out, the problem that needs fixing is:
>
> -> BUG()
> -> ebreak instruction
> -> Breakpoint exception
> -> do_trap_break()
> -> irqentry_nmi_enter()
> [ now in_nmi() / in_interrupt() ]
> -> report_bug() returns BUG_TRAP_TYPE_BUG
> -> die()
> -> make_task_dead()
> -> panic() because we're in_interrupt()
>
> As such, currently on riscv all BUG() simply completely panic() the
> entire machine, rather than just killing the one task.
Hmm, from reading some of the previous emails this morning, I got the
impression the problem was with kgdb, not BUG().
Anyway, my argument doesn't change, with the proposed patch:
BUG()
and:
local_irq_disable();
BUG();
will behave quite differently, for no sane reason.
Anyway, BUG()/trap is indeed a bit of magic, the x86 code lives in
arch/x86/kernel/traps.c:exc_invalid_op(). And it looks like we do not
indeed use NMI-like for this path, although I cannot remember why.
*however* I see your kgdb thing also uses ebreak, whereas on x86
WARN/BUG and kGDB use different exceptions (#UD for WARN/BUG and #BP for
gdb). And our #BP handler (exc_int3) very much does NMI for from-kernel.
Same for kprobes, we use #BP/int3 for that, you also have that in
EBREAK.
Anyway, you're handling 3 different cases in one exception, which is a
bit of a mess, but something like so perhaps?
---
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index 8c62c771a656..41c7faac7eb3 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -264,42 +264,58 @@ static inline unsigned long get_break_insn_length(unsigned long pc)
return GET_INSN_LENGTH(insn);
}
-static bool probe_single_step_handler(struct pt_regs *regs)
+static void handle_kernel_die(struct pt_regs *regs)
{
- bool user = user_mode(regs);
-
- return user ? uprobe_single_step_handler(regs) : kprobe_single_step_handler(regs);
+ irqentry_state_t state = irqentry_enter(regs);
+ die(regs, "Kernel BUG");
+ irqentry_exit(regs, state);
}
-static bool probe_breakpoint_handler(struct pt_regs *regs)
+static bool handle_kernel_bug(struct pt_regs *regs)
{
- bool user = user_mode(regs);
+ if (report_bug(regs->epc, regs) == BUG_TRAP_TYPE_WARN ||
+ handle_cfi_failure(regs) == BUG_TRAP_TYPE_WARN) {
+ regs->epc += get_break_insn_length(regs->epc);
+ return true;
+ }
- return user ? uprobe_breakpoint_handler(regs) : kprobe_breakpoint_handler(regs);
+ return false;
}
-void handle_break(struct pt_regs *regs)
+static bool __handle_kernel_break(struct pt_regs *regs)
{
- if (probe_single_step_handler(regs))
- return;
- if (probe_breakpoint_handler(regs))
+ if (kprobe_single_step_handler(regs) ||
+ kprobe_breakpoint_handler(regs))
+ return true;
+
+ current->thread.bad_cause = regs->cause;
+
+#ifdef CONFIG_KGDB
+ if (notify_die(DIE_TRAP, "EBREAK", regs, 0, regs->cause, SIGTRAP)
+ == NOTIFY_STOP)
+ return true;
+#endif
+ return false;
+}
+
+static bool handle_kernel_break(struct pt_regs *regs)
+{
+ irqentry_state_t state = irqentry_nmi_enter(regs);
+ bool ret = __handle_kernel_break(regs);
+ irqentry_nmi_exit(regs, state);
+ return ret;
+}
+
+static void handle_user_break(struct pt_regs *regs)
+{
+ if (uprobe_single_step_handler(regs) ||
+ uprobe_breakpoint_handler(regs))
return;
current->thread.bad_cause = regs->cause;
- if (user_mode(regs))
- force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)regs->epc);
-#ifdef CONFIG_KGDB
- else if (notify_die(DIE_TRAP, "EBREAK", regs, 0, regs->cause, SIGTRAP)
- == NOTIFY_STOP)
- return;
-#endif
- else if (report_bug(regs->epc, regs) == BUG_TRAP_TYPE_WARN ||
- handle_cfi_failure(regs) == BUG_TRAP_TYPE_WARN)
- regs->epc += get_break_insn_length(regs->epc);
- else
- die(regs, "Kernel BUG");
+ force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)regs->epc);
}
asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
@@ -308,16 +324,18 @@ asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
irqentry_enter_from_user_mode(regs);
local_irq_enable();
- handle_break(regs);
+ handle_user_break(regs);
local_irq_disable();
irqentry_exit_to_user_mode(regs);
} else {
- irqentry_state_t state = irqentry_nmi_enter(regs);
+ if (handle_kernel_bug(regs))
+ return;
- handle_break(regs);
+ if (handle_kernel_break(regs))
+ return;
- irqentry_nmi_exit(regs, state);
+ handle_kernel_die(regs);
}
}
^ permalink raw reply related
* Re: [PATCH] riscv: entry: Fixup do_trap_break from kernel side
From: Vivian Wang @ 2026-06-22 10:25 UTC (permalink / raw)
To: Peter Zijlstra, Guo Ren
Cc: Kees Cook, arnd, palmer, tglx, luto, conor.dooley, heiko, jszhang,
lazyparser, falcon, chenhuacai, apatel, atishp, mark.rutland,
bjorn, palmer, bjorn, daniel.thompson, linux-arch, linux-kernel,
linux-riscv, stable, Guo Ren
In-Reply-To: <20260622082841.GW49951@noisy.programming.kicks-ass.net>
On 6/22/26 16:28, Peter Zijlstra wrote:
> On Sun, Jun 21, 2026 at 02:52:46AM -0400, Guo Ren wrote:
>> On Fri, Jun 19, 2026 at 04:54:53PM -0700, Kees Cook wrote:
>>> *thread encromancy*
>>>
>>> On Sat, Jul 01, 2023 at 10:57:07PM -0400, guoren@kernel.org wrote:
>>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>>
>>>> The irqentry_nmi_enter/exit would force the current context into in_interrupt.
>>>> That would trigger the kernel to dead panic, but the kdb still needs "ebreak" to
>>>> debug the kernel.
>>>>
>>>> Move irqentry_nmi_enter/exit to exception_enter/exit could correct handle_break
>>>> of the kernel side.
>>>>
>>>> Before the fixup:
>>>> $echo BUG > /sys/kernel/debug/provoke-crash/DIRECT
>>>> lkdtm: Performing direct entry BUG
>>>> ------------[ cut here ]------------
>>>> kernel BUG at drivers/misc/lkdtm/bugs.c:78!
>>>> [...]
>>>> Kernel panic - not syncing: Aiee, killing interrupt handler!
>>> This appears to still be unfixed. What's the blocker? The solutions in
>>> this thread seem to work...
>>>
>>> I'd like to be exercising an Oops path via KUnit (for KCFI), and riscv
>>> just instantly falls over instead of thread-killing on the exception.
>> Thanks for reviving this thread. At the time I didn’t fully understand
>> Peter’s point. We should only use the NMI path when the trap occurs with
>> interrupts disabled.
>> Here’s the updated fix:
>>
>> do_trap_break(struct pt_regs *regs)
>> ...
>> irqentry_exit_to_user_mode(regs);
>> } else {
>> - irqentry_state_t state = irqentry_nmi_enter(regs);
>> + if (regs->status & SR_IE) {
>> + enum ctx_state prev_state = exception_enter();
>>
>> - handle_break(regs);
>> + handle_break(regs);
>>
>> - irqentry_nmi_exit(regs, state);
>> + exception_exit(prev_state);
>> + } else {
>> + irqentry_state_t state = irqentry_nmi_enter(regs);
>> +
>> + handle_break(regs);
>> +
>> + irqentry_nmi_exit(regs, state);
>> + }
>> }
>> }
>>
>> If you & Peter have no objection, I’ll post a v2.
> I still don't understand it. This cannot fix anything. Consider:
>
> EBREAK
> raw_spin_lock_irq(&your_lock)
> EBREAK
>
> So now the first 'works', but the second will crash. Additionally,
> having the EBREAK context differ so dramatically between invocations
> seems like a very bad deal to me.
To spell it out, the problem that needs fixing is:
-> BUG()
-> ebreak instruction
-> Breakpoint exception
-> do_trap_break()
-> irqentry_nmi_enter()
[ now in_nmi() / in_interrupt() ]
-> report_bug() returns BUG_TRAP_TYPE_BUG
-> die()
-> make_task_dead()
-> panic() because we're in_interrupt()
As such, currently on riscv all BUG() simply completely panic() the
entire machine, rather than just killing the one task.
How do you think this should be fixed? Here are some ideas but I'm not
familiar with generic entry stuff:
* Should we irqentry_nmi_exit() before calling die() for BUG()?
* Should we move the GENERIC_BUG trap instruction to cause illegal
instruction exception instead, for which we can write a simpler
handler that doesn't need to care about the probe stuff?
Vivian "dramforever" Wang
^ permalink raw reply
* Re: [PATCH] riscv: entry: Fixup do_trap_break from kernel side
From: Peter Zijlstra @ 2026-06-22 8:28 UTC (permalink / raw)
To: Guo Ren
Cc: Kees Cook, arnd, palmer, tglx, luto, conor.dooley, heiko, jszhang,
lazyparser, falcon, chenhuacai, apatel, atishp, mark.rutland,
bjorn, palmer, bjorn, daniel.thompson, linux-arch, linux-kernel,
linux-riscv, stable, Guo Ren
In-Reply-To: <ajeKPpg2rwadVPY4@gmail.com>
On Sun, Jun 21, 2026 at 02:52:46AM -0400, Guo Ren wrote:
> On Fri, Jun 19, 2026 at 04:54:53PM -0700, Kees Cook wrote:
> > *thread encromancy*
> >
> > On Sat, Jul 01, 2023 at 10:57:07PM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The irqentry_nmi_enter/exit would force the current context into in_interrupt.
> > > That would trigger the kernel to dead panic, but the kdb still needs "ebreak" to
> > > debug the kernel.
> > >
> > > Move irqentry_nmi_enter/exit to exception_enter/exit could correct handle_break
> > > of the kernel side.
> > >
> > > Before the fixup:
> > > $echo BUG > /sys/kernel/debug/provoke-crash/DIRECT
> > > lkdtm: Performing direct entry BUG
> > > ------------[ cut here ]------------
> > > kernel BUG at drivers/misc/lkdtm/bugs.c:78!
> > > [...]
> > > Kernel panic - not syncing: Aiee, killing interrupt handler!
> >
> > This appears to still be unfixed. What's the blocker? The solutions in
> > this thread seem to work...
> >
> > I'd like to be exercising an Oops path via KUnit (for KCFI), and riscv
> > just instantly falls over instead of thread-killing on the exception.
> Thanks for reviving this thread. At the time I didn’t fully understand
> Peter’s point. We should only use the NMI path when the trap occurs with
> interrupts disabled.
> Here’s the updated fix:
>
> do_trap_break(struct pt_regs *regs)
> ...
> irqentry_exit_to_user_mode(regs);
> } else {
> - irqentry_state_t state = irqentry_nmi_enter(regs);
> + if (regs->status & SR_IE) {
> + enum ctx_state prev_state = exception_enter();
>
> - handle_break(regs);
> + handle_break(regs);
>
> - irqentry_nmi_exit(regs, state);
> + exception_exit(prev_state);
> + } else {
> + irqentry_state_t state = irqentry_nmi_enter(regs);
> +
> + handle_break(regs);
> +
> + irqentry_nmi_exit(regs, state);
> + }
> }
> }
>
> If you & Peter have no objection, I’ll post a v2.
I still don't understand it. This cannot fix anything. Consider:
EBREAK
raw_spin_lock_irq(&your_lock)
EBREAK
So now the first 'works', but the second will crash. Additionally,
having the EBREAK context differ so dramatically between invocations
seems like a very bad deal to me.
^ permalink raw reply
* [PATCH] uapi/asm-generic/fcntl.h: fix typos in Open File Description Locks comment
From: Sara Sena @ 2026-06-21 21:55 UTC (permalink / raw)
To: arnd; +Cc: linux-arch, linux-kernel, Sara Sena
Signed-off-by: Sara Sena <sarasena.adr@gmail.com>
---
include/uapi/asm-generic/fcntl.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/uapi/asm-generic/fcntl.h b/include/uapi/asm-generic/fcntl.h
index 613475285643..06433ed5e3eb 100644
--- a/include/uapi/asm-generic/fcntl.h
+++ b/include/uapi/asm-generic/fcntl.h
@@ -138,10 +138,10 @@
* not inherited across a fork().
*
* These cmd values will set locks that conflict with process-associated
- * record locks, but are "owned" by the open file description, not the
+ * record locks, but are "owned" by the open file description, not the
* process. This means that they are inherited across fork() like BSD (flock)
* locks, and they are only released automatically when the last reference to
- * the the open file against which they were acquired is put.
+ * the open file against which they were acquired is put.
*/
#define F_OFD_GETLK 36
#define F_OFD_SETLK 37
--
2.43.0
^ permalink raw reply related
* Re: [PATCH] kprobes: Replace __ASSEMBLY__ with __ASSEMBLER__ in header file
From: Masami Hiramatsu @ 2026-06-21 13:27 UTC (permalink / raw)
To: Thomas Huth
Cc: Naveen N Rao, David S. Miller, linux-kernel, linux-trace-kernel,
linux-arch
In-Reply-To: <20260619161434.88270-1-thuth@redhat.com>
On Fri, 19 Jun 2026 18:14:34 +0200
Thomas Huth <thuth@redhat.com> wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> While the GCC and Clang compilers already define __ASSEMBLER__
> automatically when compiling assembly code, __ASSEMBLY__ is a
> macro that only gets defined by the Makefiles in the kernel.
> This can be very confusing when switching between userspace
> and kernelspace coding, or when dealing with uapi headers that
> rather should use __ASSEMBLER__ instead. So let's standardize now
> on the __ASSEMBLER__ macro that is provided by the compilers.
>
Oops, good catch!
Let me pick it.
Thanks!
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> Note: This patch has been split from an earlier bigger patch of
> mine to ease reviewing.
>
> include/asm-generic/kprobes.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/asm-generic/kprobes.h b/include/asm-generic/kprobes.h
> index 5290a2b2e15a0..16f16963d503d 100644
> --- a/include/asm-generic/kprobes.h
> +++ b/include/asm-generic/kprobes.h
> @@ -2,7 +2,7 @@
> #ifndef _ASM_GENERIC_KPROBES_H
> #define _ASM_GENERIC_KPROBES_H
>
> -#if defined(__KERNEL__) && !defined(__ASSEMBLY__)
> +#if defined(__KERNEL__) && !defined(__ASSEMBLER__)
> #ifdef CONFIG_KPROBES
> /*
> * Blacklist ganerating macro. Specify functions which is not probed
> @@ -21,6 +21,6 @@ static unsigned long __used \
> # define __kprobes
> # define nokprobe_inline inline
> #endif
> -#endif /* defined(__KERNEL__) && !defined(__ASSEMBLY__) */
> +#endif /* defined(__KERNEL__) && !defined(__ASSEMBLER__) */
>
> #endif /* _ASM_GENERIC_KPROBES_H */
> --
> 2.54.0
>
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox