From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24F9E3BB4A for ; Mon, 26 Jan 2026 15:12:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769440344; cv=none; b=UCnzl6erjmq8rHsVGQ1IG7TCjPpgdHc/ECInimH2vLFWTZQ0hqeaNO8tdp8lryFIeRTYS9k9+vHwQn8Qn10QAvSY2f7r3ffOT60MjIq+uPIqmLAyhwXIIkbK/LonGvQg7SHU6/9J0bV21nHvKObo5t3VuKeM0cL72qf1mHYy+8w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769440344; c=relaxed/simple; bh=QLEaEYFtzeNWlQABjCHCyG2y7/lGZpbeFYWClSdErQc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Lx4/ruf8ZnldCsfiICWHjmorhk4K3Xu8H56D8S8C4LPtL7+w7VyrnLgDWuvx4h9lAwQXwZQ6oVH/4YbuNxT5pPPC/ES4TsRD4qw20hlpFFD+kWlhBkkgAHkruBKdJQ0JbnbBlmVVrQg1dXgCehoRl1h5CJWWtEiLPFgb++WdP7I= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--praan.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QlEKEuvB; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--praan.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QlEKEuvB" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-c629a3276e9so488923a12.2 for ; Mon, 26 Jan 2026 07:12:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1769440342; x=1770045142; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yDrpBbgciiksQRxWRM6hJ/q0CrNBDeXNrW9xr5yyedI=; b=QlEKEuvBnh41fLeGOYiF3EizU08kiprEL6j3xXmGPQg+Frp8mcPDZbcCaEaSlxLiR2 pNEsZt5XM158Q6tpyoPhLQ4t0XBX6fVfh1e1q/tnqTH538tc80hI/EmSmSzMYr84zOXD w4tuamHjNJj54zZPFia+44Nnw9CL/v2GuhclCqHTnSooDAEcsT4PccRHPtD+B2VdBc3w fyvvR9m5Sjvbkw4X4ZwoFPb83dAAPJKpHmNkTlfu4eMdOyj0pujqokrylwgPQX8L6cNI 7FdONhZflBYIs9E/Ub57wwPHHKwvy7t6k+n2ntcKa+GxRkkwgqiCLRU1tbmgfdhSXy+6 vPZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769440342; x=1770045142; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yDrpBbgciiksQRxWRM6hJ/q0CrNBDeXNrW9xr5yyedI=; b=IRtK6mcXvqDPfxClnydP6VaIC/FHVjQxvoQQ98ZUr+nSLSOtNiG9JobJDczpEHw90E zEGcP8uQzMSlAyxtda56kJcHu8ndfwG09KdHKxAponreDYVSXOOElki8pvTpv7j9T9JA XMQ7rpc8h0tZGUwtHIKqti7g/VicB1Qf3if6uwY+X29wCiiXMD09AmTorPxRyQanKmMH p1/wsut1vdKiFSriENQTpplCSOqLcftkCUva08u+6MoFNLzldn7gifGzYfZPXhbXED9B EJPMVJHqzjryzhQQke9RUPstGFg9WD0J3+kVG1gn2ndJMkDXGpxUg9QxVyAQHMJThCpl ju8Q== X-Gm-Message-State: AOJu0Yyyza1y0D5FpfWAZys+yQ8RFwkqJ4UqZRurkOzZJV3J1FV0R6vv UDO+8u/prhthMxv+l2nVxhMq059VxSlhmIUyhj5NqHPQPmZMC5B7EH1tePEErpLuOnFARsrlZW+ EB1uYmyDeREV9Hin2+pnTKzOkTl2CshBM6VpmN8gOes3cm64JJS3/B7MgHAOhywaDCLFJNoOL3t LLPLNz8APcUGhikkylvBthsA9re8rGrA== X-Received: from pga6.prod.google.com ([2002:a05:6a02:4f86:b0:bc0:d9a9:8a8a]) (user=praan job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:7d0b:b0:350:9b6b:8ea8 with SMTP id adf61e73a8af0-38e9f1d1f05mr4804315637.51.1769440342265; Mon, 26 Jan 2026 07:12:22 -0800 (PST) Date: Mon, 26 Jan 2026 15:11:57 +0000 In-Reply-To: <20260126151157.3418145-1-praan@google.com> Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260126151157.3418145-1-praan@google.com> X-Mailer: git-send-email 2.52.0.457.g6b5491de43-goog Message-ID: <20260126151157.3418145-11-praan@google.com> Subject: [PATCH v5 10/10] iommu/arm-smmu-v3: Invoke pm_runtime before hw access From: Pranjal Shrivastava To: iommu@lists.linux.dev Cc: Will Deacon , Joerg Roedel , Robin Murphy , Jason Gunthorpe , Mostafa Saleh , Nicolin Chen , Daniel Mentz , Ashish Mhetre , Sairaj Kodilkar , Pranjal Shrivastava Content-Type: text/plain; charset="UTF-8" Invoke the pm_runtime helpers at all places before accessing the hw. The idea is to invoke runtime_pm helpers at common points which are used by exposed ops or interrupt handlers. Elide all TLB and CFG invalidations if the smmu is suspended but not ATC invalidations. Signed-off-by: Pranjal Shrivastava --- .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c | 10 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 92 +++++++++++++++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 + 3 files changed, 97 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c index 823461a26659..f1f0920ac12c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c @@ -15,6 +15,11 @@ void *arm_smmu_hw_info(struct device *dev, u32 *length, struct iommu_hw_info_arm_smmuv3 *info; u32 __iomem *base_idr; unsigned int i; + int ret; + + ret = arm_smmu_rpm_get(master->smmu); + if (ret < 0) + return ERR_PTR(-EIO); if (*type != IOMMU_HW_INFO_TYPE_DEFAULT && *type != IOMMU_HW_INFO_TYPE_ARM_SMMUV3) { @@ -24,8 +29,10 @@ void *arm_smmu_hw_info(struct device *dev, u32 *length, } info = kzalloc(sizeof(*info), GFP_KERNEL); - if (!info) + if (!info) { + arm_smmu_rpm_put(master->smmu); return ERR_PTR(-ENOMEM); + } base_idr = master->smmu->base + ARM_SMMU_IDR0; for (i = 0; i <= 5; i++) @@ -36,6 +43,7 @@ void *arm_smmu_hw_info(struct device *dev, u32 *length, *length = sizeof(*info); *type = IOMMU_HW_INFO_TYPE_ARM_SMMUV3; + arm_smmu_rpm_put(master->smmu); return info; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 9931dde41239..9e2d16c128b0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -110,7 +110,7 @@ static const char * const event_class_str[] = { static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master); /* Runtime PM helpers */ -__maybe_unused static int arm_smmu_rpm_get(struct arm_smmu_device *smmu) +int arm_smmu_rpm_get(struct arm_smmu_device *smmu) { int ret; @@ -125,7 +125,7 @@ __maybe_unused static int arm_smmu_rpm_get(struct arm_smmu_device *smmu) return 0; } -__maybe_unused static void arm_smmu_rpm_put(struct arm_smmu_device *smmu) +void arm_smmu_rpm_put(struct arm_smmu_device *smmu) { int ret; @@ -1106,7 +1106,9 @@ static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused { struct arm_smmu_cmdq_ent cmd = {0}; struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_device *smmu = master->smmu; int sid = master->streams[0].id; + int ret; if (WARN_ON(!master->stall_enabled)) return; @@ -1126,6 +1128,25 @@ static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused break; } + /* + * The SMMU is guaranteed to be active via device_link if any master is + * active. Furthermore, on suspend we set GBPA to abort, flushing any + * pending stalled transactions. + * + * Receiving a page fault while suspended implies a bug in the power + * dependency chain or a stale event. Since the SMMU is powered down + * and the command queue is inaccessible, we cannot issue the + * RESUME command and must drop it. + */ + if (!atomic_inc_not_zero(&smmu->nr_cmdq_users)) { + dev_err(smmu->dev, "Ignoring page fault while suspended\n"); + return; + } + + ret = arm_smmu_rpm_get(smmu); + if (ret < 0) + goto out; + arm_smmu_cmdq_issue_cmd(master->smmu, &cmd); /* * Don't send a SYNC, it doesn't do anything for RESUME or PRI_RESP. @@ -1133,6 +1154,9 @@ static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused * terminated... at some point in the future. PRI_RESP is fire and * forget. */ + arm_smmu_rpm_put(smmu); +out: + atomic_dec_return_release(&smmu->nr_cmdq_users); } /* Context descriptor manipulation functions */ @@ -2111,6 +2135,7 @@ static void arm_smmu_dump_event(struct arm_smmu_device *smmu, u64 *raw, static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { + int ret; u64 evt[EVTQ_ENT_DWORDS]; struct arm_smmu_event event = {0}; struct arm_smmu_device *smmu = dev; @@ -2119,6 +2144,10 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) static DEFINE_RATELIMIT_STATE(rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); + ret = arm_smmu_rpm_get(smmu); + if (ret < 0) + return IRQ_NONE; + do { while (!queue_remove_raw(q, evt)) { arm_smmu_decode_event(smmu, evt, &event); @@ -2139,6 +2168,7 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) /* Sync our overflow flag, as we believe we're up to speed */ queue_sync_cons_ovf(q); + arm_smmu_rpm_put(smmu); return IRQ_HANDLED; } @@ -2186,6 +2216,11 @@ static irqreturn_t arm_smmu_priq_thread(int irq, void *dev) struct arm_smmu_queue *q = &smmu->priq.q; struct arm_smmu_ll_queue *llq = &q->llq; u64 evt[PRIQ_ENT_DWORDS]; + int ret; + + ret = arm_smmu_rpm_get(smmu); + if (ret < 0) + return IRQ_NONE; do { while (!queue_remove_raw(q, evt)) @@ -2197,6 +2232,7 @@ static irqreturn_t arm_smmu_priq_thread(int irq, void *dev) /* Sync our overflow flag, as we believe we're up to speed */ queue_sync_cons_ovf(q); + arm_smmu_rpm_put(smmu); return IRQ_HANDLED; } @@ -2251,8 +2287,21 @@ static irqreturn_t arm_smmu_handle_gerror(struct arm_smmu_device *smmu) static irqreturn_t arm_smmu_gerror_handler(int irq, void *dev) { struct arm_smmu_device *smmu = dev; + irqreturn_t ret; + + /* If we are suspended, this is a spurious interrupt */ + if (!atomic_read(&smmu->nr_cmdq_users) || + (pm_runtime_enabled(smmu->dev) && + pm_runtime_get_if_active(smmu->dev) <= 0)) { + dev_err(smmu->dev, + "Ignoring gerror interrupt because the SMMU is suspended\n"); + return IRQ_NONE; + } - return arm_smmu_handle_gerror(smmu); + ret = arm_smmu_handle_gerror(smmu); + arm_smmu_rpm_put(smmu); + + return ret; } static irqreturn_t arm_smmu_combined_irq_thread(int irq, void *dev) @@ -2342,26 +2391,33 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, static int arm_smmu_atc_inv_master(struct arm_smmu_master *master, ioasid_t ssid) { - int i; + int i, ret; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; arm_smmu_atc_inv_to_cmd(ssid, 0, 0, &cmd); + /* ATC invalidations shouldn't be elided */ + ret = arm_smmu_rpm_get(master->smmu); + if (ret < 0) + return ret; + arm_smmu_cmdq_batch_init(master->smmu, &cmds, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(master->smmu, &cmds, &cmd); } - return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); + ret = arm_smmu_cmdq_batch_submit(master->smmu, &cmds); + arm_smmu_rpm_put(master->smmu); + return ret; } int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, unsigned long iova, size_t size) { struct arm_smmu_master_domain *master_domain; - int i; + int i, ret; unsigned long flags; struct arm_smmu_cmdq_ent cmd = { .opcode = CMDQ_OP_ATC_INV, @@ -2388,6 +2444,11 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; + /* ATC invalidations shouldn't be elided */ + ret = arm_smmu_rpm_get(smmu_domain->smmu); + if (ret < 0) + return ret; + arm_smmu_cmdq_batch_init(smmu_domain->smmu, &cmds, &cmd); spin_lock_irqsave(&smmu_domain->devices_lock, flags); @@ -2416,7 +2477,9 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); - return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); + ret = arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); + arm_smmu_rpm_put(smmu_domain->smmu); + return ret; } /* IO_PGTABLE API */ @@ -5117,10 +5180,19 @@ static int arm_smmu_device_probe(struct platform_device *pdev) static void arm_smmu_device_remove(struct platform_device *pdev) { struct arm_smmu_device *smmu = platform_get_drvdata(pdev); + int ret; iommu_device_unregister(&smmu->iommu); iommu_device_sysfs_remove(&smmu->iommu); + + ret = arm_smmu_rpm_get(smmu); + if (ret < 0) + goto free_iopf; + arm_smmu_device_disable(smmu); + arm_smmu_rpm_put(smmu); + +free_iopf: iopf_queue_free(smmu->evtq.iopf); ida_destroy(&smmu->vmid_map); } @@ -5128,8 +5200,14 @@ static void arm_smmu_device_remove(struct platform_device *pdev) static void arm_smmu_device_shutdown(struct platform_device *pdev) { struct arm_smmu_device *smmu = platform_get_drvdata(pdev); + int ret; + + ret = arm_smmu_rpm_get(smmu); + if (ret < 0) + return; arm_smmu_device_disable(smmu); + arm_smmu_rpm_put(smmu); } static int __maybe_unused arm_smmu_runtime_suspend(struct device *dev) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 6a752fe06f54..41e8da97193a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -1023,6 +1023,9 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, struct arm_smmu_cmdq *cmdq, u64 *cmds, int n, bool sync); +int arm_smmu_rpm_get(struct arm_smmu_device *smmu); +void arm_smmu_rpm_put(struct arm_smmu_device *smmu); + #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); void arm_smmu_sva_notifier_synchronize(void); -- 2.52.0.457.g6b5491de43-goog