From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1795030203; b=gaCfOGAbkixYMg+WuowQD9G/OnXcSIx32Adhr0tXKuupvj5TBpQ5sg1AWabIuWhYb3xS euovYLlD8Pnq6q64aWn1RAb9n4PrUwCFx2roVPXhGX6giUlhxDxpHS5+zCEjqI0wsrg6Q Y85WrQ3iaqxE5MQhngnBQVYnhBPfTjGDPfUydD8viSv0KxVL+0U99FOapwC2xdHLfdBRk PfxZ6KNYpXvra46gWFa0w6dPINg7QrVIyfjcnim/oQQSXO1A08+HiQXLIU5raMdNY+QM3 H+oTQwbG/vIlA1RO4keBgUf+soOVpBguEiP1v8AekksnBU2hcBM8zp4pgRK02eCxHcg== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1795030203; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:User-Agent:X-stable:X-Patchwork-Hint: MIME-Version:Content-Type:Content-Transfer-Encoding; bh=NpZ6DBZ7iAHhKbs2em8WwD0cD7m6SofQXgM6Eid0bWY=; b=DiFRWYTFS2RTLheHXjvTiS3RKn/oglztAYM3fUxGBummv8+Xydu7ZLtSFgLJjZtOxkJU DSvXe6pj0ju3bdtO/40zmPYQenZw5ZTIbQJ4slTDYeXOC+VxvE6HmjcJfvdO2x1lunZK0 88W2yoiZaQl+6ODhjq9mnRqsG6+9conwXbGy2n6dZxp/48B4/FQHpjhLhEbg0V/f3r47w jCceSVR1iKgb++GnYW6ti10IahAd5RIsP+XmpEt9F57X8aHPXfBQihn8NiIe2lVx6vUjF /5GoHxgAzlpGbnQ2CZMnkgJSj3fCQ4mgmce+Bak/YCRFXYIS/Fy/6vJoVeORtaXAqSg== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=RbWqC9fm; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 187E8606D36A; Wed, 18 Nov 2026 19:30:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=RbWqC9fm Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BD491F000C4; Wed, 18 Nov 2026 19:30:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1795030202; bh=NpZ6DBZ7iAHhKbs2em8WwD0cD7m6SofQXgM6Eid0bWY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=RbWqC9fm1miCz4OjbGtuY7k33pUTpEBs1ITUEitOWZkjz1Rtk10wD5heJuPgNiimL IYmoDneYFJJ6+oFJa98h2yrPMMREgLvGbYkf+MkHDEIaNL/xhJ1xxdO4lYTaChksU4 AV3CkAvYxRwd8WwypLxEz655rcI5ObVK8ZuMtC0k= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Guanghui Feng , Lu Baolu , Joerg Roedel , Sasha Levin Subject: [PATCH 6.12 0621/1756] iommu/vt-d: Fix IQE handling to cover all descriptors in submission range Date: Wed, 18 Nov 2026 15:45:48 +0100 Message-ID: <20261118150033.627665870@linuxfoundation.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261118150007.957976754@linuxfoundation.org> References: <20261118150007.957976754@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore X-Mailing-List: patches@lists.linux.dev List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 6.12-stable review patch. If anyone has any objections, please let me know. ------------------ From: Guanghui Feng [ Upstream commit 754d07b940c4ce025fab022b548fb5c3d11bf766 ] When an Invalidation Queue Error (IQE) occurs, hardware halts fetching and IQH points at the faulting descriptor. The previous code only checked whether IQH matched the first descriptor index of the current submission, missing faults on any other descriptor within the batch. Expand the check to cover the entire submission range [index, wait_index], accounting for circular wrap-around. Furthermore, after detecting IQE, the old recovery only replaced the single faulting slot with a copy of the wait descriptor and immediately returned -EINVAL. This left two problems: a) Hardware resumed fetching and could hit another invalid descriptor in the same abandoned batch, raising a second IQE that no submitter would claim — permanently deadlocking the queue. b) The caller reclaimed all batch slots (QI_FREE) while hardware might still be asynchronously processing descriptors from that batch, allowing concurrent overwrite and descriptor corruption. Fix both by introducing qi_drain_remaining_descs(): upon IQE detection, overwrite the stranded slots in [IQH, wait_index) with fenced no-op wait descriptors, resubmit the wait descriptor at wait_index, clear IQE, and spin until hardware signals QI_DONE (with DMAR_OPERATION_TIMEOUT). This guarantees hardware has fully drained the batch before the caller reclaims slots. Fixes: 8a1d82462540 ("iommu/vt-d: Multiple descriptors per qi_submit_sync()") Signed-off-by: Guanghui Feng Signed-off-by: Lu Baolu Signed-off-by: Joerg Roedel Signed-off-by: Sasha Levin --- drivers/iommu/intel/dmar.c | 72 ++++++++++++++++++++++++++++++++------ 1 file changed, 61 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 5598cd77c5378..cd169eaa65552 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1278,6 +1278,49 @@ static void qi_dump_fault(struct intel_iommu *iommu, u32 fault) (unsigned long long)desc->qw1); } +static int qi_drain_remaining_descs(struct intel_iommu *iommu, int head, + int wait_index) +{ + struct q_inval *qi = iommu->qi; + int shift = qi_shift(iommu); + struct qi_desc wait_desc = {}; + struct qi_desc nop_desc = {}; + cycles_t start; + + nop_desc.qw0 = QI_IWD_FENCE | QI_IWD_TYPE; + + wait_desc.qw0 = QI_IWD_STATUS_DATA(QI_DONE) | QI_IWD_PRQ_DRAIN | + QI_IWD_STATUS_WRITE | QI_IWD_FENCE | QI_IWD_TYPE; + wait_desc.qw1 = virt_to_phys(&qi->desc_status[wait_index]); + + while (head != wait_index) { + memcpy(qi->desc + (head << shift), &nop_desc, 1 << shift); + head = (head + 1) % QI_LENGTH; + } + + WRITE_ONCE(qi->desc_status[wait_index], QI_IN_USE); + memcpy(qi->desc + (wait_index << shift), &wait_desc, 1 << shift); + + /* + * Order the descriptor rewrites before the writel() that restarts + * the fetch engine. + */ + wmb(); + + writel(DMA_FSTS_IQE, iommu->reg + DMAR_FSTS_REG); + + start = get_cycles(); + while (READ_ONCE(qi->desc_status[wait_index]) != QI_DONE) { + if (DMAR_OPERATION_TIMEOUT < (get_cycles() - start)) { + pr_err("Timeout draining invalidation queue after IQE\n"); + return -ETIMEDOUT; + } + cpu_relax(); + } + + return 0; +} + static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) { u32 fault; @@ -1300,18 +1343,25 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) * is cleared. */ if (fault & DMA_FSTS_IQE) { + int head_idx, ret; + head = readl(iommu->reg + DMAR_IQH_REG); - if ((head >> shift) == index) { - struct qi_desc *desc = qi->desc + head; - - /* - * desc->qw2 and desc->qw3 are either reserved or - * used by software as private data. We won't print - * out these two qw's for security consideration. - */ - memcpy(desc, qi->desc + (wait_index << shift), - 1 << shift); - writel(DMA_FSTS_IQE, iommu->reg + DMAR_FSTS_REG); + head_idx = (head >> shift) % QI_LENGTH; + + /* + * The faulting descriptor can be anywhere within the current + * submission's range [index, wait_index]. Since the queue is + * circular, this submission may wrap around QI_LENGTH + * (index > wait_index in that case), so check both the + * non-wrapped and wrapped cases of the range. + */ + if (index <= wait_index ? + (head_idx >= index && head_idx <= wait_index) : + (head_idx >= index || head_idx <= wait_index)) { + ret = qi_drain_remaining_descs(iommu, head_idx, wait_index); + if (ret) + return ret; + pr_info("Invalidation Queue Error (IQE) cleared\n"); return -EINVAL; } -- 2.53.0