From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 62089C4829E for ; Sun, 18 Feb 2024 05:02:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=QeeHCRzDnRu/u3XqQSGbKSfj+m+JUpqNsQOUD9Iv3ow=; b=IzLSswXqmvCExI bfLFyNGNO2+S2NjsKtnxBTGRjWSUQ8ONymr3f/SofGpPIO9ldwxflYuxkKBUQ7Y5bCuPg39iw11yp mkhfMYRD4ncAAVGHneGcgp5ENvh0fE1X4YdvsN3V3XDxQ1Y3s7BKDaodSEWspw4wAgAiYcH/BxtGb 11mKxC1V6JdD0bivB9vYAiXMdvQ8+KGGWWlkybOM288f/RdcSThNiP6ohUeF+qdfTbS8dvDPangMe QR41Y+c3bLQaebgg78/WG7LsiviJS3M+GF6JIC/wfl3U++0ii1/CQ0sEnhy0fJJzN7fcZmNcI2aBE wDVVFZIp0AqvVyfDVNAA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rbZK2-000000071ic-2fqF; Sun, 18 Feb 2024 05:02:38 +0000 Received: from mail-oo1-xc29.google.com ([2607:f8b0:4864:20::c29]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rbZJz-000000071hv-2Uta for linux-arm-kernel@lists.infradead.org; Sun, 18 Feb 2024 05:02:37 +0000 Received: by mail-oo1-xc29.google.com with SMTP id 006d021491bc7-5986d902ae6so1911801eaf.3 for ; Sat, 17 Feb 2024 21:02:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708232554; x=1708837354; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=uEhuBzl/XUO5srIc9IiEtG7zueZD6dPpQ8b++F1rfE4=; b=HbtDl6fctK+ikWk6habM0yZGGSdwp4HILcqz7RA61rbS8K6Js6Z6TtNCKJ40gkiD/4 amMj01kGGOerGUT4nAjpYyVQojMuTfsjDboJYk+3KhgrDXpQinvHhHhDhWmpgekW8WhL E/FKR6Zu4Gu5lym1CWFfAXBQXHp7b/hoEduzbAh65wkmLjWIImGN+mJyarwpCmWCO76j +5gdCy64EvKQmV2UOecG+zVaxI0ATVB6CeByq5LdIgipTH8hUT1jfI8zt+sVVXMveF1M FObVSwoOksAYK08lsqKgFxRvVP9Tm2Xyej8yppeYjK8WmVSBrBl3uSClxLe7hNGIpiRX S98g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708232554; x=1708837354; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uEhuBzl/XUO5srIc9IiEtG7zueZD6dPpQ8b++F1rfE4=; b=GaPGxpZaM/VQmyiMjJGyEa4EUHieYBTRRIZ08bbmKTrqeejZ8+BAMdDW1/SbgcObAh /8CVHSYZQ4xDeQ/Bp0P6vL9W3PWzg1jgG372mCMENigw2kIZteQjYYvRjru8RM4P/19k mLrVYiQfXc9uifGnQq26wLvaKXBFRD7ZvputX6fC9gAkgN3IbI6l7lAQXuCvsk5S/o5A 6rrPucpWQHymonCzQ8YeauZzfC7FlRui8pV71x21Z4CcM1PlO1R+9xO0Lz/Qlfm3pmwq C1TMYrmBFs9Ye7Kq2Is+qR9L12aWNgUuVG/rdTB0fQEeMWlTbnDKqJ4Bh3jjxlqk2Z73 JXEQ== X-Forwarded-Encrypted: i=1; AJvYcCXrC2aonMbwH8obliRvLdF7NkjJYtZVGXab5uS/2TD+apB0dE+F92QwupYEw5VKQnN3mNu6XTjMSbpM3ZSQbySIO1qkF2gscoLtjxvXfE8uOa8qM/8= X-Gm-Message-State: AOJu0YweiVdhdv+9xZHurunKuDJZTGitjPBouDj9bJZOE32kEnUl4iAm w51Fj0tYBCI37Tw2iYAB4f194srscg8WqI2WvctvsYdJYLlnZj+yRuzt2Sh7s7jghp+y X-Google-Smtp-Source: AGHT+IFtcp5ctWAyRJeAGhJBhuhoEgIuvQ40gFn8og9Zx9QQmyVo4hFDLZYPPM/JgIF7Q/kwRxHrlw== X-Received: by 2002:a05:6358:f485:b0:17a:e6d8:b19f with SMTP id ku5-20020a056358f48500b0017ae6d8b19fmr9987069rwb.24.1708232553829; Sat, 17 Feb 2024 21:02:33 -0800 (PST) Received: from nilq-virtual-machine.localdomain ([60.24.211.11]) by smtp.gmail.com with ESMTPSA id y5-20020a634945000000b005dc48e56191sm2297762pgk.11.2024.02.17.21.02.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 17 Feb 2024 21:02:33 -0800 (PST) From: "ni.liqiang" To: Will Deacon , Robin Murphy , Joerg Roedel Cc: "ni.liqiang" , "jin . qi" , linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH] drivers/iommu: Ensure that the queue base address is successfully written during SMMU initialization. Date: Sun, 18 Feb 2024 13:02:23 +0800 Message-Id: <20240218050224.33426-1-niliqiang.io@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240217_210235_654439_63143FD4 X-CRM114-Status: GOOD ( 14.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In the system reboot test, I encountered an issue: After the OS started, the base address of CMDQ failed to be written successfully and remained at the default value of 0. Through timing analysis of CMN, it was found that although the write request for the CMDQ base precedes the write request for CMDQEN, the write response for the CMDQ base might be later than that for CMDQEN. Upon reviewing the SMMU Architecture Specification, I found the following explanation: The registers must be initialized in this order: 1. Write SMMU_CMDQ_BASE to set the queue base and size. 2. Write initial values to SMMU_CMDQ_CONS and SMMU_CMDQ_PROD. 3. Enable the queue with an Update of the respective SMMU_CR0.CMDQEN to 1. If there are no memory barriers, how can we ensure this order? Therefore, I believe that adding a memory barrier before enabling CMDQ is necessary to ensure that the base address of CMDQ is correctly written. The base addresses of EVENTQ and PRIQ would also be subject to the same situation. Could you please review if this modification seems reasonable? Thank you. Signed-off-by: ni.liqiang Reviewed-by: jin.qi Tested-by: ni.liqiang --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 0ffb1cf17e0b..ac854c46fdf3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -3324,6 +3324,11 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) writel_relaxed(smmu->cmdq.q.llq.prod, smmu->base + ARM_SMMU_CMDQ_PROD); writel_relaxed(smmu->cmdq.q.llq.cons, smmu->base + ARM_SMMU_CMDQ_CONS); + /* Ensure that SMMU_CMDQ_BASE is written completely + * when SMMU_CR0.CMDQEN == 0. + */ + __iomb(); + enables = CR0_CMDQEN; ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, ARM_SMMU_CR0ACK); @@ -3350,6 +3355,11 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) writel_relaxed(smmu->evtq.q.llq.prod, smmu->page1 + ARM_SMMU_EVTQ_PROD); writel_relaxed(smmu->evtq.q.llq.cons, smmu->page1 + ARM_SMMU_EVTQ_CONS); + /* Ensure that SMMU_EVENTQ_BASE is written completely + * when SMMU_CR0.EVENTQEN == 0. + */ + __iomb(); + enables |= CR0_EVTQEN; ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, ARM_SMMU_CR0ACK); @@ -3367,6 +3377,11 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) writel_relaxed(smmu->priq.q.llq.cons, smmu->page1 + ARM_SMMU_PRIQ_CONS); + /* Ensure that SMMU_PRIQ_BASE is written completely + * when SMMU_CR0.PRIQEN == 0. + */ + __iomb(); + enables |= CR0_PRIQEN; ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, ARM_SMMU_CR0ACK); -- 2.34.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel