From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rob Clark Subject: [PATCH] iommu: arm-smmu: Set SCTLR.HUPCF bit Date: Thu, 27 Sep 2018 18:46:07 -0400 Message-ID: <20180927224609.19515-1-robdclark@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: freedreno-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Freedreno" To: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Will Deacon , Robin Murphy , linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Rob Clark , freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, Joerg Roedel , linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: linux-arm-msm@vger.kernel.org V2Ugc2VlbSB0byBuZWVkIHRvIHNldCBlaXRoZXIgdGhpcyBvciBDRkNGRyAoc3RhbGwpLCBvdGhl cndpc2UgZ3B1CmZhdWx0cyB0cmlnZ2VyIHByb2JsZW1zIHdpdGggb3RoZXIgaW4tZmxpZ2h0IHRy YW5zYWN0aW9ucyBmcm9tIHRoZQpHUFUgY2F1c2luZyBDUCBlcnJvcnMsIGV0Yy4KCkluIHRoZSBB Uk0gU01NVSBzcGVjLCB0aGUgJ0hpdCB1bmRlciBwcmV2aW91cyBjb250ZXh0IGZhdWx0JyBiaXQg aXMKZGVzY3JpYmVkIGFzOgoKICcwJyAtIFN0YWxsIG9yIHRlcm1pbmF0ZSBzdWJzZXF1ZW50IHRy YW5zYWN0aW9ucyBpbiB0aGUgcHJlc2VuY2UKICAgICAgIG9mIGFuIG91dHN0YW5kaW5nIGNvbnRl eHQgZmF1bHQKICcxJyAtIFByb2Nlc3MgYWxsIHN1YnNlcXVlbnQgdHJhbnNhY3Rpb25zIGluZGVw ZW5kZW50bHkgb2YgYW55CiAgICAgICBvdXRzdGFuZGluZyBjb250ZXh0IGZhdWx0LgoKU2luY2Ug d2UgZG9uJ3QgZW5hYmxlIENGQ0ZHIChzdGFsbCkgdGhlIGJlaGF2aW9yIG9mIHRlcm1pbmF0aW5n Cm90aGVyIHRyYW5zYWN0aW9ucyBtYWtlcyBzZW5zZS4gIEFuZCBpcyBwcm9iYWJseSBub3Qgd2hh dCB3ZSB3YW50CihhbmQgZGVmaW5hdGVseSBub3Qgd2hhdCB3ZSB3YW50IGZvciBHUFUpLgoKU2ln bmVkLW9mZi1ieTogUm9iIENsYXJrIDxyb2JkY2xhcmtAZ21haWwuY29tPgotLS0KU28gSSBoaXQg dGhpcyBpc3N1ZSBhIGxvbmcgdGltZSBiYWNrIG9uIDgyMCAobXNtODk5NikgYW5kIGF0IHRoZQp0 aW1lIEkgc29sdmVkIGl0IHdpdGggYSBwYXRjaCB0aGF0IGVuYWJsZWQgQ0ZDRkcuICBBbmQgaXQg cmVzdXJmYWNlZAptb3JlIHJlY2VudGx5IG9uIHNkbTg0NS4gIEJ1dCBhdCB0aGUgdGltZSBDRkNG RyB3YXMgcmVqZWN0ZWQsIGlpcmMKYmVjYXVzZSBvZiBjb25jZXJuIHRoYXQgaXQgd291bGQgY2F1 c2UgcHJvYmxlbXMgb24gb3RoZXIgbm9uLXFjb20KYXJtIHNtbXUgaW1wbGVtZW50YXRpb25zLiAg QW5kIEkgdGhpbmsgSSBmb3Jnb3QgdG8gc2VuZCB0aGlzIHZlcnNpb24Kb2YgdGhlIHNvbHV0aW9u LgoKSWYgZW5hYmxpbmcgSFVQQ0YgaXMgYW50aWNpcGF0ZWQgdG8gY2F1c2UgcHJvYmxlbXMgb24g b3RoZXIgQVJNClNNTVUgaW1wbGVtZW50YXRpb25zLCBJIHRoaW5rIEkgY2FuIGNvbWUgdXAgd2l0 aCBhIHZhcmlhbnQgb2YgdGhpcwpwYXRjaCB3aGljaCBjb25kaXRpb25hbGx5IGVuYWJsZXMgaXQg Zm9yIHNuYXBkcmFnb24uCgpFaXRoZXIgd2F5LCBJJ2QgcmVhbGx5IGxpa2UgdG8gZ2V0IHNvbWUg dmFyaWFudCBvZiB0aGlzIGZpeCBtZXJnZWQKKGFuZCBwcm9iYWJseSBpdCB3b3VsZCBiZSBhIGdv b2QgaWRlYSBmb3Igc3RhYmxlIGtlcm5lbCBicmFuY2hlcwp0b28pLCBzaW5jZSBjdXJyZW50IGJl aGF2aW91ciB3aXRoIHRoZSBHUFUgbWVhbnMgZmF1bHRzIHR1cm4gaW50bwphIGZhbnRhc3RpYyBj YXNjYWRlIG9mIGZhaWwuCgogZHJpdmVycy9pb21tdS9hcm0tc21tdS1yZWdzLmggfCAxICsKIGRy aXZlcnMvaW9tbXUvYXJtLXNtbXUuYyAgICAgIHwgMiArLQogMiBmaWxlcyBjaGFuZ2VkLCAyIGlu c2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKCmRpZmYgLS1naXQgYS9kcml2ZXJzL2lvbW11L2Fy bS1zbW11LXJlZ3MuaCBiL2RyaXZlcnMvaW9tbXUvYXJtLXNtbXUtcmVncy5oCmluZGV4IGExMjI2 ZTRhYjVmOC4uMjI5MTkyNWViODAwIDEwMDY0NAotLS0gYS9kcml2ZXJzL2lvbW11L2FybS1zbW11 LXJlZ3MuaAorKysgYi9kcml2ZXJzL2lvbW11L2FybS1zbW11LXJlZ3MuaApAQCAtMTc4LDYgKzE3 OCw3IEBAIGVudW0gYXJtX3NtbXVfczJjcl9wcml2Y2ZnIHsKICNkZWZpbmUgQVJNX1NNTVVfQ0Jf QVRTUgkJMHg4ZjAKIAogI2RlZmluZSBTQ1RMUl9TMV9BU0lEUE5FCQkoMSA8PCAxMikKKyNkZWZp bmUgU0NUTFJfSFVQQ0YJCQkoMSA8PCA4KQogI2RlZmluZSBTQ1RMUl9DRkNGRwkJCSgxIDw8IDcp CiAjZGVmaW5lIFNDVExSX0NGSUUJCQkoMSA8PCA2KQogI2RlZmluZSBTQ1RMUl9DRlJFCQkJKDEg PDwgNSkKZGlmZiAtLWdpdCBhL2RyaXZlcnMvaW9tbXUvYXJtLXNtbXUuYyBiL2RyaXZlcnMvaW9t bXUvYXJtLXNtbXUuYwppbmRleCBmN2E5NmJjZjk0YTYuLjQ3ZmZjOWFhZGU3MiAxMDA2NDQKLS0t IGEvZHJpdmVycy9pb21tdS9hcm0tc21tdS5jCisrKyBiL2RyaXZlcnMvaW9tbXUvYXJtLXNtbXUu YwpAQCAtNzEzLDkgKzcxMyw5IEBAIHN0YXRpYyB2b2lkIGFybV9zbW11X3dyaXRlX2NvbnRleHRf YmFuayhzdHJ1Y3QgYXJtX3NtbXVfZGV2aWNlICpzbW11LCBpbnQgaWR4KQogCXJlZyA9IFNDVExS X0NGSUUgfCBTQ1RMUl9DRlJFIHwgU0NUTFJfQUZFIHwgU0NUTFJfVFJFIHwgU0NUTFJfTTsKIAlp ZiAoc3RhZ2UxKQogCQlyZWcgfD0gU0NUTFJfUzFfQVNJRFBORTsKKwlyZWcgfD0gU0NUTFJfSFVQ Q0Y7CiAJaWYgKElTX0VOQUJMRUQoQ09ORklHX0NQVV9CSUdfRU5ESUFOKSkKIAkJcmVnIHw9IFND VExSX0U7Ci0KIAl3cml0ZWxfcmVsYXhlZChyZWcsIGNiX2Jhc2UgKyBBUk1fU01NVV9DQl9TQ1RM Uik7CiB9CiAKLS0gCjIuMTcuMQoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KRnJlZWRyZW5vIG1haWxpbmcgbGlzdApGcmVlZHJlbm9AbGlzdHMuZnJlZWRl c2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8v ZnJlZWRyZW5vCg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: robdclark@gmail.com (Rob Clark) Date: Thu, 27 Sep 2018 18:46:07 -0400 Subject: [PATCH] iommu: arm-smmu: Set SCTLR.HUPCF bit Message-ID: <20180927224609.19515-1-robdclark@gmail.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org We seem to need to set either this or CFCFG (stall), otherwise gpu faults trigger problems with other in-flight transactions from the GPU causing CP errors, etc. In the ARM SMMU spec, the 'Hit under previous context fault' bit is described as: '0' - Stall or terminate subsequent transactions in the presence of an outstanding context fault '1' - Process all subsequent transactions independently of any outstanding context fault. Since we don't enable CFCFG (stall) the behavior of terminating other transactions makes sense. And is probably not what we want (and definately not what we want for GPU). Signed-off-by: Rob Clark --- So I hit this issue a long time back on 820 (msm8996) and at the time I solved it with a patch that enabled CFCFG. And it resurfaced more recently on sdm845. But at the time CFCFG was rejected, iirc because of concern that it would cause problems on other non-qcom arm smmu implementations. And I think I forgot to send this version of the solution. If enabling HUPCF is anticipated to cause problems on other ARM SMMU implementations, I think I can come up with a variant of this patch which conditionally enables it for snapdragon. Either way, I'd really like to get some variant of this fix merged (and probably it would be a good idea for stable kernel branches too), since current behaviour with the GPU means faults turn into a fantastic cascade of fail. drivers/iommu/arm-smmu-regs.h | 1 + drivers/iommu/arm-smmu.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-regs.h b/drivers/iommu/arm-smmu-regs.h index a1226e4ab5f8..2291925eb800 100644 --- a/drivers/iommu/arm-smmu-regs.h +++ b/drivers/iommu/arm-smmu-regs.h @@ -178,6 +178,7 @@ enum arm_smmu_s2cr_privcfg { #define ARM_SMMU_CB_ATSR 0x8f0 #define SCTLR_S1_ASIDPNE (1 << 12) +#define SCTLR_HUPCF (1 << 8) #define SCTLR_CFCFG (1 << 7) #define SCTLR_CFIE (1 << 6) #define SCTLR_CFRE (1 << 5) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index f7a96bcf94a6..47ffc9aade72 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -713,9 +713,9 @@ static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx) reg = SCTLR_CFIE | SCTLR_CFRE | SCTLR_AFE | SCTLR_TRE | SCTLR_M; if (stage1) reg |= SCTLR_S1_ASIDPNE; + reg |= SCTLR_HUPCF; if (IS_ENABLED(CONFIG_CPU_BIG_ENDIAN)) reg |= SCTLR_E; - writel_relaxed(reg, cb_base + ARM_SMMU_CB_SCTLR); } -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20848C43382 for ; Thu, 27 Sep 2018 22:46:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 83C43215F0 for ; Thu, 27 Sep 2018 22:46:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b9H9wEav" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 83C43215F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727701AbeI1FG6 (ORCPT ); Fri, 28 Sep 2018 01:06:58 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:41559 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725924AbeI1FG6 (ORCPT ); Fri, 28 Sep 2018 01:06:58 -0400 Received: by mail-qk1-f196.google.com with SMTP id n3-v6so2693438qkn.8; Thu, 27 Sep 2018 15:46:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=68dCE8AAJ7ks0wme6FIZwJzv+K1DwC/CeEghUvOZQm8=; b=b9H9wEavPlqBF88iN6gKF6vpE9NzMBkBVkMrRqBhAMJUfXnruqq8Q6/gw6N5EUidIk dn/EpxcMSoY52m6DnOn2WxCw374WC4D83AND322yCFFqdbuFqemF2/dYtAEiXr7+bESg D/x+KXaFqaA+6K8ljyu57DJR7cME0n2fdZ7kzy6wxe0JPzv0+q6sE4RDep+KGQdZb6kL T93l6nqv8F6XokwVtJDNV1qlFpsJCBSe2eZhCJFNQfNy7YZ+ATvzr0w/6BPqpbsm7VlG 4gFXpufkhpnsL1nQ0JLE3nJQUlF6gBNt/6sEw5uXZE2dcHEWkoykY3V26KoqddLvtaQT z9mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=68dCE8AAJ7ks0wme6FIZwJzv+K1DwC/CeEghUvOZQm8=; b=IvP+uFr6aX4aQpyJvwRJNvi7Hmg2hGXTZ9LEHhBPE9cqX9J4s1Nfej+Z0txY/hnjRd IoRlAp2lrLpzvoEOxbymujnLcLty0FSxNG/vrgMMO/o0pcpL81JlvFHs6YCMAmcGVFWK m7WgBhkfREi7bJUymfZSqGe5is133ORz0HdvQ0OJzQ9QAQfnFSlVYWFX11RrOhZzJKKQ WdWyE/2hZjqkUccUA7rNZs5O3rvAJUWZEJsV5GKsgPIKdSFnd/+3WCLV7lXohIShbT7Q z4Xc1Df/16DxD0UhGhLD/MnN8jndoFCtAIwSXGL0Wvdq4QspiKDASZhzbkDjp2JaL7gH ih2g== X-Gm-Message-State: ABuFfoiXR0SeH9Jh6lp9mNlyV5w92E2+5mnje43w1irXpKCPFlsqLAl1 UgvDlAgnEJV9QiTuNy/c2yk= X-Google-Smtp-Source: ACcGV60uKXiUiI+N+LKuITTXmlWXjpbfGwyrmtNNbScHDTZrVF1Vqp/cHy3qLLrKpNye0YV9/iKmBA== X-Received: by 2002:a37:e20a:: with SMTP id g10-v6mr9910556qki.45.1538088381308; Thu, 27 Sep 2018 15:46:21 -0700 (PDT) Received: from localhost ([2601:184:4780:7861:6268:7a0b:50be:cebc]) by smtp.gmail.com with ESMTPSA id 23-v6sm1969465qkn.11.2018.09.27.15.46.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 27 Sep 2018 15:46:20 -0700 (PDT) From: Rob Clark To: iommu@lists.linux-foundation.org, Will Deacon , Robin Murphy , linux-arm-kernel@lists.infradead.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Joerg Roedel , linux-kernel@vger.kernel.org Subject: [PATCH] iommu: arm-smmu: Set SCTLR.HUPCF bit Date: Thu, 27 Sep 2018 18:46:07 -0400 Message-Id: <20180927224609.19515-1-robdclark@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We seem to need to set either this or CFCFG (stall), otherwise gpu faults trigger problems with other in-flight transactions from the GPU causing CP errors, etc. In the ARM SMMU spec, the 'Hit under previous context fault' bit is described as: '0' - Stall or terminate subsequent transactions in the presence of an outstanding context fault '1' - Process all subsequent transactions independently of any outstanding context fault. Since we don't enable CFCFG (stall) the behavior of terminating other transactions makes sense. And is probably not what we want (and definately not what we want for GPU). Signed-off-by: Rob Clark --- So I hit this issue a long time back on 820 (msm8996) and at the time I solved it with a patch that enabled CFCFG. And it resurfaced more recently on sdm845. But at the time CFCFG was rejected, iirc because of concern that it would cause problems on other non-qcom arm smmu implementations. And I think I forgot to send this version of the solution. If enabling HUPCF is anticipated to cause problems on other ARM SMMU implementations, I think I can come up with a variant of this patch which conditionally enables it for snapdragon. Either way, I'd really like to get some variant of this fix merged (and probably it would be a good idea for stable kernel branches too), since current behaviour with the GPU means faults turn into a fantastic cascade of fail. drivers/iommu/arm-smmu-regs.h | 1 + drivers/iommu/arm-smmu.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-regs.h b/drivers/iommu/arm-smmu-regs.h index a1226e4ab5f8..2291925eb800 100644 --- a/drivers/iommu/arm-smmu-regs.h +++ b/drivers/iommu/arm-smmu-regs.h @@ -178,6 +178,7 @@ enum arm_smmu_s2cr_privcfg { #define ARM_SMMU_CB_ATSR 0x8f0 #define SCTLR_S1_ASIDPNE (1 << 12) +#define SCTLR_HUPCF (1 << 8) #define SCTLR_CFCFG (1 << 7) #define SCTLR_CFIE (1 << 6) #define SCTLR_CFRE (1 << 5) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index f7a96bcf94a6..47ffc9aade72 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -713,9 +713,9 @@ static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx) reg = SCTLR_CFIE | SCTLR_CFRE | SCTLR_AFE | SCTLR_TRE | SCTLR_M; if (stage1) reg |= SCTLR_S1_ASIDPNE; + reg |= SCTLR_HUPCF; if (IS_ENABLED(CONFIG_CPU_BIG_ENDIAN)) reg |= SCTLR_E; - writel_relaxed(reg, cb_base + ARM_SMMU_CB_SCTLR); } -- 2.17.1