From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C0A0C52D7B for ; Tue, 13 Aug 2024 17:52:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=A5G+oWQ6Rh5ISXgA0J/LF+tOSSrj10GPW8g20/rns1U=; b=2SfFZ34LFsX9/bLLIdRJyWEvVe ZMdE1A4mp2JgJojeyH1i1IDeErzeCyEOPnaZmMB30g655waWfCawPUZx5shaMgiDE6QiUiL4M29Hp msk2ZPSOoB1fNx3aig1+eSYJXYzERXHDpsYXHHew/fwHb1EadRAbt087zpp0yu0b3wiHCmFs5OyJ8 uvTIb98XUnBzKQwS6rPeAUz5wOcStlW4ISfzEhGocdRzw9TZpUjEidvyub129lvzTFnGy5ErD522X chAbfXn5fOyXRPLnb4Nam1M3f13Xujx+YL0fhGTq6HkK84neq51zZvuCxgcuXO/a8fMBVvh5/ajbi KkmUcaYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sdvhH-00000004YBA-1wuV; Tue, 13 Aug 2024 17:52:39 +0000 Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sdvgd-00000004Y1k-33iJ for linux-arm-kernel@lists.infradead.org; Tue, 13 Aug 2024 17:52:01 +0000 Received: by mail-qk1-x730.google.com with SMTP id af79cd13be357-7a1d024f775so384257985a.2 for ; Tue, 13 Aug 2024 10:51:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1723571518; x=1724176318; darn=lists.infradead.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=A5G+oWQ6Rh5ISXgA0J/LF+tOSSrj10GPW8g20/rns1U=; b=Skedn6LCun/J13NUPIpYKpa2ZCZzHEIzm1B8wcg0QH6ha6Ar/CZVAGLPIEgXMXDQ04 PA+IgCeu8AzQpuO6kcdk2MNLAJe5EC6hmeyWIrOTyE3WmzT53K6Dxrv6jWO/RBBVrSa0 5S+DhQ+NCZUo/xX/yE1wX8SmwbVCqKXDm2Dfw5tRsINshGUMl0u1MjS03/MfZlxkLa6B h4qZpr0bJLUH014ADC9gZhiCKSfElTzLPDgrIbp2aZ0sSeOVjtY2ILWlDMEyKnpugasl ncYRu0aBrIsU29wVN0IepnNyhvkOWujqNPOnPEMYH42aK9B1aHkWdOfwPdiD7Gm/RrRz y1FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723571518; x=1724176318; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=A5G+oWQ6Rh5ISXgA0J/LF+tOSSrj10GPW8g20/rns1U=; b=oQg2weMD6JCNee9jhj++gcS7wrw0HAZcsK6w6DSXYsMZTBo4Twg+MLDxNwSRFOze/Z MpHtKBG7Y67akP8m6QJDJKs/q63kvILxUu0eGJyf4j/0RVtWItWd194bPqCRyB1uERYC f+SCibP33E3C59l3MOOQKkZMm+FQdtRzekjLtYjbFt2e/U5Z2v6XcVqYMlxPMxygfH0g arfFat+DcbaQDv2XyezPEASTHBWHQKALe3yE1CS+xD11NDHrRO4wYTo9/n7oRHqRVmKS V0vuUQxYJ8afF7KnbMVQotyZ8znZ/EhJWqJnGVH1fcBOzRcNVmgctmOdtLOLTfE9S+BV fO8Q== X-Forwarded-Encrypted: i=1; AJvYcCX1Tiwqv9kpfvNel3gw5hD6PO67QQ4NCvt5vPknQnBEbM1JTLmpXlrasK8CmHk3IY06MJ/WWAvskftQJfrHWCGQmh60jBK2FM5Yc9+qz35k2XXX/kE= X-Gm-Message-State: AOJu0YztZ8F7Xu4qiGehTVpvwW7Ri+rKKz1d/o+my1EODBRJPG0YXQIz izS9Lr1oXCejCYvAh+fABLRlCVvZUi1VJHSuI9o+dGrCzsPwz15Da6fgJ9Ezw+g= X-Google-Smtp-Source: AGHT+IGNfxxEytwtzg4vfONpxSKV6VhkyKNxntioOTjoDBXC/X1poTHbuNKyYTIkLBbprh5aPMRUaQ== X-Received: by 2002:a05:620a:45a5:b0:7a1:6062:ed63 with SMTP id af79cd13be357-7a4ee33bdc5mr47040685a.29.1723571517942; Tue, 13 Aug 2024 10:51:57 -0700 (PDT) Received: from ziepe.ca ([128.77.69.90]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a4c7dedf8csm357399085a.77.2024.08.13.10.51.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 10:51:57 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1sdvgZ-008nyn-MJ; Tue, 13 Aug 2024 14:51:55 -0300 Date: Tue, 13 Aug 2024 14:51:55 -0300 From: Jason Gunthorpe To: Mostafa Saleh Cc: linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, will@kernel.org, robin.murphy@arm.com, joro@8bytes.org, nicolinc@nvidia.com, mshavit@google.com Subject: Re: [PATCH 2/2] iommu/arm-smmu-v3: Report stalled S2 events Message-ID: <20240813175155.GN1985367@ziepe.ca> References: <20240812205255.97781-1-smostafa@google.com> <20240812205255.97781-3-smostafa@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240812205255.97781-3-smostafa@google.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240813_105159_791757_8798A478 X-CRM114-Status: GOOD ( 30.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Aug 12, 2024 at 08:52:55PM +0000, Mostafa Saleh wrote: > Previously, S2 stall was disabled and in case there was an event it > wouldn't be reported on the assumption that it's always pinned by VFIO. > > However, now since we can enable stall, devices that use S2 outside > VFIO should be able to report the stalls similar to S1. > > Also, to keep the old behaviour were S2 events from nested domains were > not reported as they are pinned (from VFIO) add a new flag to track this. I'm not entirely clear on every detail of this stall feature... But from a core perspective device fault reporting should only ever be turned on in the STE/CD if the attached domain->iopf_handler is not NULL. If it is NULL then any access to a non-present address should trigger some kind of device error failure automatically. This is new core functionality since this code would have been originally written. Now it is all handled transparently by the core code. The driver should just deliver all fault events to iommu_report_device_fault() and it will sort it out. > Signed-off-by: Mostafa Saleh > --- > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 18 +++++++++++++----- > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ > 2 files changed, 15 insertions(+), 5 deletions(-) > > diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > index 8d573d9ca93c..ffa865529d73 100644 > --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > @@ -1733,6 +1733,7 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) > u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]); > struct iopf_fault fault_evt = { }; > struct iommu_fault *flt = &fault_evt.fault; > + struct arm_smmu_domain *smmu_domain; > > switch (FIELD_GET(EVTQ_0_ID, evt[0])) { > case EVT_ID_TRANSLATION_FAULT: > @@ -1744,10 +1745,6 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) > return -EOPNOTSUPP; > } > > - /* Stage-2 is always pinned at the moment */ > - if (evt[1] & EVTQ_1_S2) > - return -EFAULT; > - This makes sense at first blush since the domain mode shouldn't define if events should be processed or not, and the events should be failed anyhow right? If someone did turn on fault reporting in the STE then it should always be processed to conclusion. > if (!(evt[1] & EVTQ_1_STALL)) > return -EOPNOTSUPP; > > @@ -1782,6 +1779,15 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) > goto out_unlock; > } > > + /* It is guaranteed that smmu_domain exists as EVTQ_1_STALL is checked. */ > + smmu_domain = to_smmu_domain(iommu_get_domain_for_dev(master->dev)); Strongly discouraging drivers from calling iommu_get_domain_for_dev() in async paths like this. The locking is tricky and the core code does... > + /* nesting domain is always pinned at the moment */ > + if (smmu_domain->enable_nesting) { This is not necessary - a nesting domain will never have an iopf_handler set. It immediately calls iommu_report_device_fault() which will reject it because of: if (!group->attach_handle->domain->iopf_handler) goto err_abort; Which after the rework will end up in find_fault_handler() at the top of the function: https://lore.kernel.org/r/ZrTNGepJXbmfuKBK@google.com So I think these parts are not necessary. Though arguably we should be rejecting domains with iopf_handler set in some of the attach calls.. Jason