From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65570C433F5 for ; Thu, 7 Apr 2022 09:15:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Mime-Version:References:In-Reply-To: Date:Cc:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DLnpBSwMmoMynlUCkL3oEFM3V+6SuYy9ylUlnCkTNOI=; b=j/0r8UlljWiD+v KnCz27B8MFHe5gfdqYB2nmg9Wqp9X2hbtu80ol0Csn7FE40qzJ0s3OEejC1KnYrS9Jw/vEF6Iewds edq43gu8NiWJgl7Wxu/DFWhevuL47IoofLB+wG0r7VcDYlJ2bbM1qT2vHbAnUPyFT78JiDQLKdp9X Ey+lIoxRWGnxs7FTRkWdwAsFCG65s5wN9HoKWs3yzh6rXQJg8fjfDLec8vhC5sUkTgeCKq8jr6Je1 DhMHL/eq4whPz31VYslxH956EenHMboDDXOrtUCca4w2iDcdq1Im+SR3VTm8LTpOgZSGq1EBrEIgc wjPrJKmmLDtFuRgTKRBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncOCv-00Ak51-U9; Thu, 07 Apr 2022 09:13:39 +0000 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncNtV-00AbJi-V7 for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 08:53:35 +0000 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 2377o8qo019769; Thu, 7 Apr 2022 08:53:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=uyNsE8Ex9YI8S1tXwshKVJTLTCexafBD49QkhLU1s24=; b=FIAHZtfeTCueKaFXrFSiIT9mlRhR2N7A9UyZB5RgrAXC+maFHN1717MdP1JqhFOaDibp 4+Y2xKI7/hfsiKKVoGCGX6uJC0t0T162ThJkE0Y9zefCIQyEg7zHTZHyDhCUiaIfz6XF jCMe41zf57d5Y0xYQsBj7BdgvDfx0SBGvssJQCVNrtt19y5F+9TRk9aCSSRSeNf7M56r W3E725XQn1PWg6S8lhDPKeXyziP5whMW23+ig4CAqErOnAgqvf+huPCCiTW0vZaRWTOw f0W3EiowdEZxHY8Yffo+8eRLunyL7OiDNkVA3X+HjbtfvUGXWoeOEL0XClWot27ZsroZ 5g== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f9uwts6ch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 07 Apr 2022 08:53:11 +0000 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 2378ZRZ7031643; Thu, 7 Apr 2022 08:53:11 GMT Received: from ppma05fra.de.ibm.com (6c.4a.5195.ip4.static.sl-reverse.com [149.81.74.108]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f9uwts6br-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 07 Apr 2022 08:53:11 +0000 Received: from pps.filterd (ppma05fra.de.ibm.com [127.0.0.1]) by ppma05fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2378njqQ008880; Thu, 7 Apr 2022 08:53:09 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma05fra.de.ibm.com with ESMTP id 3f6e48qsa8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 07 Apr 2022 08:53:08 +0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2378r5OY48496960 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 7 Apr 2022 08:53:05 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7A230AE051; Thu, 7 Apr 2022 08:53:05 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 52950AE045; Thu, 7 Apr 2022 08:53:04 +0000 (GMT) Received: from sig-9-145-36-59.uk.ibm.com (unknown [9.145.36.59]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 7 Apr 2022 08:53:04 +0000 (GMT) Message-ID: <49d40d562dbccdac58597b863c357b32f0798284.camel@linux.ibm.com> Subject: Re: [PATCH 1/5] iommu: Replace uses of IOMMU_CAP_CACHE_COHERENCY with dev_is_dma_coherent() From: Niklas Schnelle To: Jason Gunthorpe , Christoph Hellwig Cc: Robin Murphy , Alex Williamson , Lu Baolu , Christian Benvenuti , Cornelia Huck , David Woodhouse , Gerald Schaefer , iommu@lists.linux-foundation.org, Jason Wang , Joerg Roedel , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-rdma@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , "Michael S. Tsirkin" , Nelson Escobar , netdev@vger.kernel.org, Rob Clark , Suravee Suthikulpanit , virtualization@lists.linux-foundation.org, Will Deacon , "Tian, Kevin" Date: Thu, 07 Apr 2022 10:53:03 +0200 In-Reply-To: <20220406171729.GJ2120790@nvidia.com> References: <1-v1-ef02c60ddb76+12ca2-intel_no_snoop_jgg@nvidia.com> <20220406142432.GF2120790@nvidia.com> <20220406151823.GG2120790@nvidia.com> <20220406155056.GA30433@lst.de> <20220406160623.GI2120790@nvidia.com> <20220406161031.GA31790@lst.de> <20220406171729.GJ2120790@nvidia.com> X-Mailer: Evolution 3.28.5 (3.28.5-18.el8) Mime-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: -eI2_i-ijsMVu9WpUqTiTy_ZXGgnAkK_ X-Proofpoint-GUID: d3-YNui-0WOxSLMzaNCj1z5uxlf9oVYW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.850,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-04-06_13,2022-04-06_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxscore=0 mlxlogscore=999 lowpriorityscore=0 phishscore=0 priorityscore=1501 impostorscore=0 suspectscore=0 clxscore=1011 adultscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204070043 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_015334_199056_8F6FE089 X-CRM114-Status: GOOD ( 32.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 2022-04-06 at 14:17 -0300, Jason Gunthorpe wrote: > On Wed, Apr 06, 2022 at 06:10:31PM +0200, Christoph Hellwig wrote: > > On Wed, Apr 06, 2022 at 01:06:23PM -0300, Jason Gunthorpe wrote: > > > On Wed, Apr 06, 2022 at 05:50:56PM +0200, Christoph Hellwig wrote: > > > > On Wed, Apr 06, 2022 at 12:18:23PM -0300, Jason Gunthorpe wrote: > > > > > > Oh, I didn't know about device_get_dma_attr().. > > > > > > > > Which is completely broken for any non-OF, non-ACPI plaform. > > > > > > I saw that, but I spent some time searching and could not find an > > > iommu driver that would load independently of OF or ACPI. ie no IOMMU > > > platform drivers are created by board files. Things like Intel/AMD > > > discover only from ACPI, etc. > > > > s390? > > Ah, I missed looking in s390, hyperv and virtio.. > > hyperv is not creating iommu_domains, just IRQ remapping > > virtio is using OF > > And s390 indeed doesn't obviously have OF or ACPI parts.. > > This seems like it would be consistent with other things: > > enum dev_dma_attr device_get_dma_attr(struct device *dev) > { > const struct fwnode_handle *fwnode = dev_fwnode(dev); > struct acpi_device *adev = to_acpi_device_node(fwnode); > > if (is_of_node(fwnode)) { > if (of_dma_is_coherent(to_of_node(fwnode))) > return DEV_DMA_COHERENT; > return DEV_DMA_NON_COHERENT; > } else if (adev) { > return acpi_get_dma_attr(adev); > } > > /* Platform is always DMA coherent */ > if (!IS_ENABLED(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) && > !IS_ENABLED(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) && > !IS_ENABLED(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) && > device_iommu_mapped(dev)) > return DEV_DMA_COHERENT; > return DEV_DMA_NOT_SUPPORTED; > } > EXPORT_SYMBOL_GPL(device_get_dma_attr); > > ie s390 has no of or acpi but the entire platform is known DMA > coherent at config time so allow it. Not sure we need the > device_iommu_mapped() or not. I only took a short look but I think the device_iommu_mapped() call in there is wrong. On s390 PCI always goes through IOMMU hardware both when using the IOMMU API and when using the DMA API and this hardware is always coherent. This is even true for s390 guests in QEMU/KVM and under the z/VM hypervisor. As far as I can see device_iommu_mapped()'s check for dev->iommu_group would only work while the device is under IOMMU API control not DMA API, no? Also, while it is true that s390 *hardware* devices are always cache coherent there is also the case that SWIOTLB is used for protected virtualization and then cache flushing APIs must be used. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel