From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E693FC38A30 for ; Sun, 19 Apr 2020 12:25:11 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92C7A214AF for ; Sun, 19 Apr 2020 12:25:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92C7A214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=8bytes.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 60ACB860DB; Sun, 19 Apr 2020 12:25:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2-CEvO2-xxnp; Sun, 19 Apr 2020 12:25:10 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 843778614D; Sun, 19 Apr 2020 12:25:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5EB07C07FF; Sun, 19 Apr 2020 12:25:10 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C76FEC0177 for ; Sun, 19 Apr 2020 12:25:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id B50A2875BE for ; Sun, 19 Apr 2020 12:25:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OHSpsgBaGtyW for ; Sun, 19 Apr 2020 12:25:07 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from theia.8bytes.org (8bytes.org [81.169.241.247]) by hemlock.osuosl.org (Postfix) with ESMTPS id 055F9872A6 for ; Sun, 19 Apr 2020 12:25:07 +0000 (UTC) Received: by theia.8bytes.org (Postfix, from userid 1000) id EB734342; Sun, 19 Apr 2020 14:25:03 +0200 (CEST) Date: Sun, 19 Apr 2020 14:25:02 +0200 From: Joerg Roedel To: Christoph Hellwig Subject: Re: [PATCH 3/4] dma-mapping: add a dma_ops_bypass flag to struct device Message-ID: <20200419122502.GI21900@8bytes.org> References: <20200414122506.438134-1-hch@lst.de> <20200414122506.438134-4-hch@lst.de> <20200418124205.GD6113@8bytes.org> <20200419080058.GB12222@lst.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200419080058.GB12222@lst.de> User-Agent: Mutt/1.10.1 (2018-07-13) Cc: Greg Kroah-Hartman , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Sun, Apr 19, 2020 at 10:00:58AM +0200, Christoph Hellwig wrote: > The difference is that NULL ops mean imply the direct mapping is always > used, dma_ops_bypass means a direct mapping is used if no bounce buffering > using swiotlb is needed, which should also answer your first question. > The idea is to consolidate code in the core to use an opportunistic > direct mapping instead of the dynamic iommu mapping. I though the cover > letter and commit log explained this well enough, but maybe I need to > do a better job. Ah right, now I see it, when dma_ops_bypass is set it will only use direct mapping when the available memory fits into the device's dma_masks, and calls into dma_ops otherwise. I wonder how that will interact with an IOMMU driver, which has to make sure that the direct mapping is accessible for the device at all. It can either put the device into a passthrough domain for direct mapping or into a re-mapped domain, but then all DMA-API calls need to use dma-ops. When the dma_mask covers available memory but coherent_mask doesn't, the streaming calls will use dma-direct and alloc_coherent() calls into dma-ops. There is no way for the IOMMU driver to ensure both works. So what are the conditions under which an IOMMU driver would set dma_ops_bypass to 1 and get a different result as to when setting dev->dma_ops to NULL? Regards, Joerg _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 678B4C38A30 for ; Sun, 19 Apr 2020 12:28:51 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE5C3214AF for ; Sun, 19 Apr 2020 12:28:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE5C3214AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=8bytes.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 494pz44PcczDr0T for ; Sun, 19 Apr 2020 22:28:48 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=8bytes.org (client-ip=2a01:238:4383:600:38bc:a715:4b6d:a889; helo=theia.8bytes.org; envelope-from=joro@8bytes.org; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=8bytes.org X-Greylist: delayed 85366 seconds by postgrey-1.36 at bilbo; Sun, 19 Apr 2020 22:25:14 AEST Received: from theia.8bytes.org (8bytes.org [IPv6:2a01:238:4383:600:38bc:a715:4b6d:a889]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 494ptz0QynzDr0K for ; Sun, 19 Apr 2020 22:25:12 +1000 (AEST) Received: by theia.8bytes.org (Postfix, from userid 1000) id EB734342; Sun, 19 Apr 2020 14:25:03 +0200 (CEST) Date: Sun, 19 Apr 2020 14:25:02 +0200 From: Joerg Roedel To: Christoph Hellwig Subject: Re: [PATCH 3/4] dma-mapping: add a dma_ops_bypass flag to struct device Message-ID: <20200419122502.GI21900@8bytes.org> References: <20200414122506.438134-1-hch@lst.de> <20200414122506.438134-4-hch@lst.de> <20200418124205.GD6113@8bytes.org> <20200419080058.GB12222@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200419080058.GB12222@lst.de> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexey Kardashevskiy , Greg Kroah-Hartman , Robin Murphy , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, Lu Baolu Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Sun, Apr 19, 2020 at 10:00:58AM +0200, Christoph Hellwig wrote: > The difference is that NULL ops mean imply the direct mapping is always > used, dma_ops_bypass means a direct mapping is used if no bounce buffering > using swiotlb is needed, which should also answer your first question. > The idea is to consolidate code in the core to use an opportunistic > direct mapping instead of the dynamic iommu mapping. I though the cover > letter and commit log explained this well enough, but maybe I need to > do a better job. Ah right, now I see it, when dma_ops_bypass is set it will only use direct mapping when the available memory fits into the device's dma_masks, and calls into dma_ops otherwise. I wonder how that will interact with an IOMMU driver, which has to make sure that the direct mapping is accessible for the device at all. It can either put the device into a passthrough domain for direct mapping or into a re-mapped domain, but then all DMA-API calls need to use dma-ops. When the dma_mask covers available memory but coherent_mask doesn't, the streaming calls will use dma-direct and alloc_coherent() calls into dma-ops. There is no way for the IOMMU driver to ensure both works. So what are the conditions under which an IOMMU driver would set dma_ops_bypass to 1 and get a different result as to when setting dev->dma_ops to NULL? Regards, Joerg From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 967AFC3A5A0 for ; Sun, 19 Apr 2020 12:25:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6D407214AF for ; Sun, 19 Apr 2020 12:25:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725988AbgDSMZF (ORCPT ); Sun, 19 Apr 2020 08:25:05 -0400 Received: from 8bytes.org ([81.169.241.247]:36440 "EHLO theia.8bytes.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725939AbgDSMZF (ORCPT ); Sun, 19 Apr 2020 08:25:05 -0400 Received: by theia.8bytes.org (Postfix, from userid 1000) id EB734342; Sun, 19 Apr 2020 14:25:03 +0200 (CEST) Date: Sun, 19 Apr 2020 14:25:02 +0200 From: Joerg Roedel To: Christoph Hellwig Cc: iommu@lists.linux-foundation.org, Alexey Kardashevskiy , linuxppc-dev@lists.ozlabs.org, Lu Baolu , Greg Kroah-Hartman , Robin Murphy , linux-kernel@vger.kernel.org Subject: Re: [PATCH 3/4] dma-mapping: add a dma_ops_bypass flag to struct device Message-ID: <20200419122502.GI21900@8bytes.org> References: <20200414122506.438134-1-hch@lst.de> <20200414122506.438134-4-hch@lst.de> <20200418124205.GD6113@8bytes.org> <20200419080058.GB12222@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200419080058.GB12222@lst.de> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Apr 19, 2020 at 10:00:58AM +0200, Christoph Hellwig wrote: > The difference is that NULL ops mean imply the direct mapping is always > used, dma_ops_bypass means a direct mapping is used if no bounce buffering > using swiotlb is needed, which should also answer your first question. > The idea is to consolidate code in the core to use an opportunistic > direct mapping instead of the dynamic iommu mapping. I though the cover > letter and commit log explained this well enough, but maybe I need to > do a better job. Ah right, now I see it, when dma_ops_bypass is set it will only use direct mapping when the available memory fits into the device's dma_masks, and calls into dma_ops otherwise. I wonder how that will interact with an IOMMU driver, which has to make sure that the direct mapping is accessible for the device at all. It can either put the device into a passthrough domain for direct mapping or into a re-mapped domain, but then all DMA-API calls need to use dma-ops. When the dma_mask covers available memory but coherent_mask doesn't, the streaming calls will use dma-direct and alloc_coherent() calls into dma-ops. There is no way for the IOMMU driver to ensure both works. So what are the conditions under which an IOMMU driver would set dma_ops_bypass to 1 and get a different result as to when setting dev->dma_ops to NULL? Regards, Joerg