From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A090FC30653 for ; Thu, 4 Jul 2024 13:18:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1361D6B0096; Thu, 4 Jul 2024 09:18:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C0136B0099; Thu, 4 Jul 2024 09:18:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7AAB6B009A; Thu, 4 Jul 2024 09:18:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C5AF76B0096 for ; Thu, 4 Jul 2024 09:18:53 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 67D7EA16BF for ; Thu, 4 Jul 2024 13:18:53 +0000 (UTC) X-FDA: 82302125346.06.B6A8F30 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf06.hostedemail.com (Postfix) with ESMTP id 164C1180017 for ; Thu, 4 Jul 2024 13:18:50 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="D/KR/8A6"; spf=pass (imf06.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720099106; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FiW+P+gEbk03re9caf8GbaZIS2rrKaSGcFy9JQgS9RY=; b=y/8wYefLBxtpPS68FYLw7W3TzKPv2NfFxv4BAwLyQAnZoNrRRanzgXYHnE73NJfrE36XfX pRe0NybEFtrx3z+6PcAGvE8Sk4KFYfsnK4wUUtQXaoGR9ectR0/Ged73jMsxpdOf2iufAy 9RO361a4/BKMZO+ca347Rw8sAcPS5LM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720099106; a=rsa-sha256; cv=none; b=TyT+NpKr3LXfwghTESYtCxeBsfc8S2+ZPF0Kih6EIcvfwb3PhfHHjjkcukRyj5VgQV6EIn gFyAp0X77ROyXVxbfG+jrrNFYa23RDJcnDGSeXQ8BLQPQOIPG93mCx/wylG8PC2FAFsmCZ SFYh0PLsJBRsvlr3W6YhaGVmai/gUD4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="D/KR/8A6"; spf=pass (imf06.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 1DBFFCE33EF; Thu, 4 Jul 2024 13:18:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0368AC3277B; Thu, 4 Jul 2024 13:18:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720099124; bh=nkzC1R2Sn6KADvFNDPp0hrnIwln3xVG+eb9R42IPSdQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=D/KR/8A6xd5D4R/wyMmVXVBhTLC00uoxpBMQxG4XlqhIi5KAQVaC2WfSC/JKuTQBK kEFxwYczrrp8jHD/rU7Io7O7FanCrAZSqBu/PdOnbPEDswb3D7N14ui//u++auZfNR SUqz2Rl1HBcmgvgZwCQlOkR19edRQbzDh6tI/mypTJD/RTH4hYtagEvYh76WNN1/VY UCe/SDCeqqwlLAKAvWNk+uJtnN8vjygt8Gaomlc3u7lhSQZuSImjGamtHE5l/qm9H/ l7cdU0pdx3G3PZZgD6CMT+HJGLLMzzhXBkWp2c6sJY4t+AJp3EHJFmV84AgiTky6jN DmhQgo4POHvOw== Date: Thu, 4 Jul 2024 16:18:39 +0300 From: Leon Romanovsky To: Christoph Hellwig Cc: Jens Axboe , Jason Gunthorpe , Robin Murphy , Joerg Roedel , Will Deacon , Keith Busch , "Zeng, Oak" , Chaitanya Kulkarni , Sagi Grimberg , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , Marek Szyprowski , =?iso-8859-1?B?Suly9G1l?= Glisse , Andrew Morton , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH v1 00/18] Provide a new two step DMA API mapping API Message-ID: <20240704131839.GD95824@unreal> References: <20240703054238.GA25366@lst.de> <20240703105253.GA95824@unreal> <20240703143530.GA30857@lst.de> <20240703155114.GB95824@unreal> <20240704074855.GA26913@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240704074855.GA26913@lst.de> X-Stat-Signature: acxknj5nacyk4wajh67eykw8yp3ynjey X-Rspamd-Queue-Id: 164C1180017 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720099130-283436 X-HE-Meta: U2FsdGVkX19IHC5H1Bu2VoLe9Du4WKvjvVfCPoZRjTvvQ5axwFPjnIkbmkEbFX4udDvq9HzYTzbgj/UxfTkAfnZYkpt+ZGr7/lUk9OVBIqMveyovmX4Hj7RvtUI25Z1f9Etgm1lqeQeqY1xJoUOsMxEaUp2qm17k7jU6i/jVpAGZNR2MD7YcGhM+lnTWgf46azXBnE6sID5aKlA4AP5jFU/SwkdJewPtxrGndialRYQNtvE0LhZ7AqHG+Rp2qz3TlxjSuL+BLF7qLVWYAQ+5kQdKBeW5gFzd/P+6cVQ3H4ITTQfxiJI5GRt2C5rVNdJPv76OLYcXc7aq7rx7S39Zi6dEV6xCDh8kPVMMxSXUE92Wqff8RAlUIWVN75CDnny/cHKA4hgLP01gcdrKIArbs1l7fhwQLJpewnjIRIFiH4pwRbGoFxZ1sGMPovpyWffSo3ks3t56ZWcbe92zZTjUKxyAvn/jTMIYoND2wfHyfZXsFGUB1TIALuipn+Ft5Kuw6EeHyGxu0JBuRY5EifM/YFFriYhnm/NWGlXp1VMGaKb2G8KRMcPNRc1h8+/FVy9xpGQMtwIkOvL7N3mgdZ/YU/602x/nVIDo0yiknZUov1NO8WxAIvitrROQFLoD15CW1n+rDXVuPx8uspSm8j91AJlR/8PJ/PidzArCH90roi1U+QP2r3J+v9DOo/uwSNPTzfzBSOoGbQ3M1VzQdMO/rT36FFxLBA1JsVFAYBjTO+8ZmUfhs9C7rWFpABmd9/XMuoC8z15eDNzN1j7CENaiqC2eaqLpjzlCpuAkWFFvdRiLb7UMBevKiKi86sAI7VCwAa2oYC+/oDQTfwLVV/NHfimc1HNb5M7FVjKRIKxuAgdIVkYGxYU8K12KY3GcKL3J7ghy4/F832++q8VlpV/M1hNNm4jUM6vCx1kyLLbhspqHfWDXDbkseTES1rMFurbQkEYmwSpGhLUVkfL+AQD VP+7oS4H x+HuamB8ZGFw3wh02+veKrg1kyXEkkyYKprEVMKWVkP5pWcgxUo9Jtxed+olGYF57fQaZvQjvNOBIJVB+8NdDq6H/bTcdgr+IMyMXN3xXf5tddbpkwfeSyQ9fmfqPY6gBGHac4al6pYpz+Ju42L/9qaszyT+H/ePffarcO+B8zpyMjMuaVD2YJkRcPsLNKza/uz6LaBaJrbVRFlo9mp8y7hlbqcS+g0CzLR5JjDgvWpvt6PlXmPReDtGcOD1/LlJYpK6Eb9Iua4P9a7TP32HXAp0KHA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jul 04, 2024 at 09:48:56AM +0200, Christoph Hellwig wrote: > On Wed, Jul 03, 2024 at 06:51:14PM +0300, Leon Romanovsky wrote: > > If we put aside this issue, do you think that the proposed API is the right one? > > I haven't look at it in detail yet, but from a quick look there is a > few things to note: > > > 1) The amount of code needed in nvme worries me a bit. Now NVMe a messy > driver due to the stupid PRPs vs just using SGLs, but needing a fair > amount of extra boilerplate code in drivers is a bit of a warning sign. > I plan to look into this to see if I can help on improving it, but for > that I need a working version first. Chaitanya is working on this and I'll join him to help on next Sunday, after I'll return to the office from my sick leave/ > > > 2) The amount of seemingly unrelated global headers pulled into other > global headers. Some of this might just be sloppiness, e.g. I can't > see why dma-mapping.h would actually need iommu.h to start with, > but pci.h in dma-map-ops.h is a no-go. pci.h was pulled because I needed to call to pci_p2pdma_map_type() in dma_can_use_iova(). > > 3) which brings me to real layering violations. dev_is_untrusted and > dev_use_swiotlb are DMA API internals, no way I'd ever want to expose > them. dma-map-ops.h is a semi-internal header only for implementations > of the dma ops (as very clearly documented at the top of that file), > it must not be included by drivers. Same for swiotlb.h. These item shouldn't worry you and will be changed in the final version. They are outcome of patch "RDMA/umem: Prevent UMEM ODP creation with SWIOTLB". https://lore.kernel.org/all/d18c454636bf3cfdba9b66b7cc794d713eadc4a5.1719909395.git.leon@kernel.org/ All HMM users need such "prevention" so it will be moved to a common place. > > Not quite as concerning, but doing an indirect call for each map > through dma_map_ops in addition to the iommu ops is not every efficient. > We've through for a while to allow direct calls to dma-iommu similar > how we do direct calls to dma-direct from the core mapping.c code. > This might be a good time to do that as a prep step for this work. Sure, no problem, will start in parallel to work on this. >