From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C608FD1358C for ; Mon, 28 Oct 2024 06:23:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4962F6B0089; Mon, 28 Oct 2024 02:23:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4451E6B008C; Mon, 28 Oct 2024 02:23:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 333DE6B0092; Mon, 28 Oct 2024 02:23:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 16A556B0089 for ; Mon, 28 Oct 2024 02:23:01 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 89A611A1D3A for ; Mon, 28 Oct 2024 06:22:20 +0000 (UTC) X-FDA: 82722017490.12.1FA9F40 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id 2E9AA140013 for ; Mon, 28 Oct 2024 06:22:44 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bDs6atF8; spf=pass (imf23.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730096500; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=64HvQqcLdLdKZFwtkgeyQYhtkebvmps8WRfA00nTT0k=; b=Pl840o2hDQZEokhu+tsb0wHIzUFOa1DGOCQIwCpnApWVTckT6JnSXOokrBYym3GCm/XFTB j85FSfhAqYOwuT8ii7ldRrPIb4Au1+VbfIQ05Kus663HfF37xCumQoPVceukSVFQV9UAL+ DeyGznDmbldMFV56bSVKg/76QxxtQug= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bDs6atF8; spf=pass (imf23.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730096500; a=rsa-sha256; cv=none; b=7CT32PRCxQOSPhOeM6xSPoM05SDSq4yoX7fRnf24SmBn+E1M7fN4WAtczSeTGZJOty48Fx BcivpVjMzosNXbAGLwUK6xQg7VJadC657KYXP8eIbf+Z7u0smLjnWoSMY3a+90Oi3XVD4E gHlLg7W8/uVmOjMmCGvaBN+uRK2XNtU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id A04325C56B9; Mon, 28 Oct 2024 06:22:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A0E16C4CEC3; Mon, 28 Oct 2024 06:22:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730096577; bh=Jk5IbqRh4vCRoUbHWTgVvsfDpi3BO4pWidLR8NsFZ9M=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=bDs6atF8lGVkOSu8PR/Zeze3PGZeuSNAwBS1wCvfCB64zduIV8EVNQLYrGRITgsOX m4HZ8igBY57cpFEdhFNUBm8dkFqVcwLIZJNyD3FxqvF/X6ketbJhvrbBPr+u6WZxZ3 w11mQnvuMXHiPD97WTOWcp1gf7Qn8YL4dM5TPF8cGuddYW1HkSqRGukSUz9O+9rzqF KLngHHPHYjOu7rL222u8QMXoukG9r55bOcTa7Rw2qnhI04sopB/+K6VdPCEVj61xlv ViLwDGbr7PuiCkXsEHb96dqrPmRCahF5OFg843XZ1f+zK0SivcsINr8cHzq8IxXcnI li8XZp6ad0yXA== Date: Mon, 28 Oct 2024 08:22:52 +0200 From: Leon Romanovsky To: Baolu Lu Cc: Jens Axboe , Jason Gunthorpe , Robin Murphy , Joerg Roedel , Will Deacon , Christoph Hellwig , Sagi Grimberg , Keith Busch , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , Marek Szyprowski , =?iso-8859-1?B?Suly9G1l?= Glisse , Andrew Morton , Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 07/18] dma-mapping: Implement link/unlink ranges API Message-ID: <20241028062252.GC1615717@unreal> References: <6a9366a5-7c5b-449c-b259-8e2492aae2a1@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6a9366a5-7c5b-449c-b259-8e2492aae2a1@linux.intel.com> X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 2E9AA140013 X-Stat-Signature: rdtatapzmhnamhorxttix7aabxxcdt8t X-HE-Tag: 1730096564-83257 X-HE-Meta: U2FsdGVkX1/SOlF/yBu1Y4jtDygVOemlQJOUFvOm0v4NTcOwzQP7hSYy1lCyxY5yanPMAN3y/uQVj0pHkIGRuEmW9zRW+NVaUqq1588b3sTZWJ1mLKCLOH207+vsv9MQ35Z4GxwtrYYVGvIFytLs/bi7IBIkc3KAzne+8Cv0OlFLpTQolKVfYxprgDPHmCLppj2PYG0zKDwWH5Wpjx8sUxQa9vv1v9BcZCF8Y5fFMSoUAMsaE1/o4cnTA6U6bzzq1BwkpBvpMpOGmglixxyNsxxn+FHN+zpg804iEYBv7/cdK4ovtsXMfFf1eXKQ0eLTrh1tGfmODln6rHnjZermrLRg5JUjPAf0NxT0U4e8/7U+ArU4FR+2SyVvz74MJ6L7KbDNflC/QZ+Yr8KPujNFwVwY7gZpuAbx+GXzQX3MYLFK+K9xbDemwj//seBnw1x8hwlUwecSFlJ22lSURS14/rCMuH3MIC/8UE9yZMMjsK/pFJWLggjT0lgKRCcJiNmnjFozev4xNMdkqSk0gybCQPL9MhyrL+/HJml68mL0tMU7jfX6ZlG5oXZOgQrvNToBA0vMjokiBbfICmgqkHtntkp/JICJW8nyg2+m8TFR0u3aW67twoUTk9V4qeLP4HjlDBvON5gKIF1Dz6pX0rnZFEDWY4tI7arziIty1ob6b3NcWp1lETYlQVmx0fIXC9y/lpm07P+1audeqOmn4HlPjyUaQb37K801jJ3njYn7YQngPlUQZmyLAl1koeWjQpBRAVC5O2IzA2luftDQV6DLdsXr1S4e2560qNW9PDiITNPVP4ndE27Hj/OmvCXpGRNAyKiWjz6pipjx18msol1eRGy+Uzw8FHBjt3KBOQg9WVzUa+IBJMopMKXVfIb6hGCJdMwVUzXdSh8wdIf3rfF5ATwO3pLrmV9CSVTEdyHX/Lu2be9cClJlAHjtigmPOEP/cRwmrTUXYXUeOnIV6BX 6nKH9FKP DzDgxVw1xDNGft6jAP69mRKX+i4E/NX3lltNk6StHvuyP26TAcWrJSxTbc7heZZRt6tcACN4ShCMeOb/Kq1qDv3+Yn23mQtKkGI5pEYeFjlMOwM1sqgTOlZlW4NVGt2Oj6xmA1gk47PWfqNVLLqwQb+mgIcajEG179KFQkst8AyPK52NN33B6iTgZeqJP4p8yBRVXI0nOh4nho35XtgRpcYPl/JHz0QmsInUDldmsyO6HB6I6rp0XxaxznIsHkNr5by3qXOQNiA2kLkQMJwLUB/bazQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Oct 28, 2024 at 10:00:25AM +0800, Baolu Lu wrote: > On 2024/10/27 22:21, Leon Romanovsky wrote: > > +/** > > + * dma_iova_sync - Sync IOTLB > > + * @dev: DMA device > > + * @state: IOVA state > > + * @offset: offset into the IOVA state to sync > > + * @size: size of the buffer > > + * @ret: return value from the last IOVA operation > > + * > > + * Sync IOTLB for the given IOVA state. This function should be called on > > + * the IOVA-contigous range created by one ore more dma_iova_link() calls > > + * to sync the IOTLB. > > + */ > > +int dma_iova_sync(struct device *dev, struct dma_iova_state *state, > > + size_t offset, size_t size, int ret) > > +{ > > + struct iommu_domain *domain = iommu_get_dma_domain(dev); > > + struct iommu_dma_cookie *cookie = domain->iova_cookie; > > + struct iova_domain *iovad = &cookie->iovad; > > + dma_addr_t addr = state->addr + offset; > > + size_t iova_start_pad = iova_offset(iovad, addr); > > + > > + addr -= iova_start_pad; > > + size = iova_align(iovad, size + iova_start_pad); > > + > > + if (!ret) > > + ret = iommu_sync_map(domain, addr, size); > > + if (ret) > > + iommu_unmap(domain, addr, size); > > It appears strange that mapping is not done in this helper, but > unmapping is added in the failure path. Perhaps I overlooked anything? Like iommu_sync_map() is performed on whole continuous range, the iommu_unmap() should be done on the same range. So, technically you can unmap only part of the range which called to dma_iova_link() and failed, but you will need to make sure that iommu_sync_map() is still called for "successful" part of iommu_map(). In that case, you will need to undo everything anyway and it means that you will call to iommu_unmap() on the successful part of the range anyway. dma_iova_sync() is single operation for the whole range and iommu_unmap() too, so they are bound together. > To my understanding, it should like below: > > return iommu_sync_map(domain, addr, size); > > In the drivers that make use of this interface should do something like > below: > > ret = dma_iova_sync(...); > if (ret) > dma_iova_destroy(...) It is actually what is happening in the code, but in less direct way due to unwinding of the code. As an simple example, see VFIO patch https://lore.kernel.org/all/0a517ddff099c14fac1ceb0e75f2f50ed183d09c.1730037276.git.leon@kernel.org/ where failed in dma_iova_sync() will trigger call to unregister_dma_pages() and that will call to dma_iova_destroy(). > > > + return ret; > > +} > > +EXPORT_SYMBOL_GPL(dma_iova_sync); > > Thanks, > baolu >