From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7A053C433FE for ; Sat, 20 Nov 2021 11:20:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0710860706; Sat, 20 Nov 2021 11:20:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RzUM1okmrUyr; Sat, 20 Nov 2021 11:20:53 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id E145D60008; Sat, 20 Nov 2021 11:20:52 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B8CB4C001E; Sat, 20 Nov 2021 11:20:52 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3FEDCC0012 for ; Sat, 20 Nov 2021 11:20:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2AD4960706 for ; Sat, 20 Nov 2021 11:20:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 85YsY8D06IAB for ; Sat, 20 Nov 2021 11:20:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8340A60008 for ; Sat, 20 Nov 2021 11:20:50 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10173"; a="234388336" X-IronPort-AV: E=Sophos;i="5.87,250,1631602800"; d="scan'208";a="234388336" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2021 03:20:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,250,1631602800"; d="scan'208";a="496214393" Received: from allen-box.sh.intel.com (HELO [10.239.159.118]) ([10.239.159.118]) by orsmga007.jf.intel.com with ESMTP; 20 Nov 2021 03:20:44 -0800 Subject: Re: [PATCH 01/11] iommu: Add device dma ownership set/release interfaces To: =?UTF-8?B?SsO2cmcgUsO2ZGVs?= References: <20211115020552.2378167-1-baolu.lu@linux.intel.com> <20211115020552.2378167-2-baolu.lu@linux.intel.com> <20211116134603.GA2105516@nvidia.com> <20211118133325.GO2105516@nvidia.com> <75100dfd-9cfe-9f3d-531d-b4d30de03e76@linux.intel.com> <20211119150612.jhsvsbzisvux2lga@8bytes.org> From: Lu Baolu Message-ID: <48cf6b2b-28ee-178d-6471-460e781e7b20@linux.intel.com> Date: Sat, 20 Nov 2021 19:16:13 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <20211119150612.jhsvsbzisvux2lga@8bytes.org> Content-Language: en-US Cc: Alex Williamson , "Tian, Kevin" , Chaitanya Kulkarni , "Raj, Ashok" , "kvm@vger.kernel.org" , "rafael@kernel.org" , Greg Kroah-Hartman , Cornelia Huck , "linux-kernel@vger.kernel.org" , Christoph Hellwig , "iommu@lists.linux-foundation.org" , "Pan, Jacob jun" , Jason Gunthorpe , "linux-pci@vger.kernel.org" , Bjorn Helgaas , Will Deacon , Diana Craciun X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" SGkgSm9lcmcsCgpPbiAxMS8xOS8yMSAxMTowNiBQTSwgSsO2cmcgUsO2ZGVsIHdyb3RlOgo+IE9u IEZyaSwgTm92IDE5LCAyMDIxIGF0IDA3OjE0OjEwUE0gKzA4MDAsIEx1IEJhb2x1IHdyb3RlOgo+ PiBUaGUgc2luZ2xldG9uIGdyb3VwIHJlcXVpcmVtZW50IGZvciBpb21tdV9hdHRhY2gvZGV0YWNo X2RldmljZSgpIHdhcwo+PiBhZGRlZCBieSBiZWxvdyBjb21taXQ6Cj4+Cj4+IGNvbW1pdCA0MjZh MjczODM0ZWFlNjVhYmNmYzcxMzJhMjFhODViMzE1MWUwYmNlCj4+IEF1dGhvcjogSm9lcmcgUm9l ZGVsIDxqcm9lZGVsQHN1c2UuZGU+Cj4+IERhdGU6ICAgVGh1IE1heSAyOCAxODo0MTozMCAyMDE1 ICswMjAwCj4+Cj4+ICAgICAgaW9tbXU6IExpbWl0IGlvbW11X2F0dGFjaC9kZXRhY2hfZGV2aWNl IHRvIGRldmljZXMgd2l0aCB0aGVpciBvd24gZ3JvdXAKPj4KPj4gICAgICBUaGlzIHBhdGNoIGNo YW5nZXMgdGhlIGJlaGF2aW9yIG9mIHRoZSBpb21tdV9hdHRhY2hfZGV2aWNlCj4+ICAgICAgYW5k IGlvbW11X2RldGFjaF9kZXZpY2UgZnVuY3Rpb25zLiBXaXRoIHRoaXMgY2hhbmdlIHRoZXNlCj4+ ICAgICAgZnVuY3Rpb25zIG9ubHkgd29yayBvbiBkZXZpY2VzIHRoYXQgaGF2ZSB0aGVpciBvd24g Z3JvdXAuCj4+ICAgICAgRm9yIGFsbCBvdGhlciBkZXZpY2VzIHRoZSBpb21tdV9ncm91cF9hdHRh Y2gvZGV0YWNoCj4+ICAgICAgZnVuY3Rpb25zIG11c3QgYmUgdXNlZC4KPj4KPj4gICAgICBTaWdu ZWQtb2ZmLWJ5OiBKb2VyZyBSb2VkZWwgPGpyb2VkZWxAc3VzZS5kZT4KPj4KPj4gSm9lcmcsY2Fu IHlvdSBwbGVhc2Ugc2hlZCBzb21lIGxpZ2h0IG9uIHRoZSBiYWNrZ3JvdW5kIG9mIHRoaXMKPj4g cmVxdWlyZW1lbnQ/IERvZXMgYWJvdmUgaWRlYSBvZiB0cmFuc2l0aW9uIGZyb20gc2luZ2xldG9u IGdyb3VwCj4+IHRvIGdyb3VwIHdpdGggc2luZ2xlIGRyaXZlciBib3VuZCBtYWtlIHNlbnNlIHRv IHlvdT8KPiAKPiBUaGlzIGNoYW5nZSBjYW1lIHRvIGJlIGJlY2F1c2UgdGhlIGlvbW11X2F0dGFj aC9kZXRhY2hfZGV2aWNlKCkKPiBpbnRlcmZhY2UgZG9lc24ndCBmaXQgd2VsbCBpbnRvIGEgd29y bGQgd2l0aCBpb21tdS1ncm91cHMuIERldmljZXMKPiB3aXRoaW4gYSBncm91cCBhcmUgYnkgZGVm aW5pdGlvbiBub3QgaXNvbGF0ZWQgYmV0d2VlbiBlYWNoIG90aGVyLCBzbwo+IHRoZXkgbXVzdCBh bGwgYmUgaW4gdGhlIHNhbWUgYWRkcmVzcyBzcGFjZSAoPT0gaW9tbXVfZG9tYWluKS4gU28gaXQK PiBkb2Vzbid0IG1ha2Ugc2Vuc2UgdG8gYWxsb3cgYXR0YWNoaW5nIGEgc2luZ2xlIGRldmljZSB3 aXRoaW4gYSBncm91cCB0bwo+IGEgZGlmZmVyZW50IGlvbW11X2RvbWFpbi4KClRoYW5rcyBmb3Ig dGhlIGV4cGxhbmF0aW9uLiBJdCdzIHZlcnkgaGVscGZ1bC4gVGhlcmUgc2VlbXMgdG8gYmUgYSBs b3QKb2YgZGlzY3Vzc2lvbnMgYXJvdW5kIHRoaXMsIGJ1dCBJIGRpZG4ndCBzZWUgYW55IG1lYW5p bmdmdWwgcmVhc29ucyB0bwpicmVhayB0aGUgYXNzdW1wdGlvbiBvZiAiYWxsIGRldmljZXMgaW4g YSBncm91cCBiZWluZyBpbiBhIHNhbWUgYWRkcmVzcwpzcGFjZSIuCgpCZXN0IHJlZ2FyZHMsCmJh b2x1Cgo+IAo+IEkga25vdyB0aGF0IGluIHRoZW9yeSBpdCBpcyBzYWZlIHRvIGFsbG93IGRldmlj ZXMgd2l0aGluIGEgZ3JvdXAgdG8gYmUKPiBpbiBkaWZmZXJlbnQgZG9tYWlucyBiZWNhdXNlIHRo ZXJlIGlvbW11LWdyb3VwcyBjYXRjaCBtdWx0aXBsZQo+IG5vbi1pc29sYXRpb24gY2FzZXM6Cj4g Cj4gCTEpIERldmljZXMgYmVoaW5kIGEgbm9uLUFDUyBjYXBhYmxlIGJyaWRnZSBvciBtdWx0aXBs ZSBmdW5jdGlvbnMKPiAJICAgb2YgYSBQQ0kgZGV2aWNlLiBIZXJlIGl0IGlzIHNhZmUgdG8gcHV0 IHRoZSBkZXZpY2VzIGludG8KPiAJICAgZGlmZmVyZW50IGlvbW11LWRvbWFpbnMgYXMgbG9uZyBh cyBhbGwgYWZmZWN0ZWQgZGV2aWNlcyBhcmUKPiAJICAgY29udHJvbGxlZCBieSB0aGUgc2FtZSBv d25lci4KPiAKPiAJMikgRGV2aWNlcyB3aGljaCBzaGFyZSBhIHNpbmdsZSByZXF1ZXN0LWlkIGFu ZCBjYW4ndCBiZQo+IAkgICBkaWZmZXJlbnRpYXRlZCBieSB0aGUgSU9NTVUgaGFyZHdhcmUuIFRo ZXNlIGFsd2F5cyBuZWVkIHRvIGJlCj4gCSAgIGluIHRoZSBzYW1lIGlvbW11X2RvbWFpbi4KPiAK PiBUbyBsaWZ0IHRoZSBzaW5nbGUtZG9tYWluLXBlci1ncm91cCByZXF1aXJlbWVudCB0aGUgaW9t bXUgY29yZSBjb2RlCj4gbmVlZHMgdG8gbGVhcm4gdGhlIGRpZmZlcmVuY2UgYmV0d2VlbiB0aGUg dHdvIGNhc2VzIGFib3ZlLgo+IAo+IFJlZ2FyZHMsCj4gCj4gCUpvZXJnCj4gCl9fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmlvbW11IG1haWxpbmcgbGlzdApp b21tdUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRh dGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby9pb21tdQ== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FEC7C433F5 for ; Sat, 20 Nov 2021 11:20:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231252AbhKTLXx (ORCPT ); Sat, 20 Nov 2021 06:23:53 -0500 Received: from mga07.intel.com ([134.134.136.100]:55992 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231161AbhKTLXx (ORCPT ); Sat, 20 Nov 2021 06:23:53 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10173"; a="297974069" X-IronPort-AV: E=Sophos;i="5.87,250,1631602800"; d="scan'208";a="297974069" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2021 03:20:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,250,1631602800"; d="scan'208";a="496214393" Received: from allen-box.sh.intel.com (HELO [10.239.159.118]) ([10.239.159.118]) by orsmga007.jf.intel.com with ESMTP; 20 Nov 2021 03:20:44 -0800 Cc: baolu.lu@linux.intel.com, "Tian, Kevin" , Jason Gunthorpe , Chaitanya Kulkarni , "Raj, Ashok" , "kvm@vger.kernel.org" , "rafael@kernel.org" , Greg Kroah-Hartman , Cornelia Huck , Will Deacon , "linux-kernel@vger.kernel.org" , "iommu@lists.linux-foundation.org" , Christoph Hellwig , Alex Williamson , "Pan, Jacob jun" , "linux-pci@vger.kernel.org" , Bjorn Helgaas , Diana Craciun Subject: Re: [PATCH 01/11] iommu: Add device dma ownership set/release interfaces To: =?UTF-8?B?SsO2cmcgUsO2ZGVs?= References: <20211115020552.2378167-1-baolu.lu@linux.intel.com> <20211115020552.2378167-2-baolu.lu@linux.intel.com> <20211116134603.GA2105516@nvidia.com> <20211118133325.GO2105516@nvidia.com> <75100dfd-9cfe-9f3d-531d-b4d30de03e76@linux.intel.com> <20211119150612.jhsvsbzisvux2lga@8bytes.org> From: Lu Baolu Message-ID: <48cf6b2b-28ee-178d-6471-460e781e7b20@linux.intel.com> Date: Sat, 20 Nov 2021 19:16:13 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <20211119150612.jhsvsbzisvux2lga@8bytes.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Hi Joerg, On 11/19/21 11:06 PM, Jörg Rödel wrote: > On Fri, Nov 19, 2021 at 07:14:10PM +0800, Lu Baolu wrote: >> The singleton group requirement for iommu_attach/detach_device() was >> added by below commit: >> >> commit 426a273834eae65abcfc7132a21a85b3151e0bce >> Author: Joerg Roedel >> Date: Thu May 28 18:41:30 2015 +0200 >> >> iommu: Limit iommu_attach/detach_device to devices with their own group >> >> This patch changes the behavior of the iommu_attach_device >> and iommu_detach_device functions. With this change these >> functions only work on devices that have their own group. >> For all other devices the iommu_group_attach/detach >> functions must be used. >> >> Signed-off-by: Joerg Roedel >> >> Joerg,can you please shed some light on the background of this >> requirement? Does above idea of transition from singleton group >> to group with single driver bound make sense to you? > > This change came to be because the iommu_attach/detach_device() > interface doesn't fit well into a world with iommu-groups. Devices > within a group are by definition not isolated between each other, so > they must all be in the same address space (== iommu_domain). So it > doesn't make sense to allow attaching a single device within a group to > a different iommu_domain. Thanks for the explanation. It's very helpful. There seems to be a lot of discussions around this, but I didn't see any meaningful reasons to break the assumption of "all devices in a group being in a same address space". Best regards, baolu > > I know that in theory it is safe to allow devices within a group to be > in different domains because there iommu-groups catch multiple > non-isolation cases: > > 1) Devices behind a non-ACS capable bridge or multiple functions > of a PCI device. Here it is safe to put the devices into > different iommu-domains as long as all affected devices are > controlled by the same owner. > > 2) Devices which share a single request-id and can't be > differentiated by the IOMMU hardware. These always need to be > in the same iommu_domain. > > To lift the single-domain-per-group requirement the iommu core code > needs to learn the difference between the two cases above. > > Regards, > > Joerg >