From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD1396FC1 for ; Mon, 14 Nov 2022 16:18:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668442686; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hxvXy2EJc9Qd0pwiulW/osrJ5/Pv/7EvjVfPKadkE/I=; b=iCszBhR/W/l+/xv//Y/yM+yZA60ECwUWEN8iNJ1NA4dyBAKfCFyNCqz62QxrPvU3HPLWKe 5HFGkJEU3WTjDCoAvswtmMS0/0kIwJ7uMYBqZ8jWJBwgniT4h5fhmtdozSUmQN1S1jgDLK B+To5HYNGWQz11gQf7sQwfIFsxkKGjo= Received: from mail-il1-f198.google.com (mail-il1-f198.google.com [209.85.166.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-376-mtMeh9YjOiOIt193nTe7kQ-1; Mon, 14 Nov 2022 11:18:05 -0500 X-MC-Unique: mtMeh9YjOiOIt193nTe7kQ-1 Received: by mail-il1-f198.google.com with SMTP id s4-20020a056e02216400b003021b648144so9602963ilv.19 for ; Mon, 14 Nov 2022 08:18:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hxvXy2EJc9Qd0pwiulW/osrJ5/Pv/7EvjVfPKadkE/I=; b=oWsuCBh7c7DsgsyjosWni9VF73C25fKLn3S/M21wjL+9SoC4V2kyqhIkFEo5QgF6SS zvqhLsCRfFNsqKpTYQq7Fj9gVrDXVWDpS3XI4DFkB2UcXZ6ilpCewZoE+Z/mHnHFTvwJ 61i7a0o9pZvullt8Pza0rp87Nto4f7+cL9YPZc1oNZo5aK/QLeZCSFbNrMn9OqH5nQhu LFOnuPm0N+YKfGVKAZ/Z/GtlbM/dSpA5h+Z7CnhKlYTlDulj3O1TWCWnQjbBThWgb1UL gKJPudADVzW1K3RrZxmnlQmKzIcKWmYHpBdMyfjrrd9MhSD5AMRUKW3PTKQl2eRJRcDi g8rg== X-Gm-Message-State: ANoB5pkPwQ+Fm/ssAxpzz9JvG6ds7Lb3OAFgPRoHWc3dGSEJwbDzue1w OzqTg8stM1lpD+lFOiSivbfcCs0Lz1fdzkify8aLHPTuRNkOKqmGK7VL9RUFvTIiXEQaAbRyRt2 0sLujLqLk9IF5zmc= X-Received: by 2002:a05:6e02:1d06:b0:300:f4a5:7266 with SMTP id i6-20020a056e021d0600b00300f4a57266mr6579912ila.273.1668442684531; Mon, 14 Nov 2022 08:18:04 -0800 (PST) X-Google-Smtp-Source: AA0mqf7lPMQWW+38n5rm2EgYIjtpOtDqi7Qiix94SnrgiMLorpuKojFAx39PuDP8JvI8meve6GVa+A== X-Received: by 2002:a05:6e02:1d06:b0:300:f4a5:7266 with SMTP id i6-20020a056e021d0600b00300f4a57266mr6579906ila.273.1668442684300; Mon, 14 Nov 2022 08:18:04 -0800 (PST) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id f19-20020a05660215d300b006ddf70e3ce8sm2290113iow.7.2022.11.14.08.18.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Nov 2022 08:18:03 -0800 (PST) Date: Mon, 14 Nov 2022 09:18:02 -0700 From: Alex Williamson To: "leohou@tom.com" Cc: virtualization , iommu , qemu-arm , jasowang , leohou1402 Subject: Re: vIOMMU&IOMMU gIOVA to HPA mapping Message-ID: <20221114091802.6be5099e.alex.williamson@redhat.com> In-Reply-To: <2022111420314945171759@tom.com> References: <2022111420314945171759@tom.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.34; x86_64-redhat-linux-gnu) Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Mon, 14 Nov 2022 20:31:49 +0800 "leohou@tom.com" wrote: > Hi=EF=BC=8C > Here is my application scenario=EF=BC=9A > 1. The NIC (Network Information Center) passes through to the VM(Virtual = Machine); > 2. The VM uses the user mode driver DPDK; >=20 > Question: > 1. vIOMMU maintains the mapping gIOVA->gPA, When do you use this gPA ? QEMU in the host derives the hVA from the gPA. The vIOMMU driver in QEMU is triggering the gIOVA to hVA mapping through vfio in the host. > 2. Physical IOMMU maintains the GIOVA->HPA mapping ? If so, by what mean= s (gIOVA -> HPA) mapping ? As above, the vIOMMU in the guest provides gIOVA -> gPA, in QEMU we do the gPA -> hVA, then vfio in the host kernel performs hVA -> hPA via page pinning. > 3. What does QEMU do in NIC pass-through address translation =EF=BC=9F The guest visible vIOMMU triggers MemoryListener notifications in QEMU for the device address space, which insert and removes mappings to the vfio layer below it. Thanks, Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 2002:a17:906:80c3:b0:7ae:d8f:8937 with SMTP id a3csp2932409ejx; Mon, 14 Nov 2022 08:18:14 -0800 (PST) X-Google-Smtp-Source: AA0mqf4yT46mjHRvkame6M5RtFWYnw35J8ubD14MXgj/Cq1NDEULWIKmNy4pYkyd0k7ZgOEa/hJd X-Received: by 2002:a17:903:2403:b0:186:9fb9:1f8c with SMTP id e3-20020a170903240300b001869fb91f8cmr45867plo.69.1668442694533; Mon, 14 Nov 2022 08:18:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668442694; cv=none; d=google.com; s=arc-20160816; b=QTfhoDFOdZRIVVU0Zp7mF/jguLGO4oiORldw3NBUz+f/jrIfUh4uLvCajKFor3bO9a AKC6Rvl72aayBfI5nLCGY79c2DFhoCtPWaK32bnjQfhkmuLxozVT7DhfZaaAbzpPRKsB MzxLDksNn+TZMczWNk4nImO7+YRXeWd0xB8geQ1qYWDuPQEHQGgGD0xItRlbWMWAw7Dg QVn7Yjr6R9ERzM1BbJfGKAfat5X9ifkvths+PQry3uEZpJtGR41CFzVcp/s1avwAk7wM h2K1TwzG7xdRoGdpl2dj1zAtWnvIEQRB5P/s3kibBaQqOZvvzMzG+JNcOO8su5/5tJu2 QWMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:cc :mime-version:references:in-reply-to:message-id:subject:to:from:date :dkim-signature:dkim-filter:dkim-filter:delivered-to:dkim-filter :dkim-filter; bh=HeyrOS6KbhmreczGWsziwM1ar3gbwO4l/jP1boNV7UI=; b=lsalY8lWPEtzVk1idgdKgalgv0R7/rFrfDsQjCMts6Xhv6yk98DTBmrnXkfEXJgT3T g2E/MJBNAznES/Bcr+3+wbVDH6SzXwCwts/zgvVxzZvrQ84VmQcwAJ/alCN9hEub6dKT WGEOnWmQnI/maFENiBrzo3/55N1ePiq1aVrieft7B9qGmE3c5b1hPiDgONHLWNyvT7fc Fm0KTbyCrAcuHxglQXjZMqXtUytnH2OlYnegASkww8CaAZuN9tl+98lQy+Xbe2B3dXQl RqpygnVOsqHiEv6Ot01ReV5nMiOj8r5pzrRuSCCOHYniGtEBQqkSv+orv8E0YrF0kXn5 IvUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@redhat.com header.s=mimecast20190719 header.b="L5NSdFA/"; spf=pass (google.com: domain of virtualization-bounces@lists.linux-foundation.org designates 2605:bc80:3010::136 as permitted sender) smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from smtp3.osuosl.org (smtp3.osuosl.org. [2605:bc80:3010::136]) by mx.google.com with ESMTPS id pc3-20020a17090b3b8300b0021824fcc6b9si6158400pjb.63.2022.11.14.08.18.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Nov 2022 08:18:14 -0800 (PST) Received-SPF: pass (google.com: domain of virtualization-bounces@lists.linux-foundation.org designates 2605:bc80:3010::136 as permitted sender) client-ip=2605:bc80:3010::136; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@redhat.com header.s=mimecast20190719 header.b="L5NSdFA/"; spf=pass (google.com: domain of virtualization-bounces@lists.linux-foundation.org designates 2605:bc80:3010::136 as permitted sender) smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id AB3ED60DFC; Mon, 14 Nov 2022 16:18:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AB3ED60DFC Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L5NSdFA/ X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gbiDWnrU9spO; Mon, 14 Nov 2022 16:18:12 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 4D7C860E07; Mon, 14 Nov 2022 16:18:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 4D7C860E07 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E3617C0033; Mon, 14 Nov 2022 16:18:11 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id B71B8C002D for ; Mon, 14 Nov 2022 16:18:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8D31540317 for ; Mon, 14 Nov 2022 16:18:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8D31540317 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L5NSdFA/ X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zh9TninuTbrl for ; Mon, 14 Nov 2022 16:18:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org BB4DF4160A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id BB4DF4160A for ; Mon, 14 Nov 2022 16:18:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668442688; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hxvXy2EJc9Qd0pwiulW/osrJ5/Pv/7EvjVfPKadkE/I=; b=L5NSdFA/m8vha7RHJPLdUF6dTe+i4vYrH7OGx+kVbO2wys68lk/osThEzqN/66Hp0LA9AM qveXH2y21kMEK3EQVHqAnYbLKbtyOLTDY/eChpTvn2wFzVtZ2QYWCRwyj7ZBJB0VTirolp A0QI1n31OSA/FaUT7DJqWvPY314ZV0E= Received: from mail-il1-f197.google.com (mail-il1-f197.google.com [209.85.166.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-269-jy9UCdipOzuLDQtrknzYRg-1; Mon, 14 Nov 2022 11:18:05 -0500 X-MC-Unique: jy9UCdipOzuLDQtrknzYRg-1 Received: by mail-il1-f197.google.com with SMTP id l4-20020a056e021aa400b00300ad9535c8so9557859ilv.1 for ; Mon, 14 Nov 2022 08:18:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hxvXy2EJc9Qd0pwiulW/osrJ5/Pv/7EvjVfPKadkE/I=; b=ozk/jNs24CtrJIL5OTB2n5+NseqluUuK2nOMgQ2jIvquHbJlIf4am9Bg8J4eiMQu01 GMoywthq5je5Enc9iwnOzD4l/2sVbxux9OIxnaoUhTj095IDgpwB04IlQY/rX2ufaMBl ecSangPP3uBMzR84FIeJoWa80Voy6tUXCNssa37ZyLemWYqkJN0LILhsrWPJrXVqLbmS +1xj2whLSrbmuno7IRDflVxBEAoFCm3fkXR5gZtzoiFmS0PuQmz5FZ7xO+WQJNhkTAG+ PqF2MBZDlMSv54v7cuCjKxIM1c4qoxg2fEwiT6nPz8CYMMqodrlqv/y5QJagJR8m1J16 Ebtw== X-Gm-Message-State: ANoB5pnP1HICc30W8JVSgCDRO2vXkJmK7smqxOAK7FNByx4hDeCA5aUX 57plwmGxWrabC9AuhbqqwHFDv8l+RW1WYFdkZpdF9cgJ4rgmzPA7/ZAvFK24jmuF7JZ3oVA49gc ioJGs7ccM1aDW5tqwlFZsjR+WjkHlvjc3dg/8I52zGQ== X-Received: by 2002:a05:6e02:1d06:b0:300:f4a5:7266 with SMTP id i6-20020a056e021d0600b00300f4a57266mr6579915ila.273.1668442684533; Mon, 14 Nov 2022 08:18:04 -0800 (PST) X-Received: by 2002:a05:6e02:1d06:b0:300:f4a5:7266 with SMTP id i6-20020a056e021d0600b00300f4a57266mr6579906ila.273.1668442684300; Mon, 14 Nov 2022 08:18:04 -0800 (PST) Received: from redhat.com ([38.15.36.239]) by smtp.gmail.com with ESMTPSA id f19-20020a05660215d300b006ddf70e3ce8sm2290113iow.7.2022.11.14.08.18.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Nov 2022 08:18:03 -0800 (PST) Date: Mon, 14 Nov 2022 09:18:02 -0700 From: Alex Williamson To: "leohou@tom.com" Subject: Re: vIOMMU&IOMMU gIOVA to HPA mapping Message-ID: <20221114091802.6be5099e.alex.williamson@redhat.com> In-Reply-To: <2022111420314945171759@tom.com> References: <2022111420314945171759@tom.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.34; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: iommu , qemu-arm , virtualization X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" X-TUID: 3rx9SE10h1MA T24gTW9uLCAxNCBOb3YgMjAyMiAyMDozMTo0OSArMDgwMAoibGVvaG91QHRvbS5jb20iIDxsZW9o b3VAdG9tLmNvbT4gd3JvdGU6Cgo+IEhp77yMCj4gICAgICBIZXJlIGlzIG15IGFwcGxpY2F0aW9u IHNjZW5hcmlv77yaCj4gMS4gVGhlIE5JQyAoTmV0d29yayBJbmZvcm1hdGlvbiBDZW50ZXIpIHBh c3NlcyB0aHJvdWdoIHRvIHRoZSBWTShWaXJ0dWFsIE1hY2hpbmUpOwo+IDIuIFRoZSBWTSB1c2Vz IHRoZSB1c2VyIG1vZGUgZHJpdmVyIERQREs7Cj4gCj4gUXVlc3Rpb246Cj4gMS4gdklPTU1VIG1h aW50YWlucyB0aGUgbWFwcGluZyBnSU9WQS0+Z1BBLCBXaGVuIGRvIHlvdSB1c2UgdGhpcyBnUEEg PwoKUUVNVSBpbiB0aGUgaG9zdCBkZXJpdmVzIHRoZSBoVkEgZnJvbSB0aGUgZ1BBLiAgVGhlIHZJ T01NVSBkcml2ZXIgaW4KUUVNVSBpcyB0cmlnZ2VyaW5nIHRoZSBnSU9WQSB0byBoVkEgbWFwcGlu ZyB0aHJvdWdoIHZmaW8gaW4gdGhlIGhvc3QuCgo+IDIuIFBoeXNpY2FsIElPTU1VIG1haW50YWlu cyB0aGUgR0lPVkEtPkhQQSBtYXBwaW5nID8gIElmIHNvLCBieSB3aGF0IG1lYW5zIChnSU9WQSAt PiBIUEEpIG1hcHBpbmcgPwoKQXMgYWJvdmUsIHRoZSB2SU9NTVUgaW4gdGhlIGd1ZXN0IHByb3Zp ZGVzIGdJT1ZBIC0+IGdQQSwgaW4gUUVNVSB3ZSBkbwp0aGUgZ1BBIC0+IGhWQSwgdGhlbiB2Zmlv IGluIHRoZSBob3N0IGtlcm5lbCBwZXJmb3JtcyBoVkEgLT4gaFBBIHZpYQpwYWdlIHBpbm5pbmcu Cgo+IDMuIFdoYXQgZG9lcyBRRU1VIGRvIGluIE5JQyBwYXNzLXRocm91Z2ggYWRkcmVzcyB0cmFu c2xhdGlvbiDvvJ8KClRoZSBndWVzdCB2aXNpYmxlIHZJT01NVSB0cmlnZ2VycyBNZW1vcnlMaXN0 ZW5lciBub3RpZmljYXRpb25zIGluIFFFTVUKZm9yIHRoZSBkZXZpY2UgYWRkcmVzcyBzcGFjZSwg d2hpY2ggaW5zZXJ0IGFuZCByZW1vdmVzIG1hcHBpbmdzIHRvIHRoZQp2ZmlvIGxheWVyIGJlbG93 IGl0LiAgVGhhbmtzLAoKQWxleAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KVmlydHVhbGl6YXRpb24gbWFpbGluZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxp c3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL3ZpcnR1YWxpemF0aW9u