From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5472C282EC for ; Fri, 14 Mar 2025 14:26:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tt5zH-0000gv-4g; Fri, 14 Mar 2025 10:26:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tt5zE-0000gG-U3 for qemu-devel@nongnu.org; Fri, 14 Mar 2025 10:26:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tt5z8-0006Jc-CD for qemu-devel@nongnu.org; Fri, 14 Mar 2025 10:26:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741962358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=P5UK8gpI79GcpRcsv556Da+0iwrWzl6UOzBqOaTbV5g=; b=I6QEQoan8y5zCZ0e2wgcuCWtrxouvz4YHdCz/4ihzd6zqrFtqWyn9LnY8wecDxKgmW5D39 Z+FresT3v/iDwdYsbsrOcSbjYMAcwJMMsiXsx5k6W2Yn/dVsdX2L+kf1aO1reT/IqM6J3s aQZCq+7mc+XXbioHbu8yDHTIhUq7Ecc= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-599-MoJBQokAPPOhQ1FAKh-oYQ-1; Fri, 14 Mar 2025 10:25:56 -0400 X-MC-Unique: MoJBQokAPPOhQ1FAKh-oYQ-1 X-Mimecast-MFC-AGG-ID: MoJBQokAPPOhQ1FAKh-oYQ_1741962356 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-43ced8c2eb7so16976705e9.1 for ; Fri, 14 Mar 2025 07:25:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741962356; x=1742567156; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=P5UK8gpI79GcpRcsv556Da+0iwrWzl6UOzBqOaTbV5g=; b=K4UsbYEtn8EwU2FyJbu4nAC4PSECP/NzS0s8S/0NcxrUmkLPdmbHPRNVHLzLNPguDn t1WpSNKgwCOkBsVqFb9OxFQHNTvE0ApnObwDjZWoUqLMu4pZ5/IWUPSklKfQNt+DYDML RUB0UxCLgTtknXN1QjwVcP10qO1+UVAewaeThM25rCnIps0cmvbFDufD9EYUSnoxAC13 YyeDUc3zzgPY2mwop3IPaq2Sp1JMcmbpsXzH696DKNBdojbZPz74W+lCo2GzYm6SEmgE o5zcvpkS/QADp8aHq7EGhueTUy+/6fer/qG5mIxP874LW8iF7t/7VNGiOlxoEt9M9rtf CeXA== X-Forwarded-Encrypted: i=1; AJvYcCV+LYyjPyZ6DcAw5yVjDO6dXjJ9s18R7N2y3DPi35Rx6bFk4YBFe4/TN3Rfp8g8BaRiE5G0zNgbCvr1@nongnu.org X-Gm-Message-State: AOJu0YywqsL1pL9ysnvILaqWpYOAx3bw+3MDtbmX9MwH3hVGih/xykc2 2d7KsllrEl1gQGwLZOetqrIaI+nrWPT7wvLYbmtG5TdCG1J0nB9gTK0o3TXLFLQP+kwbXU4QLH0 iE4oIc8dP4BKMm8+X/kWk+bi7iIpY7Wmj6GYZAqWrmLGY3hxje9YL X-Gm-Gg: ASbGncsSrBaj4qN81wewc7OsTaWRNsGWiR2AUSnnfEguGkt+SUlD3qPVw9PhLPT2YrS /6zWowQbVkMVJ2bKYiYi7tLMcv6XDMU86rnE5UBu2udjETg67pgxNxZWfJGIM/MlG+8OxoU7gPi PHjOhi/g17tieZNUlTINWJHBEo6Pr8/zrIBTa56lV8+V7Ahs4IDPJSnUJDZxpbM+r40Io2DAKCh 9p176JDy7tIsQdhmvdlEtK5wQT1Z2deS6AnfG8Oje0PACX66vi/ql9OngUdHgPplsVbjTRmlXYF j/1E6JkJBHHEJRCw+J/+a/JQlQf01HryNCrvmo+cDYMbTYeO89MzZg== X-Received: by 2002:a05:600c:1c85:b0:43c:fb5b:84d8 with SMTP id 5b1f17b1804b1-43d1ec81231mr39063865e9.16.1741962355606; Fri, 14 Mar 2025 07:25:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEGzjJFDFlvfAbEDl2xU6blmoVM5crKWKP7FGGnTIg8aQKlGz0ywcBynDzlNqmbraiuDdFVhQ== X-Received: by 2002:a05:600c:1c85:b0:43c:fb5b:84d8 with SMTP id 5b1f17b1804b1-43d1ec81231mr39063445e9.16.1741962355165; Fri, 14 Mar 2025 07:25:55 -0700 (PDT) Received: from ?IPV6:2a01:e0a:280:24f0:9db0:474c:ff43:9f5c? ([2a01:e0a:280:24f0:9db0:474c:ff43:9f5c]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43d1ffc3e67sm18915255e9.17.2025.03.14.07.25.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 14 Mar 2025 07:25:54 -0700 (PDT) Message-ID: <5910cf36-ca6f-489b-a7f6-b720733f16e1@redhat.com> Date: Fri, 14 Mar 2025 15:25:53 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v8 00/28] vfio-user client To: John Levon , qemu-devel@nongnu.org Cc: Jason Herne , Thanos Makatos , Halil Pasic , =?UTF-8?Q?Daniel_P=2E_Berrang=C3=A9?= , Eric Farman , Tony Krowiak , Thomas Huth , qemu-s390x@nongnu.org, Matthew Rosato , Paolo Bonzini , =?UTF-8?Q?Marc-Andr=C3=A9_Lureau?= , Stefano Garzarella , Alex Williamson , David Hildenbrand , Peter Xu , =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= , "Michael S. Tsirkin" , Steven Sistare References: <20250219144858.266455-1-john.levon@nutanix.com> Content-Language: en-US, fr From: =?UTF-8?Q?C=C3=A9dric_Le_Goater?= Autocrypt: addr=clg@redhat.com; keydata= xsFNBFu8o3UBEADP+oJVJaWm5vzZa/iLgpBAuzxSmNYhURZH+guITvSySk30YWfLYGBWQgeo 8NzNXBY3cH7JX3/a0jzmhDc0U61qFxVgrPqs1PQOjp7yRSFuDAnjtRqNvWkvlnRWLFq4+U5t yzYe4SFMjFb6Oc0xkQmaK2flmiJNnnxPttYwKBPd98WfXMmjwAv7QfwW+OL3VlTPADgzkcqj 53bfZ4VblAQrq6Ctbtu7JuUGAxSIL3XqeQlAwwLTfFGrmpY7MroE7n9Rl+hy/kuIrb/TO8n0 ZxYXvvhT7OmRKvbYuc5Jze6o7op/bJHlufY+AquYQ4dPxjPPVUT/DLiUYJ3oVBWFYNbzfOrV RxEwNuRbycttMiZWxgflsQoHF06q/2l4ttS3zsV4TDZudMq0TbCH/uJFPFsbHUN91qwwaN/+ gy1j7o6aWMz+Ib3O9dK2M/j/O/Ube95mdCqN4N/uSnDlca3YDEWrV9jO1mUS/ndOkjxa34ia 70FjwiSQAsyIwqbRO3CGmiOJqDa9qNvd2TJgAaS2WCw/TlBALjVQ7AyoPEoBPj31K74Wc4GS Rm+FSch32ei61yFu6ACdZ12i5Edt+To+hkElzjt6db/UgRUeKfzlMB7PodK7o8NBD8outJGS tsL2GRX24QvvBuusJdMiLGpNz3uqyqwzC5w0Fd34E6G94806fwARAQABzSJDw6lkcmljIExl IEdvYXRlciA8Y2xnQHJlZGhhdC5jb20+wsGRBBMBCAA7FiEEoPZlSPBIlev+awtgUaNDx8/7 7KEFAmTLlVECGwMFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQUaNDx8/77KG0eg// S0zIzTcxkrwJ/9XgdcvVTnXLVF9V4/tZPfB7sCp8rpDCEseU6O0TkOVFoGWM39sEMiQBSvyY lHrP7p7E/JYQNNLh441MfaX8RJ5Ul3btluLapm8oHp/vbHKV2IhLcpNCfAqaQKdfk8yazYhh EdxTBlzxPcu+78uE5fF4wusmtutK0JG0sAgq0mHFZX7qKG6LIbdLdaQalZ8CCFMKUhLptW71 xe+aNrn7hScBoOj2kTDRgf9CE7svmjGToJzUxgeh9mIkxAxTu7XU+8lmL28j2L5uNuDOq9vl hM30OT+pfHmyPLtLK8+GXfFDxjea5hZLF+2yolE/ATQFt9AmOmXC+YayrcO2ZvdnKExZS1o8 VUKpZgRnkwMUUReaF/mTauRQGLuS4lDcI4DrARPyLGNbvYlpmJWnGRWCDguQ/LBPpbG7djoy k3NlvoeA757c4DgCzggViqLm0Bae320qEc6z9o0X0ePqSU2f7vcuWN49Uhox5kM5L86DzjEQ RHXndoJkeL8LmHx8DM+kx4aZt0zVfCHwmKTkSTQoAQakLpLte7tWXIio9ZKhUGPv/eHxXEoS 0rOOAZ6np1U/xNR82QbF9qr9TrTVI3GtVe7Vxmff+qoSAxJiZQCo5kt0YlWwti2fFI4xvkOi V7lyhOA3+/3oRKpZYQ86Frlo61HU3r6d9wzOwU0EW7yjdQEQALyDNNMw/08/fsyWEWjfqVhW pOOrX2h+z4q0lOHkjxi/FRIRLfXeZjFfNQNLSoL8j1y2rQOs1j1g+NV3K5hrZYYcMs0xhmrZ KXAHjjDx7FW3sG3jcGjFW5Xk4olTrZwFsZVUcP8XZlArLmkAX3UyrrXEWPSBJCXxDIW1hzwp bV/nVbo/K9XBptT/wPd+RPiOTIIRptjypGY+S23HYBDND3mtfTz/uY0Jytaio9GETj+fFis6 TxFjjbZNUxKpwftu/4RimZ7qL+uM1rG1lLWc9SPtFxRQ8uLvLOUFB1AqHixBcx7LIXSKZEFU CSLB2AE4wXQkJbApye48qnZ09zc929df5gU6hjgqV9Gk1rIfHxvTsYltA1jWalySEScmr0iS YBZjw8Nbd7SxeomAxzBv2l1Fk8fPzR7M616dtb3Z3HLjyvwAwxtfGD7VnvINPbzyibbe9c6g LxYCr23c2Ry0UfFXh6UKD83d5ybqnXrEJ5n/t1+TLGCYGzF2erVYGkQrReJe8Mld3iGVldB7 JhuAU1+d88NS3aBpNF6TbGXqlXGF6Yua6n1cOY2Yb4lO/mDKgjXd3aviqlwVlodC8AwI0Sdu jWryzL5/AGEU2sIDQCHuv1QgzmKwhE58d475KdVX/3Vt5I9kTXpvEpfW18TjlFkdHGESM/Jx IqVsqvhAJkalABEBAAHCwV8EGAECAAkFAlu8o3UCGwwACgkQUaNDx8/77KEhwg//WqVopd5k 8hQb9VVdk6RQOCTfo6wHhEqgjbXQGlaxKHoXywEQBi8eULbeMQf5l4+tHJWBxswQ93IHBQjK yKyNr4FXseUI5O20XVNYDJZUrhA4yn0e/Af0IX25d94HXQ5sMTWr1qlSK6Zu79lbH3R57w9j hQm9emQEp785ui3A5U2Lqp6nWYWXz0eUZ0Tad2zC71Gg9VazU9MXyWn749s0nXbVLcLS0yop s302Gf3ZmtgfXTX/W+M25hiVRRKCH88yr6it+OMJBUndQVAA/fE9hYom6t/zqA248j0QAV/p LHH3hSirE1mv+7jpQnhMvatrwUpeXrOiEw1nHzWCqOJUZ4SY+HmGFW0YirWV2mYKoaGO2YBU wYF7O9TI3GEEgRMBIRT98fHa0NPwtlTktVISl73LpgVscdW8yg9Gc82oe8FzU1uHjU8b10lU XOMHpqDDEV9//r4ZhkKZ9C4O+YZcTFu+mvAY3GlqivBNkmYsHYSlFsbxc37E1HpTEaSWsGfA HQoPn9qrDJgsgcbBVc1gkUT6hnxShKPp4PlsZVMNjvPAnr5TEBgHkk54HQRhhwcYv1T2QumQ izDiU6iOrUzBThaMhZO3i927SG2DwWDVzZltKrCMD1aMPvb3NU8FOYRhNmIFR3fcalYr+9gD uVKe8BVz4atMOoktmt0GWTOC8P4= In-Reply-To: <20250219144858.266455-1-john.levon@nutanix.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org John, +Steven, for the Live upsate series. On 2/19/25 15:48, John Levon wrote: > This is the 8th revision of the vfio-user client implementation. The vfio-user > protocol allows for implementing (PCI) devices in another userspace process; > SPDK is one example, which includes a virtual NVMe implementation. > > The vfio-user framework consists of 3 parts: > 1) The VFIO user protocol specification. > 2) A client - the VFIO device in QEMU that encapsulates VFIO messages > and sends them to the server. > 3) A server - a remote process that emulates a device. > > This patchset implements parts 1 and 2. > > It has been tested against libvfio-user test servers as well as SPDK. > > Thanks for previous reviews & comments. This series is going in the right direction. Thanks for all the changes. However, I will postpone my review for the reasons below. VFIO initial commit in 2012 was a single file adding support for a "vfio-pci" device. Since, the subsystem has changed a lot and it grew to +16K line of code. Over the years, we've had numerous additions, VFIO platform devices, s390x VFIO AP and CCW devices, migration support with dirty tracking, support for different host IOMMU backend devices, multifd, etc. All these additions never took the opportunity to introduce a VFIO interface for external usage and PPC, s390x, migration, backends subsystems happily and randomly peek and poke into it. The subsystem is now a happy jumble of disparate functions, with weak organization and names. The vfio-common.h header file is good example of the result. It's time for a global code reshuffling. I'll take care of it (and I've already started) and this means nothing new (fixes are OK) will be merged before this is done. I hope I have made enough progress before the start of the QEMU 10.1 cycle. There won't be any functional changes, but there will be new files and new function names, so it's probably a real earthquake for your series. Thanks, C. > > Changes since v7: > > - split up pci patches for easier reviewing > - fixed lots of device ops error handling > - vfio-user code now in hw/vfio-user > - improved commit messages > - various other small cleanups > > Jagannathan Raman (20): > vfio/container: pass MemoryRegion to DMA operations > vfio: add region cache > vfio: split out VFIOKernelPCIDevice > vfio: add device IO ops vector > vfio-user: add vfio-user class and container > vfio-user: connect vfio proxy to remote server > vfio-user: implement message receive infrastructure > vfio-user: implement message send infrastructure > vfio-user: implement VFIO_USER_DEVICE_GET_INFO > vfio-user: implement VFIO_USER_DEVICE_GET_REGION_INFO > vfio-user: implement VFIO_USER_REGION_READ/WRITE > vfio-user: set up PCI in vfio_user_pci_realize() > vfio-user: implement VFIO_USER_DEVICE_GET/SET_IRQ* > vfio-user: forward MSI-X PBA BAR accesses to server > vfio-user: set up container access to the proxy > vfio-user: implement VFIO_USER_DEVICE_RESET > vfio-user: implement VFIO_USER_DMA_READ/WRITE > vfio-user: add 'no-direct-dma' option > vfio-user: add 'x-msg-timeout' option > vfio-user: add coalesced posted writes > > John Levon (7): > vfio/container: pass listener_begin/commit callbacks > vfio/container: support VFIO_DMA_UNMAP_FLAG_ALL > vfio: add vfio_attach_device_by_iommu_type() > vfio: add vfio_prepare_device() > vfio: refactor out vfio_interrupt_setup() > vfio: refactor out vfio_pci_config_setup() > vfio-user: implement VFIO_USER_DMA_MAP/UNMAP > > Thanos Makatos (1): > vfio-user: introduce vfio-user protocol specification > > MAINTAINERS | 10 +- > docs/devel/index-internals.rst | 1 + > docs/devel/vfio-user.rst | 1522 ++++++++++++++++++++++ > hw/meson.build | 1 + > hw/vfio-user/common.c | 1702 +++++++++++++++++++++++++ > hw/vfio-user/common.h | 123 ++ > hw/vfio-user/container.c | 358 ++++++ > hw/vfio-user/container.h | 24 + > hw/vfio-user/meson.build | 10 + > hw/vfio-user/pci.c | 443 +++++++ > hw/vfio-user/protocol.h | 243 ++++ > hw/vfio-user/trace-events | 18 + > hw/vfio-user/trace.h | 1 + > hw/vfio/ap.c | 4 +- > hw/vfio/ccw.c | 9 +- > hw/vfio/common.c | 137 +- > hw/vfio/container-base.c | 8 +- > hw/vfio/container.c | 78 +- > hw/vfio/helpers.c | 185 ++- > hw/vfio/igd.c | 8 +- > hw/vfio/iommufd.c | 31 +- > hw/vfio/pci.c | 591 +++++---- > hw/vfio/pci.h | 34 +- > hw/vfio/platform.c | 4 +- > hw/virtio/vhost-vdpa.c | 2 +- > include/exec/memory.h | 4 +- > include/hw/vfio/vfio-common.h | 45 +- > include/hw/vfio/vfio-container-base.h | 11 +- > meson.build | 1 + > meson_options.txt | 2 + > scripts/meson-buildoptions.sh | 4 + > system/memory.c | 7 +- > 32 files changed, 5281 insertions(+), 340 deletions(-) > create mode 100644 docs/devel/vfio-user.rst > create mode 100644 hw/vfio-user/common.c > create mode 100644 hw/vfio-user/common.h > create mode 100644 hw/vfio-user/container.c > create mode 100644 hw/vfio-user/container.h > create mode 100644 hw/vfio-user/meson.build > create mode 100644 hw/vfio-user/pci.c > create mode 100644 hw/vfio-user/protocol.h > create mode 100644 hw/vfio-user/trace-events > create mode 100644 hw/vfio-user/trace.h >