From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 596D4C25B47 for ; Fri, 27 Oct 2023 09:27:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qwJ7q-0004tZ-Gd; Fri, 27 Oct 2023 05:27:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qwJ7p-0004sf-1R for qemu-devel@nongnu.org; Fri, 27 Oct 2023 05:27:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qwJ7j-0008Nv-K1 for qemu-devel@nongnu.org; Fri, 27 Oct 2023 05:27:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698398842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=punkCYH0axpZYQOb6DOaPzm9ju5UrXeUGN2H58b+tXU=; b=jEf1IPYniALu4yEYTMHTUI99iqpiKQuEa59zwvTh3uhWSyLt2b8ITh7zTFhx5BGbYzb6GD 831k+O72G59A4IKW0N97TkRU1lxhqh822qbFhRMfdeZoEZlYa51PAOq+logRDNLoWDkkam +iy7eQgW0+QATsJqjItff75kwE9WgLg= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-179-Om5-X0iVM56Q-QZRseiP0Q-1; Fri, 27 Oct 2023 05:27:21 -0400 X-MC-Unique: Om5-X0iVM56Q-QZRseiP0Q-1 Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-66d75988385so24371696d6.2 for ; Fri, 27 Oct 2023 02:27:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698398840; x=1699003640; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=punkCYH0axpZYQOb6DOaPzm9ju5UrXeUGN2H58b+tXU=; b=KtVPmxmm7nWGkepVpRhufmNajf9WrHJ7noZZGJIYly57I1XzDB1kZ5RC8J3qzW83My z0c6GSPz2XDqUTcv/Hi7cg8UKiPUfoa8fW3zwhCqGJx+kzIzQknx+UY6ZsiksPb2sAZq ktjEjiJLUVcPOxvyizRqNdupZ48QELN2iee9/qVBaoq2g8NUHSa2r5S32XpNtbHh/zqG BUMNESoGKUm3bFSDxaX1Sx33hPkOSmcV9kJq88ITg/bSBmPZlf+Nl08FYsWyVR8mbLg3 PfJD3sOK5mogBrtOQFEU2CymcHdOt4YhVlr3vCXrZY6uzAFppHCIRqBQXXldeGskNxZQ 5fUQ== X-Gm-Message-State: AOJu0YzCLz/r5T1JeHSt36rGcDoQpaBd7iXGRefzkKQNAbXQuFTi1FXo N6oq4vV9PFsFQJHkQS51z9bcUb+Q4sGVxI3Nswwx7pqWbHzJ5L5dt4di7f1vC8CJ1yoxGkzW28B ijvfKUAX5DkRyRlw= X-Received: by 2002:a0c:aa94:0:b0:66d:9104:a334 with SMTP id f20-20020a0caa94000000b0066d9104a334mr1611546qvb.32.1698398840333; Fri, 27 Oct 2023 02:27:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGOdMzPQco+m9q0Pvzi+oo8ZkGS07BC/93xiiG6jlLTRjowScfDv2v/rJJg/bOylTVsQlqL8Q== X-Received: by 2002:a0c:aa94:0:b0:66d:9104:a334 with SMTP id f20-20020a0caa94000000b0066d9104a334mr1611528qvb.32.1698398840022; Fri, 27 Oct 2023 02:27:20 -0700 (PDT) Received: from ?IPV6:2a01:e0a:9e2:9000:b215:e40:e078:4b27? ([2a01:e0a:9e2:9000:b215:e40:e078:4b27]) by smtp.gmail.com with ESMTPSA id ei6-20020ad45a06000000b0065b2167fd63sm460837qvb.65.2023.10.27.02.27.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 27 Oct 2023 02:27:19 -0700 (PDT) Message-ID: <1b7dea00-6bdb-45e9-aae5-45346a0da467@redhat.com> Date: Fri, 27 Oct 2023 11:27:15 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 03/37] vfio/container: Move spapr specific init/deinit into spapr.c Content-Language: en-US To: Zhenzhong Duan , qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Nicholas Piggin , Daniel Henrique Barboza , David Gibson , Harsh Prateek Bora , "open list:sPAPR (pseries)" References: <20231026103104.1686921-1-zhenzhong.duan@intel.com> <20231026103104.1686921-4-zhenzhong.duan@intel.com> From: =?UTF-8?Q?C=C3=A9dric_Le_Goater?= In-Reply-To: <20231026103104.1686921-4-zhenzhong.duan@intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org On 10/26/23 12:30, Zhenzhong Duan wrote: > Move spapr specific init/deinit code into spapr.c and wrap > them with vfio_spapr_container_init/deinit, this way footprint > of spapr is further reduced, vfio_prereg_listener could also > be made static. > > vfio_listener_release is unnecessary when prereg_listener is > moved out, so have it removed. > > No functional changes intended. > > Suggested-by: Cédric Le Goater > Signed-off-by: Zhenzhong Duan Reviewed-by: Cédric Le Goater Thanks, C. > --- > include/hw/vfio/vfio-common.h | 4 +- > hw/vfio/container.c | 82 +++++------------------------------ > hw/vfio/spapr.c | 80 +++++++++++++++++++++++++++++++++- > 3 files changed, 94 insertions(+), 72 deletions(-) > > diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h > index 0c3d390e8b..8d1d4f0a89 100644 > --- a/include/hw/vfio/vfio-common.h > +++ b/include/hw/vfio/vfio-common.h > @@ -225,11 +225,14 @@ int vfio_set_dirty_page_tracking(VFIOContainer *container, bool start); > int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *vbmap, > hwaddr iova, hwaddr size); > > +/* SPAPR specific */ > int vfio_container_add_section_window(VFIOContainer *container, > MemoryRegionSection *section, > Error **errp); > void vfio_container_del_section_window(VFIOContainer *container, > MemoryRegionSection *section); > +bool vfio_spapr_container_init(VFIOContainer *container, Error **errp); > +void vfio_spapr_container_deinit(VFIOContainer *container); > > void vfio_disable_irqindex(VFIODevice *vbasedev, int index); > void vfio_unmask_single_irqindex(VFIODevice *vbasedev, int index); > @@ -289,7 +292,6 @@ vfio_get_device_info_cap(struct vfio_device_info *info, uint16_t id); > struct vfio_info_cap_header * > vfio_get_cap(void *ptr, uint32_t cap_offset, uint16_t id); > #endif > -extern const MemoryListener vfio_prereg_listener; > > int vfio_spapr_create_window(VFIOContainer *container, > MemoryRegionSection *section, > diff --git a/hw/vfio/container.c b/hw/vfio/container.c > index 7a3f005d1b..204b244b11 100644 > --- a/hw/vfio/container.c > +++ b/hw/vfio/container.c > @@ -264,14 +264,6 @@ int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *vbmap, > return ret; > } > > -static void vfio_listener_release(VFIOContainer *container) > -{ > - memory_listener_unregister(&container->listener); > - if (container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU) { > - memory_listener_unregister(&container->prereg_listener); > - } > -} > - > static struct vfio_info_cap_header * > vfio_get_iommu_type1_info_cap(struct vfio_iommu_type1_info *info, uint16_t id) > { > @@ -612,69 +604,11 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, > case VFIO_SPAPR_TCE_v2_IOMMU: > case VFIO_SPAPR_TCE_IOMMU: > { > - struct vfio_iommu_spapr_tce_info info; > - bool v2 = container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU; > - > - /* > - * The host kernel code implementing VFIO_IOMMU_DISABLE is called > - * when container fd is closed so we do not call it explicitly > - * in this file. > - */ > - if (!v2) { > - ret = ioctl(fd, VFIO_IOMMU_ENABLE); > - if (ret) { > - error_setg_errno(errp, errno, "failed to enable container"); > - ret = -errno; > - goto enable_discards_exit; > - } > - } else { > - container->prereg_listener = vfio_prereg_listener; > - > - memory_listener_register(&container->prereg_listener, > - &address_space_memory); > - if (container->error) { > - memory_listener_unregister(&container->prereg_listener); > - ret = -1; > - error_propagate_prepend(errp, container->error, > - "RAM memory listener initialization failed: "); > - goto enable_discards_exit; > - } > - } > - > - info.argsz = sizeof(info); > - ret = ioctl(fd, VFIO_IOMMU_SPAPR_TCE_GET_INFO, &info); > + ret = vfio_spapr_container_init(container, errp); > if (ret) { > - error_setg_errno(errp, errno, > - "VFIO_IOMMU_SPAPR_TCE_GET_INFO failed"); > - ret = -errno; > - if (v2) { > - memory_listener_unregister(&container->prereg_listener); > - } > goto enable_discards_exit; > } > - > - if (v2) { > - container->pgsizes = info.ddw.pgsizes; > - /* > - * There is a default window in just created container. > - * To make region_add/del simpler, we better remove this > - * window now and let those iommu_listener callbacks > - * create/remove them when needed. > - */ > - ret = vfio_spapr_remove_window(container, info.dma32_window_start); > - if (ret) { > - error_setg_errno(errp, -ret, > - "failed to remove existing window"); > - goto enable_discards_exit; > - } > - } else { > - /* The default table uses 4K pages */ > - container->pgsizes = 0x1000; > - vfio_host_win_add(container, info.dma32_window_start, > - info.dma32_window_start + > - info.dma32_window_size - 1, > - 0x1000); > - } > + break; > } > } > > @@ -704,7 +638,11 @@ listener_release_exit: > QLIST_REMOVE(group, container_next); > QLIST_REMOVE(container, next); > vfio_kvm_device_del_group(group); > - vfio_listener_release(container); > + memory_listener_unregister(&container->listener); > + if (container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU || > + container->iommu_type == VFIO_SPAPR_TCE_IOMMU) { > + vfio_spapr_container_deinit(container); > + } > > enable_discards_exit: > vfio_ram_block_discard_disable(container, false); > @@ -734,7 +672,11 @@ static void vfio_disconnect_container(VFIOGroup *group) > * group. > */ > if (QLIST_EMPTY(&container->group_list)) { > - vfio_listener_release(container); > + memory_listener_unregister(&container->listener); > + if (container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU || > + container->iommu_type == VFIO_SPAPR_TCE_IOMMU) { > + vfio_spapr_container_deinit(container); > + } > } > > if (ioctl(group->fd, VFIO_GROUP_UNSET_CONTAINER, &container->fd)) { > diff --git a/hw/vfio/spapr.c b/hw/vfio/spapr.c > index 9a7517c042..fd0613dfde 100644 > --- a/hw/vfio/spapr.c > +++ b/hw/vfio/spapr.c > @@ -15,6 +15,7 @@ > #include > #endif > #include "sysemu/kvm.h" > +#include "exec/address-spaces.h" > > #include "hw/vfio/vfio-common.h" > #include "hw/hw.h" > @@ -139,7 +140,7 @@ static void vfio_prereg_listener_region_del(MemoryListener *listener, > trace_vfio_prereg_unregister(reg.vaddr, reg.size, ret ? -errno : 0); > } > > -const MemoryListener vfio_prereg_listener = { > +static const MemoryListener vfio_prereg_listener = { > .name = "vfio-pre-reg", > .region_add = vfio_prereg_listener_region_add, > .region_del = vfio_prereg_listener_region_del, > @@ -343,3 +344,80 @@ void vfio_container_del_section_window(VFIOContainer *container, > __func__, section->offset_within_address_space); > } > } > + > +bool vfio_spapr_container_init(VFIOContainer *container, Error **errp) > +{ > + > + struct vfio_iommu_spapr_tce_info info; > + bool v2 = container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU; > + int ret, fd = container->fd; > + > + /* > + * The host kernel code implementing VFIO_IOMMU_DISABLE is called > + * when container fd is closed so we do not call it explicitly > + * in this file. > + */ > + if (!v2) { > + ret = ioctl(fd, VFIO_IOMMU_ENABLE); > + if (ret) { > + error_setg_errno(errp, errno, "failed to enable container"); > + return -errno; > + } > + } else { > + container->prereg_listener = vfio_prereg_listener; > + > + memory_listener_register(&container->prereg_listener, > + &address_space_memory); > + if (container->error) { > + ret = -1; > + error_propagate_prepend(errp, container->error, > + "RAM memory listener initialization failed: "); > + goto listener_unregister_exit; > + } > + } > + > + info.argsz = sizeof(info); > + ret = ioctl(fd, VFIO_IOMMU_SPAPR_TCE_GET_INFO, &info); > + if (ret) { > + error_setg_errno(errp, errno, > + "VFIO_IOMMU_SPAPR_TCE_GET_INFO failed"); > + ret = -errno; > + goto listener_unregister_exit; > + } > + > + if (v2) { > + container->pgsizes = info.ddw.pgsizes; > + /* > + * There is a default window in just created container. > + * To make region_add/del simpler, we better remove this > + * window now and let those iommu_listener callbacks > + * create/remove them when needed. > + */ > + ret = vfio_spapr_remove_window(container, info.dma32_window_start); > + if (ret) { > + error_setg_errno(errp, -ret, > + "failed to remove existing window"); > + goto listener_unregister_exit; > + } > + } else { > + /* The default table uses 4K pages */ > + container->pgsizes = 0x1000; > + vfio_host_win_add(container, info.dma32_window_start, > + info.dma32_window_start + > + info.dma32_window_size - 1, > + 0x1000); > + } > + > +listener_unregister_exit: > + if (v2) { > + memory_listener_unregister(&container->prereg_listener); > + } > + return ret; > +} > + > +void vfio_spapr_container_deinit(VFIOContainer *container) > +{ > + if (container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU) { > + memory_listener_unregister(&container->prereg_listener); > + } > +}