From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57810CA5FFF for ; Wed, 7 Oct 2026 07:33:21 +0000 (UTC) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B714342830; Wed, 7 Oct 2026 09:33:01 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 6324E427C7 for ; Wed, 7 Oct 2026 09:33:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1791358380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SZg1pCfQJ/5LVIl76ECo7+scNMokZ5WhjGIXJL24IEk=; b=Jt6KR6G41CK7/vzgJ3OClIaTHiC0pH35WFOccLJuJ/nRLZBuypTM762nEDMRUA75M022fs MNGaanLoCIMtjjtNS78g2MrfAGyP3TE0nnBLzOZkc7a/TMOwPVFqfHA7YlgJmSkUcyVcrM cm6+8aieiYs2wNnvvM3UwESgeuN80v0= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-18-BOW1OX2VP82wYAYR8yX4zQ-1; Wed, 7 Oct 2026 07:32:56 +0000 X-MC-Unique: BOW1OX2VP82wYAYR8yX4zQ-1 X-Mimecast-MFC-AGG-ID: BOW1OX2VP82wYAYR8yX4zQ_1791358375 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1557F1955DE2; Wed, 7 Oct 2026 07:32:55 +0000 (UTC) Received: from dmarchan.redhat.corp (headnet03.pony-001.prod.iad2.dc.redhat.com [10.2.32.114]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7BAFE1956095; Wed, 7 Oct 2026 07:32:53 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: Anatoly Burakov , Dimon Zhao , Leon Yu , Sam Chen Subject: [PATCH v19 11/26] net/nbl: do not use VFIO group bind API Date: Wed, 7 Oct 2026 09:31:49 +0200 Message-ID: <20261007073206.567001-12-david.marchand@redhat.com> In-Reply-To: <20261007073206.567001-1-david.marchand@redhat.com> References: <20261007073206.567001-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: elPYba1mgR9_357o1W7j7QUUUG26ZK26xKT6xcwx8us_1791358375 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Anatoly Burakov The NBL driver currently uses group bind API, but it is using it only to get group fd and nothing else. In context of NBL driver, this is the only usage of VFIO API's in the driver, and it is not necessary to use it for what NBL driver is trying to accomplish. Use a direct `open()` call instead, store the group fd in common structure, and find group fd's opened earlier to prevent us from opening the same group fd twice. Signed-off-by: Anatoly Burakov --- drivers/net/nbl/nbl_common/nbl_userdev.c | 63 +++++++++++++++++------ drivers/net/nbl/nbl_common/nbl_userdev.h | 1 + drivers/net/nbl/nbl_include/nbl_include.h | 1 + 3 files changed, 50 insertions(+), 15 deletions(-) diff --git a/drivers/net/nbl/nbl_common/nbl_userdev.c b/drivers/net/nbl/nbl_common/nbl_userdev.c index 261bb8a4c1..91b7c8006f 100644 --- a/drivers/net/nbl/nbl_common/nbl_userdev.c +++ b/drivers/net/nbl/nbl_common/nbl_userdev.c @@ -388,6 +388,28 @@ nbl_userdev_mem_event_callback(enum rte_mem_event type, const void *addr, size_t } } +static int +nbl_open_group_fd(int iommu_group_num) +{ + char path[PATH_MAX]; + + snprintf(path, sizeof(path), DEV_VFIO_GROUP_FMT, iommu_group_num); + return open(path, O_RDWR); +} + +static int +nbl_find_group_fd(int iommu_group_num) +{ + struct nbl_adapter *adapter; + + TAILQ_FOREACH(adapter, &nbl_adapter_list, next) { + if (adapter->common.iommu_group_num == iommu_group_num) + return adapter->common.groupfd; + } + + return -1; +} + static int nbl_mdev_map_device(struct nbl_adapter *adapter) { const struct rte_pci_device *pci_dev = adapter->pci_dev; @@ -401,6 +423,7 @@ static int nbl_mdev_map_device(struct nbl_adapter *adapter) u64 dma_limit = NBL_USERDEV_DMA_LIMIT; int ret, container_create = 0, container_set = 0; int vfio_group_fd, container = nbl_default_container; + bool new_group = false; rte_pci_device_name(&pci_dev->addr, dev_name, RTE_DEV_NAME_MAX_LEN); snprintf(pathname, sizeof(pathname), @@ -425,11 +448,18 @@ static int nbl_mdev_map_device(struct nbl_adapter *adapter) } NBL_LOG(DEBUG, "nbl vfio container %d", container); - vfio_group_fd = dev_vfio_container_group_bind(container, common->iommu_group_num); + rte_mcfg_mem_read_lock(); + vfio_group_fd = nbl_find_group_fd(common->iommu_group_num); + rte_mcfg_mem_read_unlock(); + if (vfio_group_fd < 0) { + vfio_group_fd = nbl_open_group_fd(common->iommu_group_num); + new_group = true; + } if (vfio_group_fd < 0) { NBL_LOG(ERR, "nbl vfio group bind failed, %d", vfio_group_fd); goto free_container; } + common->groupfd = vfio_group_fd; /* check if the group is viable */ ret = ioctl(vfio_group_fd, VFIO_GROUP_GET_STATUS, &group_status); @@ -518,13 +548,13 @@ static int nbl_mdev_map_device(struct nbl_adapter *adapter) close_fd: close(common->devfd); unregister_mem_event: - if (nbl_group_count == 1) { + if (container_set && nbl_group_count == 1) { rte_mcfg_mem_read_lock(); rte_mem_event_callback_unregister(NBL_USERDEV_EVENT_CLB_NAME, NULL); rte_mcfg_mem_read_unlock(); } free_dma_map: - if (nbl_group_count == 1) { + if (container_set && nbl_group_count == 1) { rte_mcfg_mem_read_lock(); nbl_userdev_dma_free(); rte_mcfg_mem_read_unlock(); @@ -535,8 +565,8 @@ static int nbl_mdev_map_device(struct nbl_adapter *adapter) nbl_group_count--; } free_group: - close(vfio_group_fd); - dev_vfio_clear_group(vfio_group_fd); + if (new_group) + close(vfio_group_fd); free_container: if (container_create) dev_vfio_container_destroy(container); @@ -547,21 +577,24 @@ static int nbl_mdev_unmap_device(struct nbl_adapter *adapter) { struct nbl_common_info *common = &adapter->common; int vfio_group_fd, ret; + bool last_group_user; rte_mcfg_mem_read_lock(); TAILQ_REMOVE(&nbl_adapter_list, adapter, next); close(common->devfd); - vfio_group_fd = dev_vfio_container_group_bind(nbl_default_container, - common->iommu_group_num); + common->devfd = -1; + vfio_group_fd = common->groupfd; + last_group_user = nbl_find_group_fd(common->iommu_group_num) < 0; NBL_LOG(DEBUG, "close vfio_group_fd %d", vfio_group_fd); - ret = ioctl(vfio_group_fd, VFIO_GROUP_UNSET_CONTAINER, &nbl_default_container); - if (ret) - NBL_LOG(ERR, "unset container, error %i (%s) %d", - errno, strerror(errno), ret); - nbl_group_count--; - ret = dev_vfio_container_group_unbind(nbl_default_container, common->iommu_group_num); - if (ret) - NBL_LOG(ERR, "vfio container group unbind failed %d", ret); + if (last_group_user) { + ret = ioctl(vfio_group_fd, VFIO_GROUP_UNSET_CONTAINER, &nbl_default_container); + if (ret) + NBL_LOG(ERR, "unset container, error %i (%s) %d", + errno, strerror(errno), ret); + nbl_group_count--; + close(vfio_group_fd); + common->groupfd = -1; + } if (!nbl_group_count) { rte_mem_event_callback_unregister(NBL_USERDEV_EVENT_CLB_NAME, NULL); nbl_userdev_dma_free(); diff --git a/drivers/net/nbl/nbl_common/nbl_userdev.h b/drivers/net/nbl/nbl_common/nbl_userdev.h index 2221e19c67..1ebc86baf6 100644 --- a/drivers/net/nbl/nbl_common/nbl_userdev.h +++ b/drivers/net/nbl/nbl_common/nbl_userdev.h @@ -11,6 +11,7 @@ #define NBL_USERDEV_INIT_COMMON(common) do { \ typeof(common) _comm = (common); \ _comm->devfd = -1; \ + _comm->groupfd = -1; \ _comm->eventfd = -1; \ _comm->specific_dma = false; \ _comm->dma_set_msb = false; \ diff --git a/drivers/net/nbl/nbl_include/nbl_include.h b/drivers/net/nbl/nbl_include/nbl_include.h index eeae6a3301..ba99b9f8e7 100644 --- a/drivers/net/nbl/nbl_include/nbl_include.h +++ b/drivers/net/nbl/nbl_include/nbl_include.h @@ -132,6 +132,7 @@ struct nbl_common_info { u16 vsi_id; u16 instance_id; int devfd; + int groupfd; int eventfd; int ifindex; int iommu_group_num; -- 2.54.0