From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by smtp.lore.kernel.org (Postfix) with ESMTP id 324EECDB46F for ; Mon, 22 Jun 2026 09:27:39 +0000 (UTC) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 702614067A; Mon, 22 Jun 2026 11:27:23 +0200 (CEST) Received: from cstnet.cn (smtp81.cstnet.cn [159.226.251.81]) by mails.dpdk.org (Postfix) with ESMTP id 121FB4042F for ; Mon, 22 Jun 2026 11:27:20 +0200 (CEST) Received: from localhost.localdomain (unknown [118.112.177.181]) by APP-03 (Coremail) with SMTP id rQCowAD32+L3_zhqrZiEFQ--.1958S2; Mon, 22 Jun 2026 17:27:19 +0800 (CST) From: liujie5@linkdatatechnology.com To: stephen@networkplumber.org Cc: dev@dpdk.org, Jie Liu Subject: [PATCH v5 21/23] common/sxe2: add callback for memory event handling Date: Mon, 22 Jun 2026 17:27:19 +0800 Message-ID: <20260622092719.3092151-1-liujie5@linkdatatechnology.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260622092324.3091185-1-liujie5@linkdatatechnology.com> References: <20260622092324.3091185-1-liujie5@linkdatatechnology.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CM-TRANSID: rQCowAD32+L3_zhqrZiEFQ--.1958S2 X-Coremail-Antispam: 1UD129KBjvJXoW3WFWxZF45Cr1fGw45JF4ruFg_yoW7ZFWrpF 48uFyYqFy5Ja17Ww13t3WDZ34rKwsYg3yDCFWxKasak3Wqqry8Zw4kKa43tr1rCrW3JF1v yFsIyFy5Gw48JrJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUkm14x267AKxVWUJVW8JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK02 1l84ACjcxK6xIIjxv20xvE14v26ryj6F1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26F4j 6r4UJwA2z4x0Y4vEx4A2jsIE14v26r4j6F4UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr 0_Cr1UM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj 6xIIjxv20xvE14v26r1q6rW5McIj6I8E87Iv67AKxVW8JVWxJwAm72CE4IkC6x0Yz7v_Jr 0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4kE6xkIj40Ew7xC 0wCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r 1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jrv_JF1lIxkGc2Ij 64vIr41lIxAIcVC0I7IYx2IY67AKxVW5JVW7JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Cr 0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JVWxJwCI 42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfU8KsjUUUUU X-Originating-IP: [118.112.177.181] X-CM-SenderInfo: xolxyxrhv6zxpqngt3pdwhux5qro0w31of0z/ X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jie Liu During memory hotplug events, the SXE2 driver needs to track memory segment layout changes to maintain internal DMA mappings. However, existing memseg walk functions (rte_memseg_walk) acquire memory locks and cannot be called from within memory event callbacks, leading to potential deadlocks. The implementation follows the standard rte_memseg_walk_t prototype, processing each memseg to update driver-specific data structures. Signed-off-by: Jie Liu --- drivers/common/sxe2/sxe2_common.c | 110 ++++++++++++++++++++++++++ drivers/common/sxe2/sxe2_common.h | 2 + drivers/common/sxe2/sxe2_ioctl_chnl.c | 2 +- 3 files changed, 113 insertions(+), 1 deletion(-) diff --git a/drivers/common/sxe2/sxe2_common.c b/drivers/common/sxe2/sxe2_common.c index c000a55cd0..5c5db85f29 100644 --- a/drivers/common/sxe2/sxe2_common.c +++ b/drivers/common/sxe2/sxe2_common.c @@ -196,6 +196,102 @@ static int32_t sxe2_parse_representor(const char *key, const char *value, void * PMD_LOG_INFO(COM, "representor arg %s: \"%s\".", key, value); +l_end: + return ret; +} +static int32_t sxe2_dma_mem_map(struct sxe2_common_device *cdev, + const void *addr, size_t len, bool do_map) +{ + struct rte_memseg_list *msl; + struct rte_memseg *ms; + size_t cur_len = 0; + int32_t ret = 0; + + msl = rte_mem_virt2memseg_list(addr); + if (msl == NULL) { + ret = -EINVAL; + PMD_LOG_ERR(COM, "Invalid virt addr=%p.", addr); + goto l_end; + } + + if ((uintptr_t)addr != RTE_ALIGN((uintptr_t)addr, msl->page_sz) || + (len != RTE_ALIGN(len, msl->page_sz))) { + ret = -EINVAL; + PMD_LOG_ERR(COM, "Addr=%p and len=%zu not align page size=%" PRIu64 ".", + addr, len, msl->page_sz); + goto l_end; + } + + /* memsegs are contiguous in memory */ + ms = rte_mem_virt2memseg(addr, msl); + while (cur_len < len) { + /* some memory segments may have invalid IOVA */ + if (ms->iova == RTE_BAD_IOVA) { + PMD_LOG_WARN(COM, "Memory segment at %p has bad IOVA, skipping.", + ms->addr); + goto next; + } + if (do_map) + sxe2_drv_dev_dma_map(cdev, ms->addr_64, + ms->iova, ms->len); + else + sxe2_drv_dev_dma_unmap(cdev, ms->iova); + +next: + cur_len += ms->len; + ++ms; + } + +l_end: + return ret; +} + +RTE_EXPORT_INTERNAL_SYMBOL(sxe2_common_mem_event_cb) +void +sxe2_common_mem_event_cb(enum rte_mem_event type, + const void *addr, size_t size, void *arg __rte_unused) +{ + struct sxe2_common_device *cdev = NULL; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + goto l_end; + + pthread_mutex_lock(&sxe2_common_devices_list_lock); + switch (type) { + case RTE_MEM_EVENT_FREE: + TAILQ_FOREACH(cdev, &sxe2_common_devices_list, next) + (void)sxe2_dma_mem_map(cdev, addr, size, 0); + break; + case RTE_MEM_EVENT_ALLOC: + TAILQ_FOREACH(cdev, &sxe2_common_devices_list, next) + (void)sxe2_dma_mem_map(cdev, addr, size, 1); + break; + default: + break; + } + pthread_mutex_unlock(&sxe2_common_devices_list_lock); +l_end: + return; +} + +static int32_t sxe2_memseg_walk_cb(const struct rte_memseg_list *msl, + const struct rte_memseg *ms, void *arg) +{ + struct sxe2_common_device *cdev = arg; + int32_t ret = 0; + + if (msl->external && !msl->heap) + goto l_end; + + if (ms->iova == RTE_BAD_IOVA) + goto l_end; + + ret = sxe2_drv_dev_dma_map(cdev, ms->addr_64, ms->iova, ms->len); + if (ret != 0) { + PMD_LOG_ERR(COM, "Fail to memseg dma map."); + goto l_end; + } + l_end: return ret; } @@ -220,6 +316,18 @@ static int32_t sxe2_common_device_setup(struct sxe2_common_device *cdev) goto l_close_dev; } + rte_mcfg_mem_read_lock(); + ret = rte_memseg_walk_thread_unsafe(sxe2_memseg_walk_cb, cdev); + if (ret) { + PMD_LOG_ERR(COM, "Fail to walk memseg, ret=%d", ret); + rte_mcfg_mem_read_unlock(); + goto l_close_dev; + } + rte_mcfg_mem_read_unlock(); + + (void)rte_mem_event_callback_register("SXE2_MEM_EVENT_CB", + sxe2_common_mem_event_cb, NULL); + goto l_end; l_close_dev: @@ -251,6 +359,7 @@ static struct sxe2_common_device *sxe2_common_device_alloc( } cdev->dev = rte_dev; cdev->class_type = class_type; + cdev->config.cmd_fd = SXE2_CMD_FD_INVALID; cdev->config.kernel_reset = false; pthread_mutex_init(&cdev->config.lock, NULL); @@ -631,6 +740,7 @@ static int32_t sxe2_common_pci_id_table_update(const struct rte_pci_id *id_table updated_table = calloc(num_ids, sizeof(*updated_table)); if (!updated_table) { + ret = -ENOMEM; PMD_LOG_ERR(COM, "Failed to allocate memory for PCI ID table"); goto l_end; } diff --git a/drivers/common/sxe2/sxe2_common.h b/drivers/common/sxe2/sxe2_common.h index b02b6317da..efc8d3585a 100644 --- a/drivers/common/sxe2/sxe2_common.h +++ b/drivers/common/sxe2/sxe2_common.h @@ -14,6 +14,8 @@ #define SXE2_COMMON_PCI_DRIVER_NAME "sxe2_pci" +#define SXE2_CMD_FD_INVALID (-1) + #define SXE2_CDEV_TO_CMD_FD(cdev) \ ((cdev)->config.cmd_fd) diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c index 173d8d57ae..a233a78136 100644 --- a/drivers/common/sxe2/sxe2_ioctl_chnl.c +++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c @@ -110,7 +110,7 @@ sxe2_drv_dev_close(struct sxe2_common_device *cdev) if (fd >= 0) close(fd); PMD_LOG_INFO(COM, "closed device fd=%d", fd); - SXE2_CDEV_TO_CMD_FD(cdev) = -1; + SXE2_CDEV_TO_CMD_FD(cdev) = SXE2_CMD_FD_INVALID; } RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_handshake) -- 2.52.0