virtio_user exception path with external memory

virtio_user exception path with external memory

[Pekka Riikonen]

It seems that currently it's not possible to use packet pools allocated 
with external memory with the virtio_user exception path use case, 
because it ignores extmem in virtio_user_mem_event_cb() (and also in the 
kernel backend in add_memseg_list()).  This is easy to test with testpmd 
with the --mp-alloc=xmemhuge or --mp-alloc=anon option 
(https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html).

The original commit that added the check (5282bb1c3 mem: allow memseg 
lists to be marked as external) talks about needing file descriptors in 
virtio that aren't available with extmem, but at least the kernel 
backend doesn't seem to need any.  The patch below fixes the issue for 
me with with the exception path.

So I'm wondering whether ignoring the extmem is actually needed, in this 
use case?

---
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c 
b/drivers/net/virtio/virtio_user/vhost_kernel.c
index e42bb35935..cd5b975675 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -189,9 +189,6 @@ add_memseg_list(const struct rte_memseg_list *msl, 
void *arg)
  	void *start_addr;
  	uint64_t len;

-	if (msl->external)
-		return 0;
-
  	if (vm->nregions >= max_regions)
  		return -1;

diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c 
b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 187f81b066..d5204a64a7 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -539,20 +539,14 @@ virtio_user_fill_intr_handle(struct 
virtio_user_dev *dev)

  static void
  virtio_user_mem_event_cb(enum rte_mem_event type __rte_unused,
-			 const void *addr,
+			 const void *addr __rte_unused,
  			 size_t len __rte_unused,
  			 void *arg)
  {
  	struct virtio_user_dev *dev = arg;
-	struct rte_memseg_list *msl;
  	uint16_t i;
  	int ret = 0;

-	/* ignore externally allocated memory */
-	msl = rte_mem_virt2memseg_list(addr);
-	if (msl->external)
-		return;
-
  	pthread_mutex_lock(&dev->mutex);

  	if (dev->started == false)
---

Pekka

Re: virtio_user exception path with external memory

[Maxime Coquelin]

Hi Pekka,

On 8/6/25 9:56 AM, Pekka Riikonen wrote:
> 
> It seems that currently it's not possible to use packet pools allocated 
> with external memory with the virtio_user exception path use case, 
> because it ignores extmem in virtio_user_mem_event_cb() (and also in the 
> kernel backend in add_memseg_list()).  This is easy to test with testpmd 
> with the --mp-alloc=xmemhuge or --mp-alloc=anon option (https:// 
> doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html).
> 
> The original commit that added the check (5282bb1c3 mem: allow memseg 
> lists to be marked as external) talks about needing file descriptors in 
> virtio that aren't available with extmem, but at least the kernel 
> backend doesn't seem to need any.  The patch below fixes the issue for 
> me with with the exception path.
> 
> So I'm wondering whether ignoring the extmem is actually needed, in this 
> use case?

It might not be needed, but we would need to take care of not breaking 
backends that does not support extmem.

Could you prepare a formal patch?

Thanks,
Maxime

> 
> ---
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/ 
> net/virtio/virtio_user/vhost_kernel.c
> index e42bb35935..cd5b975675 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -189,9 +189,6 @@ add_memseg_list(const struct rte_memseg_list *msl, 
> void *arg)
>       void *start_addr;
>       uint64_t len;
> 
> -    if (msl->external)
> -        return 0;
> -
>       if (vm->nregions >= max_regions)
>           return -1;
> 
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/ 
> net/virtio/virtio_user/virtio_user_dev.c
> index 187f81b066..d5204a64a7 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -539,20 +539,14 @@ virtio_user_fill_intr_handle(struct 
> virtio_user_dev *dev)
> 
>   static void
>   virtio_user_mem_event_cb(enum rte_mem_event type __rte_unused,
> -             const void *addr,
> +             const void *addr __rte_unused,
>                size_t len __rte_unused,
>                void *arg)
>   {
>       struct virtio_user_dev *dev = arg;
> -    struct rte_memseg_list *msl;
>       uint16_t i;
>       int ret = 0;
> 
> -    /* ignore externally allocated memory */
> -    msl = rte_mem_virt2memseg_list(addr);
> -    if (msl->external)
> -        return;
> -
>       pthread_mutex_lock(&dev->mutex);
> 
>       if (dev->started == false)
> ---
> 
> Pekka
>