[PATCH] kernel/freebsd: add NUMA domain selection for contigmem

[Kumar Sanghvi <divinekumar@gmail.com>]

From: Kumar Sanghvi <ksanghvi@zscaler.com>

On multi-socket systems, DPDK applications pin their threads to cores
on the socket local to the NIC, but the FreeBSD EAL reports all lcores
and all contigmem memory as socket 0, so an application has no way to
ask for local buffers. Physical placement is decided entirely by
contigmem, which allocates its buffers at module load with
contigmalloc(). That uses a round-robin domain policy, so the buffers
can land on a remote domain, and every DMA transfer and packet copy
then crosses the inter-socket interconnect.

Add a hw.contigmem.numa_domain loader tunable. When it is set to a
domain number, every buffer is allocated from that domain with
contigmalloc_domainset(DOMAINSET_FIXED()).

The placement is strict. If the domain cannot provide a buffer, the
module load fails and the requested domain is reported, rather than
buffers being silently placed elsewhere. An out-of-range value also
fails the load. The default, -1, keeps the existing behaviour of
allocating from any domain.

The tunable takes effect only at module load and is exposed read-only
through sysctl. EAL socket reporting is unchanged.

Usage:
  kenv hw.contigmem.numa_domain=1
  kldload contigmem

Tested on FreeBSD 13.2, dual-socket Intel Xeon Gold 6442Y (Sapphire
Rapids, 2 x 24 cores) with Intel E810 25G NICs attached to socket 1.
A DPDK application on socket 1 received packets with a poll-mode
driver and copied them to three packet-processing applications, also
on socket 1, driven by httperf TCP traffic (1000 connections, 128 KB
objects). Measured with Intel PCM (pcm, pcm-memory, pcm-numa) at
steady state, default round-robin placement vs. all buffers in
domain 1:
  - remote memory bandwidth of the poll-mode thread:
    ~1800 MB/s -> ~0 MB/s
  - incoming UPI traffic: 2340 MB/s -> 77 MB/s (-97%)
  - power: 7 W lower (5 W CPU + 2 W DIMM)

Signed-off-by: Kumar Sanghvi <ksanghvi@zscaler.com>
---
 .mailmap                               |  2 +-
 doc/guides/freebsd_gsg/build_dpdk.rst  | 23 +++++++++++++++++
 doc/guides/rel_notes/release_26_11.rst |  5 ++++
 kernel/freebsd/contigmem/contigmem.c   | 34 +++++++++++++++++++++++---
 4 files changed, 60 insertions(+), 4 deletions(-)

diff --git a/.mailmap b/.mailmap
index 6d9e262cfb..e1c99d2ba7 100644
--- a/.mailmap
+++ b/.mailmap
@@ -915,7 +915,7 @@ Krzysztof Witek <krzysztof.witek@6wind.com>
 Kuan Xu <kuanx.xu@intel.com>
 Kuba Kozak <kubax.kozak@intel.com>
 Kumar Amber <kumar.amber@intel.com>
-Kumar Sanghvi <kumaras@chelsio.com>
+Kumar Sanghvi <ksanghvi@zscaler.com> <kumaras@chelsio.com>
 Kumara Parameshwaran <kumaraparamesh92@gmail.com> <kparameshwar@vmware.com>
 Kyle Larose <klarose@sandvine.com>
 Kyo Liu <kyo.liu@nebula-matrix.com>
diff --git a/doc/guides/freebsd_gsg/build_dpdk.rst b/doc/guides/freebsd_gsg/build_dpdk.rst
index 9dafa64e84..509e06a6ac 100644
--- a/doc/guides/freebsd_gsg/build_dpdk.rst
+++ b/doc/guides/freebsd_gsg/build_dpdk.rst
@@ -107,6 +107,29 @@ Mapped buffers can be included in core dump using the following tunable::
    Buffers typically hold data processed by the application,
    like network packets, which may contain sensitive information.
 
+On multi-socket systems, the buffers can be allocated from a specific
+NUMA domain using the following tunable::
+
+   hw.contigmem.numa_domain=d
+
+Where d is a domain number from 0 to ``vm.ndomains`` - 1.
+Placing the buffers in the domain local to the NIC and to the cores
+running the DPDK application avoids cross-socket memory traffic.
+The default, -1, allocates buffers from any domain.
+The placement is strict: the module fails to load
+if the requested domain cannot provide all buffers
+or if the value is out of range.
+The tunable takes effect only when the module is loaded.
+The NUMA domain of a device can be found using
+``sysctl dev.<driver>.<unit>.%domain``,
+for example ``sysctl dev.ice.0.%domain``.
+
+.. note::
+
+   The FreeBSD EAL reports all lcores and all contigmem memory as socket 0.
+   This tunable changes only the physical placement of the buffers;
+   applications should keep requesting memory from socket 0 or ``SOCKET_ID_ANY``.
+
 The kernel environment variables can also be specified during boot by placing the
 following in ``/boot/loader.conf``:
 
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index be758a0c02..9e66c6907f 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added NUMA domain selection to the FreeBSD contigmem module.**
+
+  Added the ``hw.contigmem.numa_domain`` tunable
+  to allocate contigmem buffers from a chosen NUMA domain.
+
 * **Added division round up macro.**
 
   Added the ``RTE_DIV_ROUND_UP`` macro in ``rte_common.h``
diff --git a/kernel/freebsd/contigmem/contigmem.c b/kernel/freebsd/contigmem/contigmem.c
index cb9298c9c6..441ce51644 100644
--- a/kernel/freebsd/contigmem/contigmem.c
+++ b/kernel/freebsd/contigmem/contigmem.c
@@ -20,6 +20,7 @@ __FBSDID("$FreeBSD$");
 #include <sys/sysctl.h>
 #include <sys/vmmeter.h>
 #include <sys/eventhandler.h>
+#include <sys/domainset.h>
 
 #include <machine/bus.h>
 
@@ -59,10 +60,14 @@ static struct contigmem_buffer contigmem_buffers[RTE_CONTIGMEM_MAX_NUM_BUFS];
 static struct cdev     *contigmem_cdev = NULL;
 static int              contigmem_refcnt;
 
+/* -1 = any domain (default) */
+static int contigmem_numa_domain = -1;
+
 TUNABLE_INT("hw.contigmem.num_buffers", &contigmem_num_buffers);
 TUNABLE_QUAD("hw.contigmem.buffer_size", &contigmem_buffer_size);
 TUNABLE_QUAD("hw.contigmem.buffer_size_MB", &contigmem_buffer_size_MB);
 TUNABLE_BOOL("hw.contigmem.coredump_enable", &contigmem_coredump_enable);
+TUNABLE_INT("hw.contigmem.numa_domain", &contigmem_numa_domain);
 
 static SYSCTL_NODE(_hw, OID_AUTO, contigmem, CTLFLAG_RD, 0, "contigmem");
 
@@ -77,6 +82,9 @@ SYSCTL_INT(_hw_contigmem, OID_AUTO, num_references, CTLFLAG_RD,
 	&contigmem_refcnt, 0, "Number of references to contigmem");
 SYSCTL_BOOL(_hw_contigmem, OID_AUTO, coredump_enable, CTLFLAG_RD,
 	&contigmem_coredump_enable, 0, "Include mapped buffers in core dump");
+SYSCTL_INT(_hw_contigmem, OID_AUTO, numa_domain, CTLFLAG_RD,
+	&contigmem_numa_domain, 0,
+	"NUMA domain for buffer allocation (-1 = any domain)");
 
 static SYSCTL_NODE(_hw_contigmem, OID_AUTO, physaddr, CTLFLAG_RD, 0,
 	"physaddr");
@@ -150,11 +158,31 @@ contigmem_load(void)
 		goto error;
 	}
 
+	if (contigmem_numa_domain < -1 ||
+			contigmem_numa_domain >= vm_ndomains) {
+		printf("contigmem: invalid numa_domain %d (valid: -1 to %d)\n",
+				contigmem_numa_domain, vm_ndomains - 1);
+		error = EINVAL;
+		goto error;
+	}
+
 	for (i = 0; i < contigmem_num_buffers; i++) {
-		addr = contigmalloc(contigmem_buffer_size, M_CONTIGMEM, M_ZERO,
-			0, BUS_SPACE_MAXADDR, contigmem_buffer_size, 0);
+		if (contigmem_numa_domain >= 0)
+			addr = contigmalloc_domainset(contigmem_buffer_size,
+				M_CONTIGMEM,
+				DOMAINSET_FIXED(contigmem_numa_domain),
+				M_ZERO, 0, BUS_SPACE_MAXADDR,
+				contigmem_buffer_size, 0);
+		else
+			addr = contigmalloc(contigmem_buffer_size, M_CONTIGMEM,
+				M_ZERO, 0, BUS_SPACE_MAXADDR,
+				contigmem_buffer_size, 0);
 		if (addr == NULL) {
-			printf("contigmalloc failed for buffer %d\n", i);
+			if (contigmem_numa_domain >= 0)
+				printf("contigmem: buffer %d: no contiguous memory in NUMA domain %d\n",
+						i, contigmem_numa_domain);
+			else
+				printf("contigmalloc failed for buffer %d\n", i);
 			error = ENOMEM;
 			goto error;
 		}
-- 
2.50.1 (Apple Git-155)