RE: [RFC] mbuf: add configurable base private size for pktmbuf pools

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Randy Tice (rtice) [mailto:rtice@cisco.com] 
> Sent: Monday, 31 August 2026 20.20
> 
> Hi,
> I would like to get feedback on a proposed mbuf change before sending patches.
> Some deployments need a guaranteed private-data reservation in every packet mbuf, across multiple mbuf pools and across different consumers of the mbuf APIs.
> Today, each pktmbuf pool can request a private size when the pool is created. That works when the application owns all pool creation policy directly. However, not all relevant mbuf pools are necessarily created by application code. Some pools may be created by libraries, drivers, or other components outside direct application control.
> One example already in DPDK is vhost crypto, which creates its own mbuf pool and supplies a private size for struct vhost_crypto_data_req. There are also driver-created pktmbuf-style pools, such as cnxk inline meta pools and TAP GSO context pools. These are examples of pool-creation paths where the application may not directly control the private-size value used at creation time.
> A PMD-specific devarg could solve one instance of this problem, such as a single driver-created pool, but that seems too narrow if the requirement is not inherently PMD-specific. A deployment with multiple drivers, libraries, or other pool-creation paths outside application control could need the same base private-size adjustment. In that case, configuring the same value independently through component-specific options would be fragile and easy to get wrong.
> The proposed generic model is to add a configurable base private size for pktmbuf pools. The effective private size would be:
> align(pool_requested_priv_size + application_base_priv_size,
>       RTE_MBUF_PRIV_ALIGN)
> The tentative EAL option name is:
> --mbuf-base-priv-size=<size>
> The intent is:
> * default behavior remains unchanged when the option is not used;
> * the configured base size is added to the private size requested by each pktmbuf pool;
> * the final effective private size remains aligned to RTE_MBUF_PRIV_ALIGN;
> * pool-specific private-data requests still work as they do today;
> * DPDK centralizes the policy so pools created outside application control can reserve the same base private-data space as application-created pools.
> This is not intended to define ownership or layout of the private area. It only ensures that a deployment can reserve a common base amount of private data consistently. Applications, drivers, libraries, or components would still be responsible for their own interpretation of the reserved private area.
> Questions for the list:
> 1. Is a deployment-wide pktmbuf base private-size reservation something DPDK would consider acceptable?
> 2. Is --mbuf-base-priv-size=<size> a reasonable name, or would another name better describe the intent?
> 3. Should DPDK expose the effective-size calculation as a helper so pool-creation paths outside application control can apply the same rule?
> 4. Would maintainers prefer consumer updates in the same series as example users, or as follow-up patches after the generic mbuf/EAL change is accepted?
> 5. Would maintainers prefer this to remain component-specific, even if more than one driver, library, or pool-creation path may need to apply the same base reservation?
> The main goal is to avoid downstream mbuf layout changes and avoid component-specific configuration drift, while still allowing deployments to reserve a consistent private-data area across all packet mbuf pools, including pools created outside direct application control.
> Thanks,
> Randy

DPDK already has Dynamic Mbuf Fields for run-time management of private data across all mbuf pools.
DPDK also has the Private Data Area (priv_size), but that is individual to each mbuf pool, which does not fit your use case.

Dynamic Mbuf Fields is the perfect fit for the use case you are describing.

Currently, it only manages a few small memory areas inside the rte_mbuf structure itself, the dynfield1 array and the dynfield2 field.
But it could easily manage one more memory area associated with the rte_mbuf structure.

If we want this to be build-time configurable, it should be relatively simple to add:

In config/rte_common.h:
+#define RTE_MBUF_DYN_EXTRA_SIZE 128

In lib/mbuf/rte_mbuf_core.h:
	/** Size of the application private data. In case of an indirect
	 * mbuf, it stores the direct mbuf private data size.
	 */
	uint16_t priv_size;

	/** Timesync flags for use with IEEE1588. */
	uint16_t timesync;

	uint32_t dynfield1[9]; /**< Reserved for dynamic fields. */
+
+#if RTE_MBUF_DYN_EXTRA_SIZE
+	/** Extra dynamic fields. */
+	uint32_t dynfield3[RTE_MBUF_DYN_EXTRA_SIZE / sizeof(uint32_t)];
+#endif
};

In lib/mbuf/rte_mbuf_dyn.h:
+static_assert(RTE_MBUF_DYN_EXTRA_SIZE % RTE_CACHE_LINE_SIZE == 0,
+	"RTE_MBUF_DYN_EXTRA_SIZE must be multiple of cache line size.");

And some associated additions in lib/mbuf/rte_mbuf_dyn.c.

<feature creep>

There may also be considerations about what happens to the extra data when:
- Copying an mbuf.
- Attaching an mbuf to another mbuf.
- Detaching an mbuf from another mbuf.
- Cloning an mbuf.

(The considerations apply to both the packet mbuf itself, and for the non-first segments of a segmented packet mbuf.)

The developer of the Dynamic Mbuf Fields library was foreseeable enough to add a "flags" parameter for dynamic mbuf field creation.
This could be used to specify what happens to each registered dynamic field in the events enumerated above.

</feature creep>

IMO, the extended size of Dynamic Mbuf Fields should be build-time configurable.

If the community wants the extended size of Dynamic Mbuf Fields run-time configurable, the size should be an EAL startup parameter.
It could be named: --mbuf-dyn-extra-size=<size>.
The major difference in implementation is that the space for the extra dynfields must be dynamically allocated with the mbufs at mbuf pool creation.
Notice that the memory for the extra dynfields should be positioned between the mbuf structure and the private data area, so their offsets remain the same, also for two mbuf pools having different Private Data Area sizes.

-Morten