EAL feature request

["Morten Brørup" <mb@smartsharesystems.com>]

Bruce, David,

I don't know if this would be useful or not, but here goes...

Situation today:

DPDK has 3 types of threads with a valid lcore_id:
1. EAL threads, 
2. Service threads,
3. Non-EAL threads.

EAL threads are controlled by DPDK, and associated with physical cores at startup using the --lcores=<lcore_id>@<cpu_ids> parameter, and their OS cpuset/affinity etc. is managed by DPDK.

Service threads are also managed by DPDK.

Non-EAL threads are application created threads, which are simply assigned a valid lcore_id, but their OS cpuset/affinity etc. remains managed by the application.

Furthermore, macros such as RTE_LCORE_FOREACH_WORKER() only iterate over EAL threads, and rte_lcore_is_enabled() is only true for EAL threads.
AFAIU, this means that DPDK only considers EAL threads as fastpath ("WORKER") threads.
(The main thread is also an EAL thread, although it is usually not a fastpath thread. But its OS cpuset/affinity etc. is managed by DPDK, which is good.)


Feature request:

Functions to create and destroy EAL threads at runtime, so applications like Grout could dynamically create/destroy threads that:
1. DPDK considers EAL (i.e. WORKER) threads,
2. Have their OS cpuset/affinity etc. managed by DPDK.

Such functions would probably need to take <lcore_id> and <cpu_ids> parameters, for dynamically updating the list of DPDK managed lcores and CPUs at runtime.

Maybe the ability to dynamically manipulate the <lcore_id>@<cpu_ids> mapping could be a separate set of APIs, so rte_thread_create_eal() would not require those parameters. (Don't know, just brainstorming.)


Reason for asking:

Because DPDK doesn't provide this feature (hot-plug/remove of EAL threads), Grout has to take on the task of performing its own thread management (OS cpuset/affinity etc.), register its fastpath threads as Non-EAL threads, and cannot use RTE_LCORE_FOREACH_WORKER()/rte_lcore_is_enabled() for them, but considers Non-EAL as fastpath threads.


Generally it would be logical for an application to register its control plane threads as Non-EAL threads to get the benefits that come with a valid lcore_id. But how then can DPDK discriminate between control plane and fastpath threads? Only the application knows.

It would be nice if DPDK's perception of which threads are fastpath or control plane was aligned with the application's perception.


NB:
For completeness, I should mention threads with no lcore_id, created via rte_thread_create_control(), and the OS cpuset/affinity etc. managed by the application.


Venlig hilsen / Kind regards,
-Morten Brørup