Re: [PATCH 00/30] RSEQ node id and mm concurrency id extensions

[Marco Elver <elver@google.com>]

Hi Mathieu, all,

On Tue, Nov 22, 2022 at 03:39PM -0500, Mathieu Desnoyers wrote:
> Extend the rseq ABI to expose NUMA node ID, mm_cid, and mm_numa_cid
> fields.
> 
> The NUMA node ID field allows implementing a faster getcpu(2) in libc.
> 
> The per-memory-map concurrency id (mm_cid) [1] allows ideal scaling
> (down or up) of user-space per-cpu data structures. The concurrency ids
> allocated within a memory map are tracked by the scheduler, which takes
> into account the number of concurrently running threads, thus implicitly
> considering the number of threads, the cpu affinity, the cpusets
> applying to those threads, and the number of logical cores on the
> system.
> 
> The NUMA-aware concurrency id (mm_numa_cid) is similar to the mm_cid,
> except that it keeps track of the NUMA node ids with which each cid has
> been associated. On NUMA systems, when a NUMA-aware concurrency ID is
> observed by user-space to be associated with a NUMA node, it is
> guaranteed to never change NUMA node unless a kernel-level NUMA
> configuration change happens. This is useful for NUMA-aware per-cpu data
> structures running in environments where a process or a set of processes
> belonging to cpuset are pinned to a set of cores which belong to a
> subset of the system's NUMA nodes.
[...]

Just out of curiosity: is anyone aware of any libraries that have
started using CIDs? It looks like the cost of CID assignment is always
paid (even though it should be small), I'm trying to understand if after
1.5 years there are common libraries that have started using it and what
their exact usecase is.

I'm aware that TCMalloc was the inspiration for vCPUs [1], then renamed to
CIDs, but am wondering if other users are out there.

Thanks,
-- Marco

[1] https://lore.kernel.org/lkml/20220218210633.23345-10-mathieu.desnoyers@efficios.com/