Re: Prevent inconsistent CPU state after sequence of dlclose/dlopen

Peter Zijlstra
2025-01-10T16:54:18Z

On Fri, Jan 10, 2025 at 10:55:36AM -0500, Mathieu Desnoyers wrote:
> Hi,
>
> I was discussing with Mark Rutland recently, and he pointed out that a
> sequence of dlclose/dlopen mapping new code at the same addresses in
> multithreaded environments is an issue on ARM, and possibly on Intel/AMD
> with the newer TLB broadcast maintenance.

What is the exact race? Should not munmap() invalidate the TLBs before
it allows overlapping mmap() to complete?

Any concurrent access after munmap() / before mmap() completes is UB
anyway, no?

> I maintain the membarrier(2) system call, which provides a
> MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE command for this
> purpose. It's been there since Linux 4.16. It can be configured
> out (CONFIG_MEMBARRIER=n), but it's enabled by default.
>
> Calling this after dlclose() in glibc would prevent this issue.
>
> Is it handled in some other way, or should we open a bugzilla
> entry to track this ?

The problem is that the membarrier() call has significant cost, and is
only really needed if dlopen() is called right after (in the same
location).

Unconditionally adding that barrier, just in case, might regress things,
no?