Re: Segfault in mlx5 driver on infiniband after application fork

[Jason Gunthorpe <jgg@ziepe.ca>]

On Mon, Feb 12, 2024 at 11:37:39AM -0500, Kevan Rehm wrote:
> 
> 
> > On Feb 12, 2024, at 11:12 AM, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > 
> > On Mon, Feb 12, 2024 at 11:04:36AM -0500, Kevan Rehm wrote:
> > 
> >> Those routines call ibv_dontfork_range on the page after it’s been
> >> allocated via posix_memalign().  _add_page() then adds the new page
> >> to the mlx5_context field dbr_available_pages.
> > 
> > Oh, if this is your trouble then upgrade your kernel. This part is
> > fixed on kernels that have working fork support.
> 
> That’s the bit that confuses me; all this is happening in user
> space, what is different in the kernel that would prevent this
> problem from occurring in user space?  Any guess as to how much
> newer a kernel must be?

Newer kernels are detected and disable the DONT_FORK calls in verbs.

rdma-core support is present since:

commit 67b00c3835a3480a035a9e1bcf5695f5c0e8568e
Author: Gal Pressman <galpress@amazon.com>
Date:   Sun Apr 4 17:24:54 2021 +0300

    verbs: Report when ibv_fork_init() is not needed
    
    Identify kernels which do not require ibv_fork_init() to be called and
    report it through the ibv_is_fork_initialized() verb.
    
    The feature detection is done through a new read-only attribute in the
    get sys netlink command. If the attribute is not reported, assume old
    kernel without COF support. If the attribute is reported, use the
    returned value.
    
    This allows ibv_is_fork_initialized() to return the previously unused
    IBV_FORK_UNNEEDED value, which takes precedence over the
    DISABLED/ENABLED values. Meaning that if the kernel does not require a
    call to ibv_fork_init(), IBV_FORK_UNNEEDED will be returned regardless
    of whether ibv_fork_init() was called or not.
    
    Signed-off-by: Gal Pressman <galpress@amazon.com>

The kernel support was in v5.13-rc1~78^2~1

And backported in a few cases.

Jason