* [PATCH rdma-next 1/9] RDMA/mlx5: Validate port number in handle_port_change()
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 2/9] RDMA/mlx5: Don't let a slave port mark its master active Edward Srouji
` (8 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
handle_port_change() takes the port number from a 4-bit EQE field and,
on a P_Key change, uses it to index devr.ports[] before anything
validates it; mlx5_ib_handle_event() checks it only afterwards.
Validate the port first.
Found by inspection; firmware is not expected to report such a port.
Fixes: 7722f47e71e5 ("IB/mlx5: Create GSI transmission QPs when P_Key table is changed")
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/hw/mlx5/main.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 65455aedd559b6291831492e92a68b24a6bd361e..789e17bd317a48db502937a6816e95aa22f48e6f 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3062,6 +3062,11 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe,
{
u32 port = (eqe->data.port.port >> 4) & 0xf;
+ if (!rdma_is_port_valid(&ibdev->ib_dev, port)) {
+ mlx5_ib_warn(ibdev, "warning: event on port %d\n", port);
+ return -EINVAL;
+ }
+
ibev->element.port_num = port;
switch (eqe->sub_type) {
@@ -3139,11 +3144,6 @@ static void mlx5_ib_handle_event(struct work_struct *_work)
ibev.device = &ibdev->ib_dev;
- if (!rdma_is_port_valid(&ibdev->ib_dev, ibev.element.port_num)) {
- mlx5_ib_warn(ibdev, "warning: event on port %d\n", ibev.element.port_num);
- goto out;
- }
-
if (ibdev->ib_active)
ib_dispatch_event(&ibev);
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 2/9] RDMA/mlx5: Don't let a slave port mark its master active
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 1/9] RDMA/mlx5: Validate port number in handle_port_change() Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 3/9] RDMA/mlx5: Flush system error and slave port event work Edward Srouji
` (7 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
mlx5r_mp_probe() sets ib_active on the master a slave port binds to.
Commit 97f30d324ce6 ("RDMA/mlx5: Recover from fatal event in dual port
mode") added that because a fatal event on the slave port used to
clear the master's ib_active. Since commit ebc2164a4cd4 ("RDMA/mlx5:
Fix UMR hang in LAG error state unload") system errors are handled by
a notifier on the master's own core device, and an error on the slave
no longer deactivates the master.
A master is on mlx5_ib_dev_list from MLX5_IB_STAGE_INIT until its
cleanup, while its ib_active is set only once probe completes, and a
slave port can bind to it at any point in between, as binding only
needs a free port slot. Setting ib_active on bind can then activate a
master that is still probing, is being removed or has failed, so that
events are handled for a device that is not registered yet or is being
torn down. Remove it.
The bind itself, and the rdma_roce_rescan_device() that follows it, can
still race a master that is probing or being removed; that is a
separate problem and is not addressed here.
Fixes: 97f30d324ce6 ("RDMA/mlx5: Recover from fatal event in dual port mode")
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/hw/mlx5/main.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 789e17bd317a48db502937a6816e95aa22f48e6f..4504ba112034f4a5cfad16183602f209941c8835 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -5297,7 +5297,6 @@ static int mlx5r_mp_probe(struct auxiliary_device *adev,
if (bound) {
rdma_roce_rescan_device(&dev->ib_dev);
- mpi->ibdev->ib_active = true;
break;
}
}
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 3/9] RDMA/mlx5: Flush system error and slave port event work
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 1/9] RDMA/mlx5: Validate port number in handle_port_change() Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 2/9] RDMA/mlx5: Don't let a slave port mark its master active Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 4/9] RDMA/core: Derive enforce_security from the cache event Edward Srouji
` (6 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
Work queued on mlx5_ib_event_wq holds no references, and nothing waits
for the work of the system error and slave port notifiers once they
are unregistered. Their handlers can then use the device after
__mlx5_ib_remove() frees it, or work->mpi after mlx5r_mp_remove()
frees it.
Flush the workqueue after unregistering them: the system error
notifier in its stage cleanup, and the slave port notifier in
mlx5r_mp_remove() and mlx5_ib_cleanup_multiport_master(), after
dropping mlx5_ib_multiport_mutex, which the slave port handler takes.
The system error notifier stays registered until after
ib_unregister_device(), as commit ebc2164a4cd4 ("RDMA/mlx5: Fix UMR
hang in LAG error state unload") requires.
Fixes: 5d44adebbb7e ("IB/mlx5: Free mpi in mp_slave mode")
Fixes: ebc2164a4cd4 ("RDMA/mlx5: Fix UMR hang in LAG error state unload")
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/hw/mlx5/main.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 4504ba112034f4a5cfad16183602f209941c8835..775c3612a427a66ccb597595e149e3eada486add 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3248,6 +3248,7 @@ static int mlx5_ib_stage_sys_error_notifier_init(struct mlx5_ib_dev *dev)
static void mlx5_ib_stage_sys_error_notifier_cleanup(struct mlx5_ib_dev *dev)
{
mlx5_notifier_unregister(dev->mdev, &dev->sys_error_events);
+ flush_workqueue(mlx5_ib_event_wq);
}
static int mlx5_ib_get_plane_num(struct mlx5_core_dev *mdev, u8 *num_plane)
@@ -4085,6 +4086,7 @@ static void mlx5_ib_cleanup_multiport_master(struct mlx5_ib_dev *dev)
mlx5_ib_dbg(dev, "removing from devlist\n");
list_del(&dev->ib_dev_list);
mutex_unlock(&mlx5_ib_multiport_mutex);
+ flush_workqueue(mlx5_ib_event_wq);
mlx5_nic_vport_disable_roce(dev->mdev);
}
@@ -5323,6 +5325,7 @@ static void mlx5r_mp_remove(struct auxiliary_device *adev)
else
list_del(&mpi->list);
mutex_unlock(&mlx5_ib_multiport_mutex);
+ flush_workqueue(mlx5_ib_event_wq);
kfree(mpi);
}
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 4/9] RDMA/core: Derive enforce_security from the cache event
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
` (2 preceding siblings ...)
2026-11-22 17:39 ` [PATCH rdma-next 3/9] RDMA/mlx5: Flush system error and slave port event work Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 5/9] RDMA/core: Add ib_dispatch_event_sync() Edward Srouji
` (5 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
ib_dispatch_event() marks the cache update work of P_Key and GID
changes with enforce_security, which ib_cache_event_task() then passes
to ib_cache_update(). The work item carries the event itself, so derive
the flag from it in ib_cache_event_task() and drop it from struct
ib_update_work.
No functional change.
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/core/cache.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
index 647a547e2d7fb20721ed5fecdccd723891ba2ef2..c338c2e52d8bd041bdc194ca574b7c35cc9b6d0d 100644
--- a/drivers/infiniband/core/cache.c
+++ b/drivers/infiniband/core/cache.c
@@ -52,7 +52,6 @@ struct ib_pkey_cache {
struct ib_update_work {
struct work_struct work;
struct ib_event event;
- bool enforce_security;
};
union ib_gid zgid;
@@ -1536,7 +1535,8 @@ static void ib_cache_event_task(struct work_struct *_work)
work->event.event == IB_EVENT_GID_CHANGE ||
work->event.event == IB_EVENT_CLIENT_REREGISTER,
work->event.event == IB_EVENT_PKEY_CHANGE,
- work->enforce_security);
+ work->event.event == IB_EVENT_PKEY_CHANGE ||
+ work->event.event == IB_EVENT_GID_CHANGE);
/* GID event is notified already for individual GID entries by
* dispatch_gid_change_event(). Hence, notifiy for rest of the
@@ -1589,10 +1589,6 @@ void ib_dispatch_event(const struct ib_event *event)
INIT_WORK(&work->work, ib_generic_event_task);
work->event = *event;
- if (event->event == IB_EVENT_PKEY_CHANGE ||
- event->event == IB_EVENT_GID_CHANGE)
- work->enforce_security = true;
-
queue_work(ib_wq, &work->work);
}
EXPORT_SYMBOL(ib_dispatch_event);
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 5/9] RDMA/core: Add ib_dispatch_event_sync()
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
` (3 preceding siblings ...)
2026-11-22 17:39 ` [PATCH rdma-next 4/9] RDMA/core: Derive enforce_security from the cache event Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 6/9] RDMA/mlx5: Pass the port and subtype to handle_port_change() Edward Srouji
` (4 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
ib_dispatch_event() hands each event to a work item on ib_wq, which is
not ordered, so events dispatched one after the other can be processed
in either order, and a cache update can publish an older state after a
newer one. A driver has no way to wait for an event it dispatched
short of flushing ib_wq, which waits for the work of every device,
including work that may itself be waiting for that driver.
Add ib_dispatch_event_sync(), which processes the event in the
caller's context and returns the error of the cache update, so that
events a driver passes to it one after the other are processed in that
order. They are not ordered against events passed to
ib_dispatch_event(). It must be called from a workqueue, the context
event handlers are promised, and not from an event handler, where it
would take event_handler_rwsem recursively and could deadlock behind a
waiting writer; lockdep checks the latter.
The processing moves to a helper shared with the work item: the cache
is still updated before the handlers are called, P_Key and GID changes
still revalidate security, and the handlers are still not called when
the update fails, nor for a GID change. The handlers get a copy of the
event, as they do from the work item.
A later patch uses it in mlx5_ib to replay port events that arrived
while the device was not yet active.
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/core/cache.c | 63 ++++++++++++++++++++++++++++++++++-------
include/rdma/ib_verbs.h | 1 +
2 files changed, 53 insertions(+), 11 deletions(-)
diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
index c338c2e52d8bd041bdc194ca574b7c35cc9b6d0d..4e376a866715d4e0613087dccde232934171c1e1 100644
--- a/drivers/infiniband/core/cache.c
+++ b/drivers/infiniband/core/cache.c
@@ -1522,29 +1522,36 @@ ib_cache_update(struct ib_device *device, u32 port, bool update_gids,
return ret;
}
-static void ib_cache_event_task(struct work_struct *_work)
+static int ib_cache_event_process(struct ib_event *event)
{
- struct ib_update_work *work =
- container_of(_work, struct ib_update_work, work);
int ret;
/* Before distributing the cache update event, first sync
* the cache.
*/
- ret = ib_cache_update(work->event.device, work->event.element.port_num,
- work->event.event == IB_EVENT_GID_CHANGE ||
- work->event.event == IB_EVENT_CLIENT_REREGISTER,
- work->event.event == IB_EVENT_PKEY_CHANGE,
- work->event.event == IB_EVENT_PKEY_CHANGE ||
- work->event.event == IB_EVENT_GID_CHANGE);
+ ret = ib_cache_update(event->device, event->element.port_num,
+ event->event == IB_EVENT_GID_CHANGE ||
+ event->event == IB_EVENT_CLIENT_REREGISTER,
+ event->event == IB_EVENT_PKEY_CHANGE,
+ event->event == IB_EVENT_PKEY_CHANGE ||
+ event->event == IB_EVENT_GID_CHANGE);
/* GID event is notified already for individual GID entries by
* dispatch_gid_change_event(). Hence, notifiy for rest of the
* events.
*/
- if (!ret && work->event.event != IB_EVENT_GID_CHANGE)
- ib_dispatch_event_clients(&work->event);
+ if (!ret && event->event != IB_EVENT_GID_CHANGE)
+ ib_dispatch_event_clients(event);
+
+ return ret;
+}
+static void ib_cache_event_task(struct work_struct *_work)
+{
+ struct ib_update_work *work =
+ container_of(_work, struct ib_update_work, work);
+
+ ib_cache_event_process(&work->event);
kfree(work);
}
@@ -1593,6 +1600,40 @@ void ib_dispatch_event(const struct ib_event *event)
}
EXPORT_SYMBOL(ib_dispatch_event);
+/**
+ * ib_dispatch_event_sync - Process an asynchronous event before returning
+ * @event: Event to process
+ *
+ * Does in the caller's context what ib_dispatch_event() leaves to a work
+ * item, by the same rules: an event that affects the cache updates it first,
+ * and the registered event handlers are then called, unless that update
+ * failed or the event is a GID change. All of it completes before this
+ * returns, so events a caller passes here one after the other are processed
+ * in that order. They are not ordered against events passed to
+ * ib_dispatch_event().
+ *
+ * Context: Workqueue context, which event handlers are promised, with no
+ * locks held that the cache update or the handlers may take. Not from an
+ * event handler.
+ *
+ * Return: 0, or the error of the cache update.
+ */
+int ib_dispatch_event_sync(const struct ib_event *event)
+{
+ struct ib_event copy = *event;
+
+ might_sleep();
+ lockdep_assert_not_held(&event->device->event_handler_rwsem);
+
+ if (!is_cache_update_event(©)) {
+ ib_dispatch_event_clients(©);
+ return 0;
+ }
+
+ return ib_cache_event_process(©);
+}
+EXPORT_SYMBOL(ib_dispatch_event_sync);
+
int ib_cache_setup_one(struct ib_device *device)
{
u32 p;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 29228226a236638aa746b9f5e0172acded69f135..ab6b9af1a6d3018c441e608b0fbaa9afbd1aa3bf 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -3308,6 +3308,7 @@ bool ib_modify_qp_is_ok(enum ib_qp_state cur_state, enum ib_qp_state next_state,
void ib_register_event_handler(struct ib_event_handler *event_handler);
void ib_unregister_event_handler(struct ib_event_handler *event_handler);
void ib_dispatch_event(const struct ib_event *event);
+int ib_dispatch_event_sync(const struct ib_event *event);
int ib_query_port(struct ib_device *device,
u32 port_num, struct ib_port_attr *port_attr);
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 6/9] RDMA/mlx5: Pass the port and subtype to handle_port_change()
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
` (4 preceding siblings ...)
2026-11-22 17:39 ` [PATCH rdma-next 5/9] RDMA/core: Add ib_dispatch_event_sync() Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 7/9] RDMA/mlx5: Activate the device in a stage of its own Edward Srouji
` (3 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
handle_port_change() decodes the port and the subtype from the EQE.
Take them as arguments instead, so that a later patch can also pass
port events it recorded.
No functional change.
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/hw/mlx5/main.c | 23 +++++++++++++++--------
1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 775c3612a427a66ccb597595e149e3eada486add..954b141a9c927d5cf4f20a17cc633de09624b0ef 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3041,10 +3041,15 @@ static void delay_drop_handler(struct work_struct *work)
mutex_unlock(&delay_drop->lock);
}
+static u32 mlx5_ib_eqe_port_num(struct mlx5_eqe *eqe)
+{
+ return (eqe->data.port.port >> 4) & 0xf;
+}
+
static void handle_general_event(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe,
struct ib_event *ibev)
{
- u32 port = (eqe->data.port.port >> 4) & 0xf;
+ u32 port = mlx5_ib_eqe_port_num(eqe);
switch (eqe->sub_type) {
case MLX5_GENERAL_SUBTYPE_DELAY_DROP_TIMEOUT:
@@ -3057,11 +3062,9 @@ static void handle_general_event(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe
}
}
-static int handle_port_change(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe,
+static int handle_port_change(struct mlx5_ib_dev *ibdev, u32 port, u8 sub_type,
struct ib_event *ibev)
{
- u32 port = (eqe->data.port.port >> 4) & 0xf;
-
if (!rdma_is_port_valid(&ibdev->ib_dev, port)) {
mlx5_ib_warn(ibdev, "warning: event on port %d\n", port);
return -EINVAL;
@@ -3069,7 +3072,7 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe,
ibev->element.port_num = port;
- switch (eqe->sub_type) {
+ switch (sub_type) {
case MLX5_PORT_CHANGE_SUBTYPE_ACTIVE:
case MLX5_PORT_CHANGE_SUBTYPE_DOWN:
case MLX5_PORT_CHANGE_SUBTYPE_INITIALIZED:
@@ -3088,7 +3091,7 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe,
IB_LINK_LAYER_ETHERNET)
return -EINVAL;
- ibev->event = (eqe->sub_type == MLX5_PORT_CHANGE_SUBTYPE_ACTIVE) ?
+ ibev->event = (sub_type == MLX5_PORT_CHANGE_SUBTYPE_ACTIVE) ?
IB_EVENT_PORT_ACTIVE : IB_EVENT_PORT_ERR;
break;
@@ -3131,10 +3134,14 @@ static void mlx5_ib_handle_event(struct work_struct *_work)
}
switch (work->event) {
- case MLX5_EVENT_TYPE_PORT_CHANGE:
- if (handle_port_change(ibdev, work->param, &ibev))
+ case MLX5_EVENT_TYPE_PORT_CHANGE: {
+ struct mlx5_eqe *eqe = work->param;
+
+ if (handle_port_change(ibdev, mlx5_ib_eqe_port_num(eqe),
+ eqe->sub_type, &ibev))
goto out;
break;
+ }
case MLX5_EVENT_TYPE_GENERAL_EVENT:
handle_general_event(ibdev, work->param, &ibev);
fallthrough;
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 7/9] RDMA/mlx5: Activate the device in a stage of its own
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
` (5 preceding siblings ...)
2026-11-22 17:39 ` [PATCH rdma-next 6/9] RDMA/mlx5: Pass the port and subtype to handle_port_change() Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 8/9] RDMA/mlx5: Replay port configuration events after probe Edward Srouji
` (2 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
__mlx5_ib_add() sets ib_active after its last stage, and
__mlx5_ib_remove() clears it before cleaning up any. Do both in a new
last stage, MLX5_IB_STAGE_ACTIVE, so that a later patch can fail the
activation and start work from it, and stop that work on removal.
No functional change.
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/hw/mlx5/main.c | 23 ++++++++++++++++++++---
drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 +
2 files changed, 21 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 954b141a9c927d5cf4f20a17cc633de09624b0ef..80399920078edfca8f672fb77514ba6b90966068 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -5022,12 +5022,21 @@ static void mlx5_ib_stage_dev_notifier_cleanup(struct mlx5_ib_dev *dev)
cancel_work_sync(&devr->ports[port].pkey_change_work);
}
+static int mlx5_ib_stage_active_init(struct mlx5_ib_dev *dev)
+{
+ dev->ib_active = true;
+ return 0;
+}
+
+static void mlx5_ib_stage_active_cleanup(struct mlx5_ib_dev *dev)
+{
+ dev->ib_active = false;
+}
+
void __mlx5_ib_remove(struct mlx5_ib_dev *dev,
const struct mlx5_ib_profile *profile,
int stage)
{
- dev->ib_active = false;
-
/* Number of stages to cleanup */
while (stage) {
stage--;
@@ -5055,7 +5064,6 @@ int __mlx5_ib_add(struct mlx5_ib_dev *dev,
}
}
- dev->ib_active = true;
return 0;
err_out:
@@ -5129,6 +5137,9 @@ static const struct mlx5_ib_profile pf_profile = {
STAGE_CREATE(MLX5_IB_STAGE_RESTRACK,
mlx5_ib_restrack_init,
NULL),
+ STAGE_CREATE(MLX5_IB_STAGE_ACTIVE,
+ mlx5_ib_stage_active_init,
+ mlx5_ib_stage_active_cleanup),
};
const struct mlx5_ib_profile raw_eth_profile = {
@@ -5189,6 +5200,9 @@ const struct mlx5_ib_profile raw_eth_profile = {
STAGE_CREATE(MLX5_IB_STAGE_RESTRACK,
mlx5_ib_restrack_init,
NULL),
+ STAGE_CREATE(MLX5_IB_STAGE_ACTIVE,
+ mlx5_ib_stage_active_init,
+ mlx5_ib_stage_active_cleanup),
};
static const struct mlx5_ib_profile plane_profile = {
@@ -5216,6 +5230,9 @@ static const struct mlx5_ib_profile plane_profile = {
STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
mlx5_ib_stage_ib_reg_init,
mlx5_ib_stage_ib_reg_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_ACTIVE,
+ mlx5_ib_stage_active_init,
+ mlx5_ib_stage_active_cleanup),
};
static struct ib_device *mlx5_ib_add_sub_dev(struct ib_device *parent,
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index b2d4db344071bcc068a767f94ffb64b80df9499d..3884e5524e7dfb2a37515a6f4e8caaa82224c964 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -940,6 +940,7 @@ enum mlx5_ib_stages {
MLX5_IB_STAGE_POST_IB_REG_UMR,
MLX5_IB_STAGE_DELAY_DROP,
MLX5_IB_STAGE_RESTRACK,
+ MLX5_IB_STAGE_ACTIVE,
MLX5_IB_STAGE_MAX,
};
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 8/9] RDMA/mlx5: Replay port configuration events after probe
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
` (6 preceding siblings ...)
2026-11-22 17:39 ` [PATCH rdma-next 7/9] RDMA/mlx5: Activate the device in a stage of its own Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:39 ` [PATCH rdma-next 9/9] RDMA/mlx5: Register event notifier before IB registration Edward Srouji
2026-11-22 17:58 ` [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe sashiko-bot
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
ib_register_device() lets ib_core and its clients read the port's LID
and P_Key table, and they refresh what they read only on the matching
event. mlx5_ib dispatches no event until the device becomes active at
the end of probe, so a configuration change in between is lost for
good.
On hypervisor setups the SM can program a VF's partitions while it
probes, putting the tenant P_Key at index 0 in front of the default
one. ib_sa then keeps the stale index of the default P_Key and its
queries are dropped:
log_trap_info: Bad P_Key:0x802a on SL:0 from LID1:724 (GUID
0xc42a1030048d4bc) QP1:0x1 to LID2:546 (GUID 0xc42a10300e720ce)
QP2:0x1 port 20
so IPoIB never gets a carrier.
Commit 97834eba7c19 ("net/mlx5: Delay events till ib registration
ends") made mlx5_core hold such events back until the add completed,
but commit df097a278c75 ("IB/mlx5: Use the new mlx5 core notifier
API") moved mlx5_ib to a notifier that does not. This went unnoticed
for years, as a lost P_Key change matters only when it changes
index 0, used by IPoIB, or moves the default P_Key, whose index ib_sa
uses. OpenSM keeps the default P_Key at index 0 unless a partition is
marked indx0, as the tenant partition above is, and even then only a
small fraction of probes lose the race with the SM's write.
Commit ede132a5cf55 ("RDMA/mlx5: Move events notifier registration to
be after device registration") widened the window in which a change
breaks the SA queries: one during ib_register_device() no longer
rebuilds the GSI transmit QPs either.
Record LID and P_Key changes and client reregistration requests on
InfiniBand ports while the device is inactive, and replay them when it
becomes active, ahead of any later event. The replay processes them
with ib_dispatch_event_sync(), so that a replayed change cannot
overwrite the result of a newer one. GUID changes are not recorded, as
on InfiniBand ports ib_core notifies no client of them; a lost one
leaves only the GID cache stale. Port state changes are not recorded
either, being sensitive to the order in which ib_core delivers events.
While the device is inactive, the notifier now queues no handler at
all, rather than running the handlers without dispatching to ib_core.
A device that hits a system error during probe now fails it with -EIO
rather than being activated, which would also replay the recorded
events into it; its clients are then removed instead of being left with
a device that never reports the error.
ib_active becomes a bool, as the lock now covering it does not cover
is_rep and lag_active, which shared its byte as bit fields.
Events during ib_register_device() are still missed, as the notifier
is registered after it; the next patch fixes that.
Fixes: df097a278c75 ("IB/mlx5: Use the new mlx5 core notifier API")
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/hw/mlx5/main.c | 116 +++++++++++++++++++++++++++++++++--
drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 +++-
2 files changed, 123 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 80399920078edfca8f672fb77514ba6b90966068..8a9e68782a5c449e5565a098e75636d5054ee6c1 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3118,6 +3118,73 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, u32 port, u8 sub_type,
return 0;
}
+#define MLX5_IB_PORT_CONFIG_SUBTYPES \
+ (BIT(MLX5_PORT_CHANGE_SUBTYPE_LID) | \
+ BIT(MLX5_PORT_CHANGE_SUBTYPE_PKEY) | \
+ BIT(MLX5_PORT_CHANGE_SUBTYPE_CLIENT_REREG))
+
+static void record_port_event(struct mlx5_ib_dev *dev, u32 port, u8 sub_type)
+{
+ lockdep_assert_held(&dev->pending_events_lock);
+
+ /*
+ * Only InfiniBand ports are recorded, as all their port events come
+ * through this notifier. Port state changes are not, as unlike
+ * configuration changes they are sensitive to the order in which
+ * ib_core delivers the replayed events.
+ */
+ if (!rdma_is_port_valid(&dev->ib_dev, port) ||
+ mlx5_ib_port_link_layer(&dev->ib_dev, port) !=
+ IB_LINK_LAYER_INFINIBAND ||
+ sub_type >= BITS_PER_LONG ||
+ !(BIT(sub_type) & MLX5_IB_PORT_CONFIG_SUBTYPES))
+ return;
+
+ dev->port[port - 1].pending_port_events |= BIT(sub_type);
+}
+
+static bool mlx5_ib_dev_active(struct mlx5_ib_dev *dev)
+{
+ bool active;
+
+ spin_lock_irq(&dev->pending_events_lock);
+ active = dev->ib_active;
+ spin_unlock_irq(&dev->pending_events_lock);
+
+ return active;
+}
+
+static void mlx5_ib_replay_port_events(struct work_struct *_work)
+{
+ struct mlx5_ib_dev *dev = container_of(_work, struct mlx5_ib_dev,
+ pending_events_work);
+ struct ib_event ibev = { .device = &dev->ib_dev };
+ unsigned long pending;
+ u32 port, sub_type;
+ int err;
+
+ rdma_for_each_port(&dev->ib_dev, port) {
+ spin_lock_irq(&dev->pending_events_lock);
+ pending = dev->port[port - 1].pending_port_events;
+ dev->port[port - 1].pending_port_events = 0;
+ spin_unlock_irq(&dev->pending_events_lock);
+
+ for_each_set_bit(sub_type, &pending, BITS_PER_LONG) {
+ if (!mlx5_ib_dev_active(dev))
+ return;
+
+ if (handle_port_change(dev, port, sub_type, &ibev))
+ continue;
+
+ /* Processed before events queued behind this work */
+ err = ib_dispatch_event_sync(&ibev);
+ if (err)
+ mlx5_ib_warn(dev, "replayed event %d on port %u failed, %d\n",
+ ibev.event, port, err);
+ }
+ }
+}
+
static void mlx5_ib_handle_event(struct work_struct *_work)
{
struct mlx5_ib_event_work *work =
@@ -3161,14 +3228,31 @@ static void mlx5_ib_handle_event(struct work_struct *_work)
static int mlx5_ib_event(struct notifier_block *nb,
unsigned long event, void *param)
{
+ struct mlx5_ib_dev *dev =
+ container_of(nb, struct mlx5_ib_dev, mdev_events);
struct mlx5_ib_event_work *work;
+ unsigned long flags;
+
+ /* Until the device is active, keep only what is to be replayed */
+ spin_lock_irqsave(&dev->pending_events_lock, flags);
+ if (!dev->ib_active) {
+ if (event == MLX5_EVENT_TYPE_PORT_CHANGE) {
+ struct mlx5_eqe *eqe = param;
+
+ record_port_event(dev, mlx5_ib_eqe_port_num(eqe),
+ eqe->sub_type);
+ }
+ spin_unlock_irqrestore(&dev->pending_events_lock, flags);
+ return NOTIFY_OK;
+ }
+ spin_unlock_irqrestore(&dev->pending_events_lock, flags);
work = kmalloc_obj(*work, GFP_ATOMIC);
if (!work)
return NOTIFY_DONE;
INIT_WORK(&work->work, mlx5_ib_handle_event);
- work->dev = container_of(nb, struct mlx5_ib_dev, mdev_events);
+ work->dev = dev;
work->is_slave = false;
work->param = param;
work->event = event;
@@ -3203,6 +3287,7 @@ static void mlx5_ib_handle_sys_error_event(struct work_struct *_work)
container_of(_work, struct mlx5_ib_event_work, work);
struct mlx5_ib_dev *ibdev = work->dev;
struct ib_event ibev;
+ bool active;
ibev.event = IB_EVENT_DEVICE_FATAL;
mlx5_ib_handle_internal_error(ibdev);
@@ -3214,10 +3299,13 @@ static void mlx5_ib_handle_sys_error_event(struct work_struct *_work)
goto out;
}
- if (ibdev->ib_active)
- ib_dispatch_event(&ibev);
-
+ spin_lock_irq(&ibdev->pending_events_lock);
+ active = ibdev->ib_active;
ibdev->ib_active = false;
+ spin_unlock_irq(&ibdev->pending_events_lock);
+
+ if (active)
+ ib_dispatch_event(&ibev);
out:
kfree(work);
}
@@ -4457,6 +4545,8 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev)
}
mutex_init(&dev->lb.mutex);
+ spin_lock_init(&dev->pending_events_lock);
+ INIT_WORK(&dev->pending_events_work, mlx5_ib_replay_port_events);
err = mlx5r_cmd_query_special_mkeys(dev);
if (err)
@@ -5024,13 +5114,31 @@ static void mlx5_ib_stage_dev_notifier_cleanup(struct mlx5_ib_dev *dev)
static int mlx5_ib_stage_active_init(struct mlx5_ib_dev *dev)
{
+ /*
+ * Fail a device that hit a system error during probe, as the error
+ * handler may already have run. The replay is queued in the same
+ * critical section, so that it runs before any event the notifier
+ * queues once it sees the device active.
+ */
+ spin_lock_irq(&dev->pending_events_lock);
+ if (dev->mdev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR) {
+ spin_unlock_irq(&dev->pending_events_lock);
+ return -EIO;
+ }
dev->ib_active = true;
+ queue_work(mlx5_ib_event_wq, &dev->pending_events_work);
+ spin_unlock_irq(&dev->pending_events_lock);
+
return 0;
}
static void mlx5_ib_stage_active_cleanup(struct mlx5_ib_dev *dev)
{
+ spin_lock_irq(&dev->pending_events_lock);
dev->ib_active = false;
+ spin_unlock_irq(&dev->pending_events_lock);
+
+ cancel_work_sync(&dev->pending_events_work);
}
void __mlx5_ib_remove(struct mlx5_ib_dev *dev,
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 3884e5524e7dfb2a37515a6f4e8caaa82224c964..7e1e0bcd096c994473dcfe2db5655b47bd48915e 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -860,6 +860,8 @@ struct mlx5_ib_port {
struct mlx5_ib_dbg_cc_params *dbg_cc_params;
struct mlx5_roce roce;
struct mlx5_eswitch_rep *rep;
+ /* Port change subtypes to replay, protected by pending_events_lock */
+ unsigned long pending_port_events;
#ifdef CONFIG_MLX5_MACSEC
struct mlx5_reserved_gids *reserved_gids;
#endif
@@ -1100,13 +1102,21 @@ struct mlx5_ib_dev {
*/
struct mutex data_direct_lock;
struct notifier_block mdev_events;
+ /* Protects ib_active writes and the ports' pending_port_events */
+ spinlock_t pending_events_lock;
+ struct work_struct pending_events_work;
struct notifier_block sys_error_events;
struct notifier_block lag_events;
int num_ports;
/* serialize update of capability mask
*/
struct mutex cap_mask_mutex;
- u8 ib_active:1;
+ /*
+ * Written under pending_events_lock, and read under it when the read
+ * must be ordered with activation or with port event recording and
+ * replay.
+ */
+ bool ib_active;
u8 is_rep:1;
u8 lag_active:1;
u8 fill_delay;
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH rdma-next 9/9] RDMA/mlx5: Register event notifier before IB registration
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
` (7 preceding siblings ...)
2026-11-22 17:39 ` [PATCH rdma-next 8/9] RDMA/mlx5: Replay port configuration events after probe Edward Srouji
@ 2026-11-22 17:39 ` Edward Srouji
2026-11-22 17:58 ` [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe sashiko-bot
9 siblings, 0 replies; 11+ messages in thread
From: Edward Srouji @ 2026-11-22 17:39 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe, Maor Gottlieb, Chiara Meiohas,
Mark Bloch, Maher Sanalla, Patrisious Haddad
Cc: linux-rdma, linux-kernel, Edward Srouji, Michael Gur
The device event notifier is registered after ib_register_device(),
so a port change during registration is missed once ib_core or a
client has read the port configuration.
Register it before ib_register_device(), as the system error notifier
already is. Until the device is active the notifier queues no handler,
so the P_Key change work still cannot run while ib_mad creates or
destroys the GSI QPs.
Commit ede132a5cf55 ("RDMA/mlx5: Move events notifier registration to
be after device registration") moved the notifier after registration
so that it is unregistered before the GSI QPs are destroyed. What has
to come first is stopping its work. A new stage,
MLX5_IB_STAGE_EVENT_FLUSH, cleaned up right after MLX5_IB_STAGE_ACTIVE
deactivates the device, flushes mlx5_ib_event_wq and then cancels the
P_Key change work, so a handler queued while the device was active can
no longer rearm it, the race that commit narrowed. A handler that runs
later, queued by a callback that saw the device active or by a slave
port, stops at the ib_active test now at the top of
mlx5_ib_handle_event(), which replaces the tests before dispatching.
MACsec event registration keeps its place, in a stage of its own.
The workqueue is shared by all mlx5_ib devices, and a flush may wait
for another device's replay. That should not deadlock: the replay
sleeps only on its own device's firmware commands, cache and event
handlers, and the slave port handlers on mlx5_ib_multiport_mutex,
which no flush holds; none of these take the E-switch reps_lock held
across representor removal.
Fixes: ede132a5cf55 ("RDMA/mlx5: Move events notifier registration to be after device registration")
Fixes: 7722f47e71e5 ("IB/mlx5: Create GSI transmission QPs when P_Key table is changed")
Reviewed-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
drivers/infiniband/hw/mlx5/main.c | 72 ++++++++++++++++++++++++++----------
drivers/infiniband/hw/mlx5/mlx5_ib.h | 4 +-
2 files changed, 55 insertions(+), 21 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 8a9e68782a5c449e5565a098e75636d5054ee6c1..1e58aa132c5b5c6d6de694f298fe1d6ed11e4cd1 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3075,14 +3075,12 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, u32 port, u8 sub_type,
switch (sub_type) {
case MLX5_PORT_CHANGE_SUBTYPE_ACTIVE:
case MLX5_PORT_CHANGE_SUBTYPE_DOWN:
- case MLX5_PORT_CHANGE_SUBTYPE_INITIALIZED:
- if (ibdev->ib_active) {
- struct ib_event speed_event = {};
+ case MLX5_PORT_CHANGE_SUBTYPE_INITIALIZED: {
+ struct ib_event speed_event = {};
- speed_event.device = &ibdev->ib_dev;
- speed_event.event = IB_EVENT_DEVICE_SPEED_CHANGE;
- ib_dispatch_event(&speed_event);
- }
+ speed_event.device = &ibdev->ib_dev;
+ speed_event.event = IB_EVENT_DEVICE_SPEED_CHANGE;
+ ib_dispatch_event(&speed_event);
/* In RoCE, port up/down events are handled in
* mlx5_netdev_event().
@@ -3094,6 +3092,7 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, u32 port, u8 sub_type,
ibev->event = (sub_type == MLX5_PORT_CHANGE_SUBTYPE_ACTIVE) ?
IB_EVENT_PORT_ACTIVE : IB_EVENT_PORT_ERR;
break;
+ }
case MLX5_PORT_CHANGE_SUBTYPE_LID:
ibev->event = IB_EVENT_LID_CHANGE;
@@ -3200,6 +3199,15 @@ static void mlx5_ib_handle_event(struct work_struct *_work)
ibdev = work->dev;
}
+ /*
+ * Teardown may have cancelled the work the handlers below rearm. A
+ * device seen active here stays usable until this returns: the system
+ * error handler runs on this same ordered workqueue, and teardown
+ * flushes it before releasing anything used below.
+ */
+ if (!mlx5_ib_dev_active(ibdev))
+ goto out;
+
switch (work->event) {
case MLX5_EVENT_TYPE_PORT_CHANGE: {
struct mlx5_eqe *eqe = work->param;
@@ -3217,9 +3225,7 @@ static void mlx5_ib_handle_event(struct work_struct *_work)
}
ibev.device = &ibdev->ib_dev;
-
- if (ibdev->ib_active)
- ib_dispatch_event(&ibev);
+ ib_dispatch_event(&ibev);
out:
kfree(work);
@@ -5095,18 +5101,32 @@ static int mlx5_ib_stage_dev_notifier_init(struct mlx5_ib_dev *dev)
dev->mdev_events.notifier_call = mlx5_ib_event;
mlx5_notifier_register(dev->mdev, &dev->mdev_events);
- mlx5r_macsec_event_register(dev);
-
return 0;
}
static void mlx5_ib_stage_dev_notifier_cleanup(struct mlx5_ib_dev *dev)
+{
+ mlx5_notifier_unregister(dev->mdev, &dev->mdev_events);
+ flush_workqueue(mlx5_ib_event_wq);
+}
+
+static int mlx5_ib_stage_macsec_notifier_init(struct mlx5_ib_dev *dev)
+{
+ mlx5r_macsec_event_register(dev);
+ return 0;
+}
+
+static void mlx5_ib_stage_macsec_notifier_cleanup(struct mlx5_ib_dev *dev)
+{
+ mlx5r_macsec_event_unregister(dev);
+}
+
+static void mlx5_ib_stage_event_flush_cleanup(struct mlx5_ib_dev *dev)
{
struct mlx5_ib_resources *devr = &dev->devr;
int port;
- mlx5r_macsec_event_unregister(dev);
- mlx5_notifier_unregister(dev->mdev, &dev->mdev_events);
+ flush_workqueue(mlx5_ib_event_wq);
for (port = 0; port < ARRAY_SIZE(devr->ports); ++port)
cancel_work_sync(&devr->ports[port].pkey_change_work);
@@ -5230,12 +5250,15 @@ static const struct mlx5_ib_profile pf_profile = {
STAGE_CREATE(MLX5_IB_STAGE_SYS_ERROR_NOTIFIER,
mlx5_ib_stage_sys_error_notifier_init,
mlx5_ib_stage_sys_error_notifier_cleanup),
- STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
- mlx5_ib_stage_ib_reg_init,
- mlx5_ib_stage_ib_reg_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_DEVICE_NOTIFIER,
mlx5_ib_stage_dev_notifier_init,
mlx5_ib_stage_dev_notifier_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
+ mlx5_ib_stage_ib_reg_init,
+ mlx5_ib_stage_ib_reg_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_MACSEC_NOTIFIER,
+ mlx5_ib_stage_macsec_notifier_init,
+ mlx5_ib_stage_macsec_notifier_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_POST_IB_REG_UMR,
mlx5_ib_stage_post_ib_reg_umr_init,
NULL),
@@ -5245,6 +5268,9 @@ static const struct mlx5_ib_profile pf_profile = {
STAGE_CREATE(MLX5_IB_STAGE_RESTRACK,
mlx5_ib_restrack_init,
NULL),
+ STAGE_CREATE(MLX5_IB_STAGE_EVENT_FLUSH,
+ NULL,
+ mlx5_ib_stage_event_flush_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_ACTIVE,
mlx5_ib_stage_active_init,
mlx5_ib_stage_active_cleanup),
@@ -5293,12 +5319,15 @@ const struct mlx5_ib_profile raw_eth_profile = {
STAGE_CREATE(MLX5_IB_STAGE_SYS_ERROR_NOTIFIER,
mlx5_ib_stage_sys_error_notifier_init,
mlx5_ib_stage_sys_error_notifier_cleanup),
- STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
- mlx5_ib_stage_ib_reg_init,
- mlx5_ib_stage_ib_reg_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_DEVICE_NOTIFIER,
mlx5_ib_stage_dev_notifier_init,
mlx5_ib_stage_dev_notifier_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
+ mlx5_ib_stage_ib_reg_init,
+ mlx5_ib_stage_ib_reg_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_MACSEC_NOTIFIER,
+ mlx5_ib_stage_macsec_notifier_init,
+ mlx5_ib_stage_macsec_notifier_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_POST_IB_REG_UMR,
mlx5_ib_stage_post_ib_reg_umr_init,
NULL),
@@ -5308,6 +5337,9 @@ const struct mlx5_ib_profile raw_eth_profile = {
STAGE_CREATE(MLX5_IB_STAGE_RESTRACK,
mlx5_ib_restrack_init,
NULL),
+ STAGE_CREATE(MLX5_IB_STAGE_EVENT_FLUSH,
+ NULL,
+ mlx5_ib_stage_event_flush_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_ACTIVE,
mlx5_ib_stage_active_init,
mlx5_ib_stage_active_cleanup),
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 7e1e0bcd096c994473dcfe2db5655b47bd48915e..53ca49e6c4b4b6c7bedd3a1cba57a2dd5fa5093f 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -937,11 +937,13 @@ enum mlx5_ib_stages {
MLX5_IB_STAGE_PRE_IB_REG_UMR,
MLX5_IB_STAGE_WHITELIST_UID,
MLX5_IB_STAGE_SYS_ERROR_NOTIFIER,
- MLX5_IB_STAGE_IB_REG,
MLX5_IB_STAGE_DEVICE_NOTIFIER,
+ MLX5_IB_STAGE_IB_REG,
+ MLX5_IB_STAGE_MACSEC_NOTIFIER,
MLX5_IB_STAGE_POST_IB_REG_UMR,
MLX5_IB_STAGE_DELAY_DROP,
MLX5_IB_STAGE_RESTRACK,
+ MLX5_IB_STAGE_EVENT_FLUSH,
MLX5_IB_STAGE_ACTIVE,
MLX5_IB_STAGE_MAX,
};
--
2.49.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe
2026-11-22 17:39 [PATCH rdma-next 0/9] RDMA/mlx5: Don't lose port configuration events during probe Edward Srouji
` (8 preceding siblings ...)
2026-11-22 17:39 ` [PATCH rdma-next 9/9] RDMA/mlx5: Register event notifier before IB registration Edward Srouji
@ 2026-11-22 17:58 ` sashiko-bot
9 siblings, 0 replies; 11+ messages in thread
From: sashiko-bot @ 2026-11-22 17:58 UTC (permalink / raw)
To: Edward Srouji; +Cc: linux-rdma
Sashiko has reviewed this 9-patch series and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261122-mlx5-port-events-probe-v1-0-0d54e3d53c61@nvidia.com
^ permalink raw reply [flat|nested] 11+ messages in thread