* [PATCH] RDMA/hfi1: free CPU to SDMA map nodes after an RCU grace period
@ 2026-11-16 13:09 Ziheng Wei
2026-11-16 13:26 ` sashiko-bot
0 siblings, 1 reply; 2+ messages in thread
From: Ziheng Wei @ 2026-11-16 13:09 UTC (permalink / raw)
To: Dennis Dalessandro, Jason Gunthorpe, Leon Romanovsky
Cc: linux-rdma, linux-kernel, stable
sdma_select_user_engine() looks up the per-CPU sdma_rht_node in
dd->sdma_rht under rcu_read_lock() and reads rht_node->map[vl],
map->mask and map->sde[]. sdma_set_cpu_to_sde_map(), called when the
cpu_list sysfs attribute of an SDMA engine is written, removes a node
whose maps have become empty with rhashtable_remove_fast() and then
kfree()s the maps and the node right away. There is no RCU grace
period between the removal and the free, so a user SDMA submission on
that CPU can read the node and the map after they have been freed. If
the memory has been reused, the map pointer and the engine pointer read
from it are arbitrary, and the caller goes on to use that engine.
Writing cpu_list requires CAP_SYS_ADMIN. The reader is reachable by any
user that can open /dev/hfi1_N, when the task submitting user SDMA
requests is pinned to a single CPU. So this needs an administrator to
change the SDMA engine CPU lists while jobs are running.
The debugfs dump in sdma_seqfile_dump_cpu_list() has the same problem,
and in addition does not hold the RCU read lock at all while walking
the node, because rhashtable_lookup_fast() drops it before returning.
Free the node and its maps from an RCU callback, as is already done for
dd->sdma_map, and do the debugfs lookup and walk inside an RCU read-side
critical section. hfi1_free_devdata() calls rcu_barrier() before
sdma_clean(), so no callback can still be pending when the hash table
is destroyed.
Found by LLM-assisted static analysis. Not reproduced, as I have no
Omni-Path hardware; build tested only.
Fixes: 0cb2aa690c7e ("IB/hfi1: Add sysfs interface for affinity setup")
Fixes: af3674d62d34 ("IB/hfi1: Add new debugfs sdma_cpu_list file")
Cc: stable@vger.kernel.org
Signed-off-by: Ziheng Wei <zihengw03@163.com>
---
Notes:
The changelog was drafted with an AI assistant from my analysis;
I have reviewed the code, the fix and the text.
drivers/infiniband/hw/hfi1/sdma.c | 45 ++++++++++++++++++-------------
1 file changed, 27 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/sdma.c b/drivers/infiniband/hw/hfi1/sdma.c
index a71fc8f2d..bf9e362f1 100644
--- a/drivers/infiniband/hw/hfi1/sdma.c
+++ b/drivers/infiniband/hw/hfi1/sdma.c
@@ -801,6 +801,7 @@ struct sdma_rht_node {
unsigned long cpu_id;
struct sdma_rht_map_elem *map[HFI1_MAX_VLS_SUPPORTED];
struct rhash_head node;
+ struct rcu_head rcu;
};
#define NR_CPUS_HINT 192
@@ -886,6 +887,22 @@ static void sdma_cleanup_sde_map(struct sdma_rht_map_elem *map,
}
}
+static void sdma_rht_free(void *ptr, void *arg)
+{
+ struct sdma_rht_node *rht_node = ptr;
+ int i;
+
+ for (i = 0; i < HFI1_MAX_VLS_SUPPORTED; i++)
+ kfree(rht_node->map[i]);
+
+ kfree(rht_node);
+}
+
+static void sdma_rht_free_rcu(struct rcu_head *head)
+{
+ sdma_rht_free(container_of(head, struct sdma_rht_node, rcu), NULL);
+}
+
/*
* Prevents concurrent reads and writes of the sdma engine cpu_mask
*/
@@ -1027,10 +1044,11 @@ ssize_t sdma_set_cpu_to_sde_map(struct sdma_engine *sde, const char *buf,
sdma_rht_params);
WARN_ON(ret);
- for (i = 0; i < HFI1_MAX_VLS_SUPPORTED; i++)
- kfree(rht_node->map[i]);
-
- kfree(rht_node);
+ /*
+ * sdma_select_user_engine() may still be
+ * using the node under rcu_read_lock().
+ */
+ call_rcu(&rht_node->rcu, sdma_rht_free_rcu);
}
}
}
@@ -1055,17 +1073,6 @@ ssize_t sdma_get_cpu_to_sde_map(struct sdma_engine *sde, char *buf)
return strnlen(buf, PAGE_SIZE);
}
-static void sdma_rht_free(void *ptr, void *arg)
-{
- struct sdma_rht_node *rht_node = ptr;
- int i;
-
- for (i = 0; i < HFI1_MAX_VLS_SUPPORTED; i++)
- kfree(rht_node->map[i]);
-
- kfree(rht_node);
-}
-
/**
* sdma_seqfile_dump_cpu_list() - debugfs dump the cpu to sdma mappings
* @s: seq file
@@ -1081,10 +1088,10 @@ void sdma_seqfile_dump_cpu_list(struct seq_file *s,
struct sdma_rht_node *rht_node;
int i, j;
- rht_node = rhashtable_lookup_fast(dd->sdma_rht, &cpuid,
- sdma_rht_params);
+ rcu_read_lock();
+ rht_node = rhashtable_lookup(dd->sdma_rht, &cpuid, sdma_rht_params);
if (!rht_node)
- return;
+ goto out;
seq_printf(s, "cpu%3lu: ", cpuid);
for (i = 0; i < HFI1_MAX_VLS_SUPPORTED; i++) {
@@ -1107,6 +1114,8 @@ void sdma_seqfile_dump_cpu_list(struct seq_file *s,
}
seq_puts(s, "\n");
+out:
+ rcu_read_unlock();
}
/*
base-commit: 97d73c8251226cf27cc6656b4c542869bb8c69fa
--
2.43.0
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH] RDMA/hfi1: free CPU to SDMA map nodes after an RCU grace period
2026-11-16 13:09 [PATCH] RDMA/hfi1: free CPU to SDMA map nodes after an RCU grace period Ziheng Wei
@ 2026-11-16 13:26 ` sashiko-bot
0 siblings, 0 replies; 2+ messages in thread
From: sashiko-bot @ 2026-11-16 13:26 UTC (permalink / raw)
To: Ziheng Wei; +Cc: linux-rdma
> sdma_select_user_engine() looks up the per-CPU sdma_rht_node in
> dd->sdma_rht under rcu_read_lock() and reads rht_node->map[vl],
> map->mask and map->sde[]. sdma_set_cpu_to_sde_map(), called when the
> cpu_list sysfs attribute of an SDMA engine is written, removes a node
> whose maps have become empty with rhashtable_remove_fast() and then
> kfree()s the maps and the node right away. There is no RCU grace
> period between the removal and the free, so a user SDMA submission on
> that CPU can read the node and the map after they have been freed. If
> the memory has been reused, the map pointer and the engine pointer read
> from it are arbitrary, and the caller goes on to use that engine.
>
> Writing cpu_list requires CAP_SYS_ADMIN. The reader is reachable by any
> user that can open /dev/hfi1_N, when the task submitting user SDMA
> requests is pinned to a single CPU. So this needs an administrator to
> change the SDMA engine CPU lists while jobs are running.
> [ ... ]
>
> Fixes: 0cb2aa690c7e ("IB/hfi1: Add sysfs interface for affinity setup")
> Fixes: af3674d62d34 ("IB/hfi1: Add new debugfs sdma_cpu_list file")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ziheng Wei <zihengw03@163.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261116130940.1629295-1-zihengw03@163.com?part=1
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-11-16 13:26 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 13:09 [PATCH] RDMA/hfi1: free CPU to SDMA map nodes after an RCU grace period Ziheng Wei
2026-11-16 13:26 ` sashiko-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox