* [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
Ultra Ethernet defines a reliable unconnected transport. A queue pair
sends to any peer in its job without a connection, incoming traffic is
demultiplexed on a JobID rather than on a connection, and what a device
orders is far weaker than IB or RoCE and varies between devices. None of
that has a home in the current uverbs ABI.
This series adds the generic half. The objects, methods, and attributes
any UET device implements. No driver is part of it, though two already
exist against this ABI. A reference device model that implements all of
it, and HPE's Slingshot driver implementing a large subset. That is the
argument for ib_core carrying this. What is device specific should come
from private interfaces per vendor.
Patch 1 widens the uverbs object key. The two new core objects would
otherwise leave drivers no free object keys.
Patches 2-4 expose the transport. The two UET encapsulations as GID
types, the capabilities and limits a device advertises, and IB_QPT_RU
with its 32-bit queue pair number.
Patch 5 widens address handles, which an unconnected peer needs since
it is named by both a network address and a queue pair number.
Patch 6 adds a query for what a queue pair type actually orders.
Patch 7 adds jobs and job keys, the objects an unconnected transport
demultiplexes and authorizes on. A job can be exported to a file
descriptor, so an unprivileged process can join one set up for it.
Patches 8-10 let a queue pair be created ready to send, since a
transport with nothing to negotiate gains nothing from the modify
sequence. Patch 8 is a no-functional-change refactor pulling the
attribute copy out of modify_qp. Patch 9 lets create take those
attributes. Patch 10 adds the attributes a reliable unconnected
transport needs.
Patches 11-13 cover memory registration with a job binding, a chosen
key, derived regions, 64-bit keys for devices whose keys do not fit
in 32 bits, and attaching a region to one queue pair so the network
reaches it only over that QP.
New attributes are optional, so existing userspace is unaffected and a
device without a feature reports it absent rather than failing. The
attributes that restrict access are sent mandatory by userspace, so a
kernel that does not know one refuses the request rather than creating
an object with less protection than was asked for.
Tested against a reference device implementing the ABI, and against rxe
to confirm nothing already upstream regressed. The userspace half is a
matching rdma-core series.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
Eric Davis (13):
RDMA/core: widen the uverbs object key
RDMA/core: add the UET GID types
RDMA/core: report the reliable unconnected device capabilities
RDMA/core: add the reliable unconnected queue pair type
RDMA/core: extended address handle attributes
RDMA/core: query the semantics a queue pair type guarantees
RDMA/core: job and job key objects
RDMA/uverbs: factor out the queue pair attribute copy
RDMA/core: create a queue pair ready to send
RDMA/core: queue pair attributes for unconnected transports
RDMA/core: extended memory registration
RDMA/core: 64-bit memory keys
RDMA/core: attach a memory region to a queue pair
drivers/infiniband/core/Makefile | 3 +-
drivers/infiniband/core/cache.c | 12 +-
drivers/infiniband/core/cma.c | 4 +
drivers/infiniband/core/device.c | 16 +-
drivers/infiniband/core/nldev.c | 4 +
drivers/infiniband/core/rdma_core.h | 2 +
drivers/infiniband/core/restrack.c | 17 +-
drivers/infiniband/core/roce_gid_mgmt.c | 6 +-
drivers/infiniband/core/user_mad.c | 4 +-
drivers/infiniband/core/uverbs.h | 16 +
drivers/infiniband/core/uverbs_cmd.c | 261 ++++----
drivers/infiniband/core/uverbs_main.c | 5 +
drivers/infiniband/core/uverbs_std_types.c | 73 ++-
drivers/infiniband/core/uverbs_std_types_device.c | 73 ++-
drivers/infiniband/core/uverbs_std_types_job.c | 763 ++++++++++++++++++++++
drivers/infiniband/core/uverbs_std_types_mr.c | 222 ++++++-
drivers/infiniband/core/uverbs_std_types_qp.c | 340 +++++++++-
drivers/infiniband/core/uverbs_uapi.c | 2 +
drivers/infiniband/core/verbs.c | 109 +++-
include/rdma/ib_verbs.h | 242 ++++++-
include/rdma/restrack.h | 8 +
include/rdma/uverbs_ioctl.h | 4 +-
include/uapi/rdma/ib_user_ioctl_cmds.h | 120 ++++
include/uapi/rdma/ib_user_ioctl_verbs.h | 100 +++
include/uapi/rdma/ib_user_verbs.h | 25 +
25 files changed, 2270 insertions(+), 161 deletions(-)
---
base-commit: d37e98844afea39a9614cac0de837701263c90f7
change-id: 20260929-uet-ib-core-d82e941f5e52
Best regards,
--
Eric Davis <eric.davis@broadcom.com>
^ permalink raw reply [flat|nested] 60+ messages in thread* [PATCH rdma-next 01/13] RDMA/core: widen the uverbs object key
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
Two more core objects would leave drivers no free object keys. mlx5
alone uses all ten that remain. Add a bit to the object key.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
include/rdma/uverbs_ioctl.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index 1aaaadb8daf6..f4d93bc3625a 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -144,7 +144,7 @@ enum uapi_radix_data {
UVERBS_API_METHOD_KEY_BITS + UVERBS_API_METHOD_KEY_SHIFT - 1,
UVERBS_API_METHOD_KEY_SHIFT),
- UVERBS_API_OBJ_KEY_BITS = 5,
+ UVERBS_API_OBJ_KEY_BITS = 6,
UVERBS_API_OBJ_KEY_SHIFT =
UVERBS_API_METHOD_KEY_BITS + UVERBS_API_METHOD_KEY_SHIFT,
UVERBS_API_OBJ_KEY_NUM_CORE = 20,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 01/13] RDMA/core: widen the uverbs object key
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
Two more core objects would leave drivers no free object keys. mlx5
alone uses all ten that remain. Add a bit to the object key.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
include/rdma/uverbs_ioctl.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index 1aaaadb8daf6..f4d93bc3625a 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -144,7 +144,7 @@ enum uapi_radix_data {
UVERBS_API_METHOD_KEY_BITS + UVERBS_API_METHOD_KEY_SHIFT - 1,
UVERBS_API_METHOD_KEY_SHIFT),
- UVERBS_API_OBJ_KEY_BITS = 5,
+ UVERBS_API_OBJ_KEY_BITS = 6,
UVERBS_API_OBJ_KEY_SHIFT =
UVERBS_API_METHOD_KEY_BITS + UVERBS_API_METHOD_KEY_SHIFT,
UVERBS_API_OBJ_KEY_NUM_CORE = 20,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 01/13] RDMA/core: widen the uverbs object key
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:29 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:29 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> Two more core objects would leave drivers no free object keys. mlx5
> alone uses all ten that remain. Add a bit to the object key.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=1
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
Ultra Ethernet runs over two encapsulations: UDP and IP. Each becomes
a new GID type. A port can then hold both UET and RoCE GIDs, with the
transport following from the GID a queue pair is given, the driver
enforces that.
These UET ports build their GID table from a netdev, as RoCE does. Added
rdma_port_gids_from_netdev() and use it where the code tested for RoCE
alone.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/cache.c | 12 +++++----
drivers/infiniband/core/cma.c | 4 +++
drivers/infiniband/core/device.c | 2 +-
drivers/infiniband/core/nldev.c | 2 ++
drivers/infiniband/core/roce_gid_mgmt.c | 6 +++--
drivers/infiniband/core/verbs.c | 4 +++
include/rdma/ib_verbs.h | 48 ++++++++++++++++++++++++++++++---
include/uapi/rdma/ib_user_ioctl_verbs.h | 2 ++
8 files changed, 69 insertions(+), 11 deletions(-)
diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
index 647a547e2d7f..d5e17a36f186 100644
--- a/drivers/infiniband/core/cache.c
+++ b/drivers/infiniband/core/cache.c
@@ -139,6 +139,8 @@ static const char * const gid_type_str[] = {
[IB_GID_TYPE_IB] = "IB/RoCE v1",
[IB_GID_TYPE_ROCE] = "IB/RoCE v1",
[IB_GID_TYPE_ROCE_UDP_ENCAP] = "RoCE v2",
+ [IB_GID_TYPE_UET_UDP] = "UET UDP",
+ [IB_GID_TYPE_UET_IP] = "UET IP",
};
const char *ib_cache_gid_type_str(enum ib_gid_type gid_type)
@@ -392,10 +394,10 @@ static void del_gid(struct ib_device *ib_dev, u32 port,
write_lock_irq(&table->rwlock);
entry = table->data_vec[ix];
entry->state = GID_TABLE_ENTRY_PENDING_DEL;
- /*
- * For non RoCE protocol, GID entry slot is ready to use.
+ /* Where the GID table is not derived from a netdev, the entry slot is
+ * ready to use.
*/
- if (!rdma_protocol_roce(ib_dev, port))
+ if (!rdma_port_gids_from_netdev(ib_dev, port))
table->data_vec[ix] = NULL;
write_unlock_irq(&table->rwlock);
@@ -447,7 +449,7 @@ static int add_modify_gid(struct ib_gid_table *table,
if (!entry)
return -ENOMEM;
- if (rdma_protocol_roce(attr->device, attr->port_num)) {
+ if (rdma_port_gids_from_netdev(attr->device, attr->port_num)) {
ret = add_roce_gid(entry);
if (ret)
goto done;
@@ -1459,7 +1461,7 @@ ib_cache_update(struct ib_device *device, u32 port, bool update_gids,
goto err;
}
- if (!rdma_protocol_roce(device, port) && update_gids) {
+ if (!rdma_port_gids_from_netdev(device, port) && update_gids) {
ret = config_non_roce_gid_cache(device, port,
tprops);
if (ret)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 337a49d1acf7..7dedf93ac465 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -293,6 +293,10 @@ int cma_set_default_gid_type(struct cma_device *cma_dev,
if (!rdma_is_port_valid(cma_dev->device, port))
return -EINVAL;
+ if (default_gid_type == IB_GID_TYPE_UET_UDP ||
+ default_gid_type == IB_GID_TYPE_UET_IP)
+ return -EINVAL;
+
if (default_gid_type == IB_GID_TYPE_IB &&
rdma_protocol_roce_eth_encap(cma_dev->device, port))
default_gid_type = IB_GID_TYPE_ROCE;
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 29af9fe860e2..7c65a9373701 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -2567,7 +2567,7 @@ void ib_enum_roce_netdev(struct ib_device *ib_dev,
u32 port;
rdma_for_each_port (ib_dev, port)
- if (rdma_protocol_roce(ib_dev, port)) {
+ if (rdma_port_gids_from_netdev(ib_dev, port)) {
struct net_device *idev =
ib_device_get_netdev(ib_dev, port);
diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 849a5db54e1c..91b7be84b15f 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -350,6 +350,8 @@ static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL, "iw");
else if (rdma_protocol_roce(device, port))
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL, "roce");
+ else if (rdma_protocol_uet(device, port))
+ ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL, "uet");
else if (rdma_protocol_usnic(device, port))
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL,
"usnic");
diff --git a/drivers/infiniband/core/roce_gid_mgmt.c b/drivers/infiniband/core/roce_gid_mgmt.c
index 4a2a4638b476..214af921341a 100644
--- a/drivers/infiniband/core/roce_gid_mgmt.c
+++ b/drivers/infiniband/core/roce_gid_mgmt.c
@@ -75,6 +75,8 @@ static const struct {
} PORT_CAP_TO_GID_TYPE[] = {
{rdma_protocol_roce_eth_encap, IB_GID_TYPE_ROCE},
{rdma_protocol_roce_udp_encap, IB_GID_TYPE_ROCE_UDP_ENCAP},
+ {rdma_protocol_uet_udp, IB_GID_TYPE_UET_UDP},
+ {rdma_protocol_uet_ip, IB_GID_TYPE_UET_IP},
};
#define CAP_TO_GID_TABLE_SIZE ARRAY_SIZE(PORT_CAP_TO_GID_TYPE)
@@ -84,7 +86,7 @@ unsigned long roce_gid_type_mask_support(struct ib_device *ib_dev, u32 port)
int i;
unsigned int ret_flags = 0;
- if (!rdma_protocol_roce(ib_dev, port))
+ if (!rdma_port_gids_from_netdev(ib_dev, port))
return 1UL << IB_GID_TYPE_IB;
for (i = 0; i < CAP_TO_GID_TABLE_SIZE; i++)
@@ -526,7 +528,7 @@ void rdma_roce_rescan_port(struct ib_device *ib_dev, u32 port)
{
struct net_device *ndev = NULL;
- if (rdma_protocol_roce(ib_dev, port)) {
+ if (rdma_port_gids_from_netdev(ib_dev, port)) {
ndev = ib_device_get_netdev(ib_dev, port);
if (!ndev)
return;
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index edf62652f9d3..24db88e6d3fa 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -862,6 +862,10 @@ int ib_init_ah_attr_from_wc(struct ib_device *device, u32 port_num,
might_sleep();
+ if (rdma_protocol_uet(device, port_num) &&
+ !rdma_protocol_roce(device, port_num))
+ return -EPROTONOSUPPORT;
+
memset(ah_attr, 0, sizeof *ah_attr);
ah_attr->type = rdma_ah_find_type(device, port_num);
if (rdma_cap_eth_ah(device, port_num)) {
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index d6eaa3fee66e..4e3a5c38cbe8 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -145,6 +145,8 @@ enum ib_gid_type {
IB_GID_TYPE_IB = IB_UVERBS_GID_TYPE_IB,
IB_GID_TYPE_ROCE = IB_UVERBS_GID_TYPE_ROCE_V1,
IB_GID_TYPE_ROCE_UDP_ENCAP = IB_UVERBS_GID_TYPE_ROCE_V2,
+ IB_GID_TYPE_UET_UDP = IB_UVERBS_GID_TYPE_UET_UDP,
+ IB_GID_TYPE_UET_IP = IB_UVERBS_GID_TYPE_UET_IP,
IB_GID_TYPE_SIZE
};
@@ -664,10 +666,16 @@ void rdma_free_hw_stats_struct(struct rdma_hw_stats *stats);
#define RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP 0x00800000
#define RDMA_CORE_CAP_PROT_RAW_PACKET 0x01000000
#define RDMA_CORE_CAP_PROT_USNIC 0x02000000
+#define RDMA_CORE_CAP_PROT_UET_UDP 0x04000000
+#define RDMA_CORE_CAP_PROT_UET_IP 0x08000000
+
+#define RDMA_CORE_CAP_PROT_UET (RDMA_CORE_CAP_PROT_UET_UDP \
+ | RDMA_CORE_CAP_PROT_UET_IP)
#define RDMA_CORE_PORT_IB_GRH_REQUIRED (RDMA_CORE_CAP_IB_GRH_REQUIRED \
| RDMA_CORE_CAP_PROT_ROCE \
- | RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP)
+ | RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP \
+ | RDMA_CORE_CAP_PROT_UET)
#define RDMA_CORE_PORT_IBA_IB (RDMA_CORE_CAP_PROT_IB \
| RDMA_CORE_CAP_IB_MAD \
@@ -691,6 +699,11 @@ void rdma_free_hw_stats_struct(struct rdma_hw_stats *stats);
#define RDMA_CORE_PORT_INTEL_OPA (RDMA_CORE_PORT_IBA_IB \
| RDMA_CORE_CAP_OPA_MAD)
+#define RDMA_CORE_PORT_UET_UDP (RDMA_CORE_CAP_PROT_UET_UDP \
+ | RDMA_CORE_CAP_ETH_AH)
+#define RDMA_CORE_PORT_UET_IP (RDMA_CORE_CAP_PROT_UET_IP \
+ | RDMA_CORE_CAP_ETH_AH)
+
#define RDMA_CORE_PORT_RAW_PACKET (RDMA_CORE_CAP_PROT_RAW_PACKET)
#define RDMA_CORE_PORT_USNIC (RDMA_CORE_CAP_PROT_USNIC)
@@ -3407,6 +3420,34 @@ static inline bool rdma_protocol_roce_eth_encap(const struct ib_device *device,
RDMA_CORE_CAP_PROT_ROCE;
}
+static inline bool rdma_protocol_uet(const struct ib_device *device,
+ u32 port_num)
+{
+ return device->port_data[port_num].immutable.core_cap_flags &
+ RDMA_CORE_CAP_PROT_UET;
+}
+
+static inline bool rdma_protocol_uet_udp(const struct ib_device *device,
+ u32 port_num)
+{
+ return device->port_data[port_num].immutable.core_cap_flags &
+ RDMA_CORE_CAP_PROT_UET_UDP;
+}
+
+static inline bool rdma_protocol_uet_ip(const struct ib_device *device,
+ u32 port_num)
+{
+ return device->port_data[port_num].immutable.core_cap_flags &
+ RDMA_CORE_CAP_PROT_UET_IP;
+}
+
+static inline bool rdma_port_gids_from_netdev(const struct ib_device *device,
+ u32 port_num)
+{
+ return rdma_protocol_roce(device, port_num) ||
+ rdma_protocol_uet(device, port_num);
+}
+
static inline bool rdma_protocol_iwarp(const struct ib_device *device,
u32 port_num)
{
@@ -3677,7 +3718,7 @@ static inline size_t rdma_max_mad_size(const struct ib_device *device,
static inline bool rdma_cap_roce_gid_table(const struct ib_device *device,
u32 port_num)
{
- return rdma_protocol_roce(device, port_num) &&
+ return rdma_port_gids_from_netdev(device, port_num) &&
device->ops.add_gid && device->ops.del_gid;
}
@@ -4984,7 +5025,8 @@ void rdma_move_ah_attr(struct rdma_ah_attr *dest, struct rdma_ah_attr *src);
static inline enum rdma_ah_attr_type rdma_ah_find_type(struct ib_device *dev,
u32 port_num)
{
- if (rdma_protocol_roce(dev, port_num))
+ if (rdma_protocol_roce(dev, port_num) ||
+ rdma_protocol_uet(dev, port_num))
return RDMA_AH_ATTR_TYPE_ROCE;
if (rdma_protocol_ib(dev, port_num)) {
if (rdma_cap_opa_ah(dev, port_num))
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 21f86cc7bb1f..ed72f48960f8 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -263,6 +263,8 @@ enum ib_uverbs_gid_type {
IB_UVERBS_GID_TYPE_IB,
IB_UVERBS_GID_TYPE_ROCE_V1,
IB_UVERBS_GID_TYPE_ROCE_V2,
+ IB_UVERBS_GID_TYPE_UET_UDP,
+ IB_UVERBS_GID_TYPE_UET_IP,
};
struct ib_uverbs_gid_entry {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
Ultra Ethernet runs over two encapsulations: UDP and IP. Each becomes
a new GID type. A port can then hold both UET and RoCE GIDs, with the
transport following from the GID a queue pair is given, the driver
enforces that.
These UET ports build their GID table from a netdev, as RoCE does. Added
rdma_port_gids_from_netdev() and use it where the code tested for RoCE
alone.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/cache.c | 12 +++++----
drivers/infiniband/core/cma.c | 4 +++
drivers/infiniband/core/device.c | 2 +-
drivers/infiniband/core/nldev.c | 2 ++
drivers/infiniband/core/roce_gid_mgmt.c | 6 +++--
drivers/infiniband/core/verbs.c | 4 +++
include/rdma/ib_verbs.h | 48 ++++++++++++++++++++++++++++++---
include/uapi/rdma/ib_user_ioctl_verbs.h | 2 ++
8 files changed, 69 insertions(+), 11 deletions(-)
diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
index 647a547e2d7f..d5e17a36f186 100644
--- a/drivers/infiniband/core/cache.c
+++ b/drivers/infiniband/core/cache.c
@@ -139,6 +139,8 @@ static const char * const gid_type_str[] = {
[IB_GID_TYPE_IB] = "IB/RoCE v1",
[IB_GID_TYPE_ROCE] = "IB/RoCE v1",
[IB_GID_TYPE_ROCE_UDP_ENCAP] = "RoCE v2",
+ [IB_GID_TYPE_UET_UDP] = "UET UDP",
+ [IB_GID_TYPE_UET_IP] = "UET IP",
};
const char *ib_cache_gid_type_str(enum ib_gid_type gid_type)
@@ -392,10 +394,10 @@ static void del_gid(struct ib_device *ib_dev, u32 port,
write_lock_irq(&table->rwlock);
entry = table->data_vec[ix];
entry->state = GID_TABLE_ENTRY_PENDING_DEL;
- /*
- * For non RoCE protocol, GID entry slot is ready to use.
+ /* Where the GID table is not derived from a netdev, the entry slot is
+ * ready to use.
*/
- if (!rdma_protocol_roce(ib_dev, port))
+ if (!rdma_port_gids_from_netdev(ib_dev, port))
table->data_vec[ix] = NULL;
write_unlock_irq(&table->rwlock);
@@ -447,7 +449,7 @@ static int add_modify_gid(struct ib_gid_table *table,
if (!entry)
return -ENOMEM;
- if (rdma_protocol_roce(attr->device, attr->port_num)) {
+ if (rdma_port_gids_from_netdev(attr->device, attr->port_num)) {
ret = add_roce_gid(entry);
if (ret)
goto done;
@@ -1459,7 +1461,7 @@ ib_cache_update(struct ib_device *device, u32 port, bool update_gids,
goto err;
}
- if (!rdma_protocol_roce(device, port) && update_gids) {
+ if (!rdma_port_gids_from_netdev(device, port) && update_gids) {
ret = config_non_roce_gid_cache(device, port,
tprops);
if (ret)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 337a49d1acf7..7dedf93ac465 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -293,6 +293,10 @@ int cma_set_default_gid_type(struct cma_device *cma_dev,
if (!rdma_is_port_valid(cma_dev->device, port))
return -EINVAL;
+ if (default_gid_type == IB_GID_TYPE_UET_UDP ||
+ default_gid_type == IB_GID_TYPE_UET_IP)
+ return -EINVAL;
+
if (default_gid_type == IB_GID_TYPE_IB &&
rdma_protocol_roce_eth_encap(cma_dev->device, port))
default_gid_type = IB_GID_TYPE_ROCE;
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 29af9fe860e2..7c65a9373701 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -2567,7 +2567,7 @@ void ib_enum_roce_netdev(struct ib_device *ib_dev,
u32 port;
rdma_for_each_port (ib_dev, port)
- if (rdma_protocol_roce(ib_dev, port)) {
+ if (rdma_port_gids_from_netdev(ib_dev, port)) {
struct net_device *idev =
ib_device_get_netdev(ib_dev, port);
diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 849a5db54e1c..91b7be84b15f 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -350,6 +350,8 @@ static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL, "iw");
else if (rdma_protocol_roce(device, port))
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL, "roce");
+ else if (rdma_protocol_uet(device, port))
+ ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL, "uet");
else if (rdma_protocol_usnic(device, port))
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_PROTOCOL,
"usnic");
diff --git a/drivers/infiniband/core/roce_gid_mgmt.c b/drivers/infiniband/core/roce_gid_mgmt.c
index 4a2a4638b476..214af921341a 100644
--- a/drivers/infiniband/core/roce_gid_mgmt.c
+++ b/drivers/infiniband/core/roce_gid_mgmt.c
@@ -75,6 +75,8 @@ static const struct {
} PORT_CAP_TO_GID_TYPE[] = {
{rdma_protocol_roce_eth_encap, IB_GID_TYPE_ROCE},
{rdma_protocol_roce_udp_encap, IB_GID_TYPE_ROCE_UDP_ENCAP},
+ {rdma_protocol_uet_udp, IB_GID_TYPE_UET_UDP},
+ {rdma_protocol_uet_ip, IB_GID_TYPE_UET_IP},
};
#define CAP_TO_GID_TABLE_SIZE ARRAY_SIZE(PORT_CAP_TO_GID_TYPE)
@@ -84,7 +86,7 @@ unsigned long roce_gid_type_mask_support(struct ib_device *ib_dev, u32 port)
int i;
unsigned int ret_flags = 0;
- if (!rdma_protocol_roce(ib_dev, port))
+ if (!rdma_port_gids_from_netdev(ib_dev, port))
return 1UL << IB_GID_TYPE_IB;
for (i = 0; i < CAP_TO_GID_TABLE_SIZE; i++)
@@ -526,7 +528,7 @@ void rdma_roce_rescan_port(struct ib_device *ib_dev, u32 port)
{
struct net_device *ndev = NULL;
- if (rdma_protocol_roce(ib_dev, port)) {
+ if (rdma_port_gids_from_netdev(ib_dev, port)) {
ndev = ib_device_get_netdev(ib_dev, port);
if (!ndev)
return;
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index edf62652f9d3..24db88e6d3fa 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -862,6 +862,10 @@ int ib_init_ah_attr_from_wc(struct ib_device *device, u32 port_num,
might_sleep();
+ if (rdma_protocol_uet(device, port_num) &&
+ !rdma_protocol_roce(device, port_num))
+ return -EPROTONOSUPPORT;
+
memset(ah_attr, 0, sizeof *ah_attr);
ah_attr->type = rdma_ah_find_type(device, port_num);
if (rdma_cap_eth_ah(device, port_num)) {
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index d6eaa3fee66e..4e3a5c38cbe8 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -145,6 +145,8 @@ enum ib_gid_type {
IB_GID_TYPE_IB = IB_UVERBS_GID_TYPE_IB,
IB_GID_TYPE_ROCE = IB_UVERBS_GID_TYPE_ROCE_V1,
IB_GID_TYPE_ROCE_UDP_ENCAP = IB_UVERBS_GID_TYPE_ROCE_V2,
+ IB_GID_TYPE_UET_UDP = IB_UVERBS_GID_TYPE_UET_UDP,
+ IB_GID_TYPE_UET_IP = IB_UVERBS_GID_TYPE_UET_IP,
IB_GID_TYPE_SIZE
};
@@ -664,10 +666,16 @@ void rdma_free_hw_stats_struct(struct rdma_hw_stats *stats);
#define RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP 0x00800000
#define RDMA_CORE_CAP_PROT_RAW_PACKET 0x01000000
#define RDMA_CORE_CAP_PROT_USNIC 0x02000000
+#define RDMA_CORE_CAP_PROT_UET_UDP 0x04000000
+#define RDMA_CORE_CAP_PROT_UET_IP 0x08000000
+
+#define RDMA_CORE_CAP_PROT_UET (RDMA_CORE_CAP_PROT_UET_UDP \
+ | RDMA_CORE_CAP_PROT_UET_IP)
#define RDMA_CORE_PORT_IB_GRH_REQUIRED (RDMA_CORE_CAP_IB_GRH_REQUIRED \
| RDMA_CORE_CAP_PROT_ROCE \
- | RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP)
+ | RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP \
+ | RDMA_CORE_CAP_PROT_UET)
#define RDMA_CORE_PORT_IBA_IB (RDMA_CORE_CAP_PROT_IB \
| RDMA_CORE_CAP_IB_MAD \
@@ -691,6 +699,11 @@ void rdma_free_hw_stats_struct(struct rdma_hw_stats *stats);
#define RDMA_CORE_PORT_INTEL_OPA (RDMA_CORE_PORT_IBA_IB \
| RDMA_CORE_CAP_OPA_MAD)
+#define RDMA_CORE_PORT_UET_UDP (RDMA_CORE_CAP_PROT_UET_UDP \
+ | RDMA_CORE_CAP_ETH_AH)
+#define RDMA_CORE_PORT_UET_IP (RDMA_CORE_CAP_PROT_UET_IP \
+ | RDMA_CORE_CAP_ETH_AH)
+
#define RDMA_CORE_PORT_RAW_PACKET (RDMA_CORE_CAP_PROT_RAW_PACKET)
#define RDMA_CORE_PORT_USNIC (RDMA_CORE_CAP_PROT_USNIC)
@@ -3407,6 +3420,34 @@ static inline bool rdma_protocol_roce_eth_encap(const struct ib_device *device,
RDMA_CORE_CAP_PROT_ROCE;
}
+static inline bool rdma_protocol_uet(const struct ib_device *device,
+ u32 port_num)
+{
+ return device->port_data[port_num].immutable.core_cap_flags &
+ RDMA_CORE_CAP_PROT_UET;
+}
+
+static inline bool rdma_protocol_uet_udp(const struct ib_device *device,
+ u32 port_num)
+{
+ return device->port_data[port_num].immutable.core_cap_flags &
+ RDMA_CORE_CAP_PROT_UET_UDP;
+}
+
+static inline bool rdma_protocol_uet_ip(const struct ib_device *device,
+ u32 port_num)
+{
+ return device->port_data[port_num].immutable.core_cap_flags &
+ RDMA_CORE_CAP_PROT_UET_IP;
+}
+
+static inline bool rdma_port_gids_from_netdev(const struct ib_device *device,
+ u32 port_num)
+{
+ return rdma_protocol_roce(device, port_num) ||
+ rdma_protocol_uet(device, port_num);
+}
+
static inline bool rdma_protocol_iwarp(const struct ib_device *device,
u32 port_num)
{
@@ -3677,7 +3718,7 @@ static inline size_t rdma_max_mad_size(const struct ib_device *device,
static inline bool rdma_cap_roce_gid_table(const struct ib_device *device,
u32 port_num)
{
- return rdma_protocol_roce(device, port_num) &&
+ return rdma_port_gids_from_netdev(device, port_num) &&
device->ops.add_gid && device->ops.del_gid;
}
@@ -4984,7 +5025,8 @@ void rdma_move_ah_attr(struct rdma_ah_attr *dest, struct rdma_ah_attr *src);
static inline enum rdma_ah_attr_type rdma_ah_find_type(struct ib_device *dev,
u32 port_num)
{
- if (rdma_protocol_roce(dev, port_num))
+ if (rdma_protocol_roce(dev, port_num) ||
+ rdma_protocol_uet(dev, port_num))
return RDMA_AH_ATTR_TYPE_ROCE;
if (rdma_protocol_ib(dev, port_num)) {
if (rdma_cap_opa_ah(dev, port_num))
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 21f86cc7bb1f..ed72f48960f8 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -263,6 +263,8 @@ enum ib_uverbs_gid_type {
IB_UVERBS_GID_TYPE_IB,
IB_UVERBS_GID_TYPE_ROCE_V1,
IB_UVERBS_GID_TYPE_ROCE_V2,
+ IB_UVERBS_GID_TYPE_UET_UDP,
+ IB_UVERBS_GID_TYPE_UET_IP,
};
struct ib_uverbs_gid_entry {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:35 ` sashiko-bot
2026-10-15 15:57 ` Eric Davis
-1 siblings, 1 reply; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:35 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] RDMA/core: Incomplete UET Address Handle initialization breaks incoming WC
- [High] RDMA/cma: UET paths incorrectly classified as InfiniBand, dropping DMAC
- [High] RDMA/cma: Missing netdev checks cause UET to fallback to InfiniBand logic
--
commit 7479c85e5472864148025ac811bf445ba1cec04d
Author: Eric Davis <eric.davis@broadcom.com>
RDMA/core: add the UET GID types
This commit introduces the UET GID types for UDP and IP encapsulations,
allowing a port to hold both UET and RoCE GIDs. It also adds
rdma_port_gids_from_netdev() to build the GID table from a netdev for
UET ports similarly to how it is done for RoCE.
> diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
> index edf62652f9d3e..24db88e6d3fa0 100644
> --- a/drivers/infiniband/core/verbs.c
> +++ b/drivers/infiniband/core/verbs.c
> @@ -862,6 +862,10 @@ int ib_init_ah_attr_from_wc(struct ib_device *device, u32 port_num,
>
> might_sleep();
>
> + if (rdma_protocol_uet(device, port_num) &&
> + !rdma_protocol_roce(device, port_num))
> + return -EPROTONOSUPPORT;
> +
[Severity: High]
Does this explicitly fail Address Handle initialization for UET-only ports?
Returning -EPROTONOSUPPORT here blocks Address Handle creation for incoming
packets on UET-only ports, preventing the host from responding to connections.
Additionally, for dual-mode ports, this function extracts the IP network
type and passes it to ib_network_to_gid_type(). Because ib_network_to_gid_type()
was not updated to handle UET, it unconditionally maps IPv4 and IPv6 to
IB_GID_TYPE_ROCE_UDP_ENCAP:
include/rdma/ib_verbs.h:ib_network_to_gid_type() {
if (network_type == RDMA_NETWORK_IPV4 ||
network_type == RDMA_NETWORK_IPV6)
return IB_GID_TYPE_ROCE_UDP_ENCAP;
...
}
Could this cause incoming UET packets to be incorrectly mapped to a RoCEv2
GID index, breaking the UET transport response?
[ ... ]
> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index d6eaa3fee66e3..4e3a5c38cbe8b 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
> @@ -145,6 +145,8 @@ enum ib_gid_type {
> IB_GID_TYPE_IB = IB_UVERBS_GID_TYPE_IB,
> IB_GID_TYPE_ROCE = IB_UVERBS_GID_TYPE_ROCE_V1,
> IB_GID_TYPE_ROCE_UDP_ENCAP = IB_UVERBS_GID_TYPE_ROCE_V2,
> + IB_GID_TYPE_UET_UDP = IB_UVERBS_GID_TYPE_UET_UDP,
> + IB_GID_TYPE_UET_IP = IB_UVERBS_GID_TYPE_UET_IP,
> IB_GID_TYPE_SIZE
> };
[Severity: High]
When these new GID types are passed to the Subnet Administrator (SA) layer,
are they incorrectly classified as InfiniBand path records?
In sa_conv_gid_to_pathrec_type() in include/rdma/ib_sa.h, there are no cases
for the new UET GID types, causing them to fall through to the default case:
include/rdma/ib_sa.h:sa_conv_gid_to_pathrec_type() {
...
default:
return SA_PATH_REC_TYPE_IB;
}
Later, when CMA attempts to set the destination MAC address for the path record,
sa_path_set_dmac() checks if the path is RoCE:
include/rdma/ib_sa.h:sa_path_set_dmac() {
if (sa_path_is_roce(rec))
memcpy(rec->roce.dmac, dmac, ETH_ALEN);
...
}
Because the path is classified as IB rather than Ethernet/RoCE, this check
fails. Will this silently discard the destination MAC address, causing outgoing
packets to be dropped due to an uninitialized MAC?
[ ... ]
> @@ -3407,6 +3420,34 @@ static inline bool rdma_protocol_roce_eth_encap(const struct ib_device *device,
> RDMA_CORE_CAP_PROT_ROCE;
> }
>
> +static inline bool rdma_protocol_uet(const struct ib_device *device,
> + u32 port_num)
> +{
> + return device->port_data[port_num].immutable.core_cap_flags &
> + RDMA_CORE_CAP_PROT_UET;
> +}
> +
> +static inline bool rdma_protocol_uet_udp(const struct ib_device *device,
> + u32 port_num)
> +{
> + return device->port_data[port_num].immutable.core_cap_flags &
> + RDMA_CORE_CAP_PROT_UET_UDP;
> +}
> +
> +static inline bool rdma_protocol_uet_ip(const struct ib_device *device,
> + u32 port_num)
> +{
> + return device->port_data[port_num].immutable.core_cap_flags &
> + RDMA_CORE_CAP_PROT_UET_IP;
> +}
> +
> +static inline bool rdma_port_gids_from_netdev(const struct ib_device *device,
> + u32 port_num)
> +{
> + return rdma_protocol_roce(device, port_num) ||
> + rdma_protocol_uet(device, port_num);
> +}
> +
[Severity: High]
While this helper correctly abstracts the GID table logic for UET and RoCE,
are there missing netdev checks in the CMA and Multicast layers that still
explicitly check rdma_protocol_roce() instead of this new helper?
In cma_ib_acquire_dev(), UET-only ports fail the RoCE check and fall into the
else block intended for InfiniBand:
drivers/infiniband/core/cma.c:cma_ib_acquire_dev() {
if (rdma_protocol_roce(req->device, req->port))
rdma_ip2gid((struct sockaddr *)&id_priv->id.route.addr.src_addr, &gid);
else
memcpy(&gid, dev_addr->src_dev_addr + rdma_addr_gid_offset(dev_addr), sizeof(gid));
...
}
Since Ethernet MAC addresses are only 6 bytes, does copying 16 bytes from
dev_addr->src_dev_addr lead to an out-of-bounds read and a malformed GID?
Similarly, in rdma_resolve_route(), UET fails the protocol checks and falls
through to return -ENOSYS:
drivers/infiniband/core/cma.c:rdma_resolve_route() {
...
else if (rdma_protocol_roce(id->device, id->port_num)) {
ret = cma_resolve_iboe_route(id_priv);
...
}
Finally, in ib_init_ah_from_mcmember(), UET ports fail the !rdma_protocol_roce
check and return -EINVAL:
drivers/infiniband/core/multicast.c:ib_init_ah_from_mcmember() {
...
else if (!rdma_protocol_roce(device, port_num))
return -EINVAL;
...
}
Will these missing updates break route resolution and multicast joins for UET
devices?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=2
^ permalink raw reply [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
2026-10-14 22:35 ` sashiko-bot
@ 2026-10-15 15:57 ` Eric Davis
2026-10-15 17:33 ` Jason Gunthorpe
0 siblings, 1 reply; 60+ messages in thread
From: Eric Davis @ 2026-10-15 15:57 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rdma, Jason Gunthorpe, Leon Romanovsky
[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]
On Wed, Oct 14, 2026 at 3:35 PM <sashiko-bot@kernel.org> wrote:
> Sashiko AI review found 3 potential issue(s) to consider:
Disagree with all three and no change planned. None of these paths runs on
a UET port.
A UET port's core_cap_flags are RDMA_CORE_CAP_PROT_UET_[UDP|IP] plus
RDMA_CORE_CAP_ETH_AH. It has no IB_MAD, IB_CM, IB_SA or IB_MCAST bit.
> Does this explicitly fail Address Handle initialization for UET-only ports?
Yes, deliberately. ib_init_ah_attr_from_wc() is called by the MAD agent,
the CM, mad_rmpp, and user_mad. These all need IB_MAD, so none of them
sees a completion from a UET-only port. The check is a guard as an
unconnected peer is named through an extended address handle (patch 5) or
the job's address table (patch 7), never from a WC.
> Could this cause incoming UET packets to be incorrectly mapped to a RoCEv2
> GID index, breaking the UET transport response?
On a port that runs both RoCE and UET the MAD traffic is RoCE, so mapping
it to RoCEv2 is correct.
> In sa_conv_gid_to_pathrec_type() in include/rdma/ib_sa.h, there are no
> cases for the new UET GID types
[...]
> In cma_ib_acquire_dev(), UET-only ports fail the RoCE check and fall
> into the else block intended for InfiniBand
cma_supported() only takes a device that has an IB_CM or IW_CM port, so
rdma_cm never binds a UET-only port, and cma_ib_acquire_dev(),
rdma_resolve_route() and ib_init_ah_from_mcmember() are not reached. On
a dual-mode port, rdma_cm keeps using RoCE GIDs: this patch makes
cma_set_default_gid_type() refuse both UET types, so a UET GID never
reaches the SA path record code.
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=2
Thanks,
- eric
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5461 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
2026-10-15 15:57 ` Eric Davis
@ 2026-10-15 17:33 ` Jason Gunthorpe
2026-10-15 18:57 ` Sean Hefty
0 siblings, 1 reply; 60+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 17:33 UTC (permalink / raw)
To: Eric Davis; +Cc: sashiko-reviews, linux-rdma, Leon Romanovsky
On Thu, Oct 15, 2026 at 08:57:44AM -0700, Eric Davis wrote:
> On Wed, Oct 14, 2026 at 3:35 PM <sashiko-bot@kernel.org> wrote:
> > Sashiko AI review found 3 potential issue(s) to consider:
>
> Disagree with all three and no change planned. None of these paths runs on
> a UET port.
>
> A UET port's core_cap_flags are RDMA_CORE_CAP_PROT_UET_[UDP|IP] plus
> RDMA_CORE_CAP_ETH_AH. It has no IB_MAD, IB_CM, IB_SA or IB_MCAST bit.
>
> > Does this explicitly fail Address Handle initialization for UET-only ports?
>
> Yes, deliberately. ib_init_ah_attr_from_wc() is called by the MAD agent,
> the CM, mad_rmpp, and user_mad. These all need IB_MAD, so none of them
> sees a completion from a UET-only port. The check is a guard as an
> unconnected peer is named through an extended address handle (patch 5) or
> the job's address table (patch 7), never from a WC.
How much of this follows the work Sean did in the consortium? I
thought he had defined AHs?
> > Could this cause incoming UET packets to be incorrectly mapped to a RoCEv2
> > GID index, breaking the UET transport response?
>
> On a port that runs both RoCE and UET the MAD traffic is RoCE, so mapping
> it to RoCEv2 is correct.
You will need seperate ROCE and UET ib_devices with the current state
of affairs, we don't support multi-protocol today.
Jason
^ permalink raw reply [flat|nested] 60+ messages in thread
* RE: [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
2026-10-15 17:33 ` Jason Gunthorpe
@ 2026-10-15 18:57 ` Sean Hefty
2026-10-15 19:54 ` Jason Gunthorpe
0 siblings, 1 reply; 60+ messages in thread
From: Sean Hefty @ 2026-10-15 18:57 UTC (permalink / raw)
To: Jason Gunthorpe, Eric Davis
Cc: sashiko-reviews@lists.linux.dev, linux-rdma@vger.kernel.org,
Leon Romanovsky
> On Thu, Oct 15, 2026 at 08:57:44AM -0700, Eric Davis wrote:
> > On Wed, Oct 14, 2026 at 3:35 PM <sashiko-bot@kernel.org> wrote:
> > > Sashiko AI review found 3 potential issue(s) to consider:
> >
> > Disagree with all three and no change planned. None of these paths
> > runs on a UET port.
> >
> > A UET port's core_cap_flags are RDMA_CORE_CAP_PROT_UET_[UDP|IP] plus
> > RDMA_CORE_CAP_ETH_AH. It has no IB_MAD, IB_CM, IB_SA or IB_MCAST
> bit.
> >
> > > Does this explicitly fail Address Handle initialization for UET-only ports?
> >
> > Yes, deliberately. ib_init_ah_attr_from_wc() is called by the MAD
> > agent, the CM, mad_rmpp, and user_mad. These all need IB_MAD, so none
> > of them sees a completion from a UET-only port. The check is a guard
> > as an unconnected peer is named through an extended address handle
> > (patch 5) or the job's address table (patch 7), never from a WC.
>
> How much of this follows the work Sean did in the consortium? I thought he
> had defined AHs?
Eric and I worked closely. Most of the UEC work focused on the user-space interfaces, but the overall structure should be aligned.
UEC defined an extended AH. ib_init_ah_attr_from_wc() could probably be extended to support UET, I don't think it would be used, however.
> > > Could this cause incoming UET packets to be incorrectly mapped to a
> > > RoCEv2 GID index, breaking the UET transport response?
> >
> > On a port that runs both RoCE and UET the MAD traffic is RoCE, so
> > mapping it to RoCEv2 is correct.
>
> You will need seperate ROCE and UET ib_devices with the current state of
> affairs, we don't support multi-protocol today.
UEC intentionally considered multi-protocol devices, with a vendor deciding whether it would need to provide separate SW devices. E.g. using UET for MPI, but RoCE for storage. Protocol selection is via GID, same as RoCEv1 vs v2.
What blocks multi-protocol support?
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
2026-10-15 18:57 ` Sean Hefty
@ 2026-10-15 19:54 ` Jason Gunthorpe
2026-10-15 21:14 ` Sean Hefty
0 siblings, 1 reply; 60+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 19:54 UTC (permalink / raw)
To: Sean Hefty
Cc: Eric Davis, sashiko-reviews@lists.linux.dev,
linux-rdma@vger.kernel.org, Leon Romanovsky
On Thu, Oct 15, 2026 at 06:57:31PM +0000, Sean Hefty wrote:
> > On Thu, Oct 15, 2026 at 08:57:44AM -0700, Eric Davis wrote:
> > > On Wed, Oct 14, 2026 at 3:35 PM <sashiko-bot@kernel.org> wrote:
> > > > Sashiko AI review found 3 potential issue(s) to consider:
> > >
> > > Disagree with all three and no change planned. None of these paths
> > > runs on a UET port.
> > >
> > > A UET port's core_cap_flags are RDMA_CORE_CAP_PROT_UET_[UDP|IP] plus
> > > RDMA_CORE_CAP_ETH_AH. It has no IB_MAD, IB_CM, IB_SA or IB_MCAST
> > bit.
> > >
> > > > Does this explicitly fail Address Handle initialization for UET-only ports?
> > >
> > > Yes, deliberately. ib_init_ah_attr_from_wc() is called by the MAD
> > > agent, the CM, mad_rmpp, and user_mad. These all need IB_MAD, so none
> > > of them sees a completion from a UET-only port. The check is a guard
> > > as an unconnected peer is named through an extended address handle
> > > (patch 5) or the job's address table (patch 7), never from a WC.
> >
> > How much of this follows the work Sean did in the consortium? I thought he
> > had defined AHs?
>
> Eric and I worked closely. Most of the UEC work focused on the user-space interfaces, but the overall structure should be aligned.
>
> UEC defined an extended AH. ib_init_ah_attr_from_wc() could probably be extended to support UET, I don't think it would be used, however.
>
> > > > Could this cause incoming UET packets to be incorrectly mapped to a
> > > > RoCEv2 GID index, breaking the UET transport response?
> > >
> > > On a port that runs both RoCE and UET the MAD traffic is RoCE, so
> > > mapping it to RoCEv2 is correct.
> >
> > You will need seperate ROCE and UET ib_devices with the current state of
> > affairs, we don't support multi-protocol today.
>
> UEC intentionally considered multi-protocol devices, with a vendor deciding whether it would need to provide separate SW devices. E.g. using UET for MPI, but RoCE for storage. Protocol selection is via GID, same as RoCEv1 vs v2.
>
> What blocks multi-protocol support?
Principally I don't think we have the APIs to discover and link the
GUID to protocol to QP type appropriately..
Jason
^ permalink raw reply [flat|nested] 60+ messages in thread
* RE: [PATCH rdma-next 02/13] RDMA/core: add the UET GID types
2026-10-15 19:54 ` Jason Gunthorpe
@ 2026-10-15 21:14 ` Sean Hefty
0 siblings, 0 replies; 60+ messages in thread
From: Sean Hefty @ 2026-10-15 21:14 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Eric Davis, sashiko-reviews@lists.linux.dev,
linux-rdma@vger.kernel.org, Leon Romanovsky
> > > > > Could this cause incoming UET packets to be incorrectly mapped
> > > > > to a
> > > > > RoCEv2 GID index, breaking the UET transport response?
> > > >
> > > > On a port that runs both RoCE and UET the MAD traffic is RoCE, so
> > > > mapping it to RoCEv2 is correct.
> > >
> > > You will need seperate ROCE and UET ib_devices with the current
> > > state of affairs, we don't support multi-protocol today.
> >
> > UEC intentionally considered multi-protocol devices, with a vendor deciding
> whether it would need to provide separate SW devices. E.g. using UET for
> MPI, but RoCE for storage. Protocol selection is via GID, same as RoCEv1 vs
> v2.
> >
> > What blocks multi-protocol support?
>
> Principally I don't think we have the APIs to discover and link the GUID to
> protocol to QP type appropriately..
I'm missing something regarding the GUID comment.
The QP protocol comes from GID index, which is discoverable via ibv_gid_entry::gid_type. (UET QPs require the GID index on create.)
E.g. a dual RoCE + UET device would report node_type = UNSPECIFIED & transport_type = UNSPECIFIED. GID entries indicate supported protocols.
For compatibility, it may be possible for a device to report node_type = CA & transport_type = IB and expose UET GIDs only through ibv_query_gid_ex(). I haven't thought this option all the way through.
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 03/13] RDMA/core: report the reliable unconnected device capabilities
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
Add the capability bits a device uses to advertise reliable unconnected
support alongside additional features. The transport itself, 64-bit
immediate data, 64-bit memory keys, caller-chosen keys, and the four
memory region access restrictions.
Report the job, job key, and address table limits through the extended
query_device response so userspace can size against them.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_cmd.c | 3 +++
include/rdma/ib_verbs.h | 14 ++++++++++++++
include/uapi/rdma/ib_user_verbs.h | 25 +++++++++++++++++++++++++
3 files changed, 42 insertions(+)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index aca7c6ab55cd..ed55fbbdfc18 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -3597,6 +3597,9 @@ static int ib_uverbs_ex_query_device(struct uverbs_attr_bundle *attrs)
resp.cq_moderation_caps.max_cq_moderation_period =
attr.cq_caps.max_cq_moderation_period;
resp.max_dm_size = attr.max_dm_size;
+ resp.max_job_ids = attr.max_job_ids;
+ resp.max_job_keys = attr.max_job_keys;
+ resp.max_addr_entries = attr.max_addr_entries;
resp.response_length = uverbs_response_length(attrs, sizeof(resp));
return uverbs_response(attrs, &resp, sizeof(resp));
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 4e3a5c38cbe8..02f3312c7cf2 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -278,6 +278,14 @@ enum ib_device_cap_flags {
IB_DEVICE_FLUSH_PERSISTENT = IB_UVERBS_DEVICE_FLUSH_PERSISTENT,
IB_DEVICE_ATOMIC_WRITE = IB_UVERBS_DEVICE_ATOMIC_WRITE,
IB_DEVICE_CC_DMA_BOUNCE = IB_UVERBS_DEVICE_CC_DMA_BOUNCE,
+ IB_DEVICE_RU = IB_UVERBS_DEVICE_RU,
+ IB_DEVICE_IMM64 = IB_UVERBS_DEVICE_IMM64,
+ IB_DEVICE_KEY64 = IB_UVERBS_DEVICE_KEY64,
+ IB_DEVICE_USER_RKEY = IB_UVERBS_DEVICE_USER_RKEY,
+ IB_DEVICE_MR_UNRESTRICTED = IB_UVERBS_DEVICE_MR_UNRESTRICTED,
+ IB_DEVICE_MR_JOB_RESTRICTED = IB_UVERBS_DEVICE_MR_JOB_RESTRICTED,
+ IB_DEVICE_MR_QP_RESTRICTED = IB_UVERBS_DEVICE_MR_QP_RESTRICTED,
+ IB_DEVICE_MR_QP_JOB_RESTRICTED = IB_UVERBS_DEVICE_MR_QP_JOB_RESTRICTED,
};
enum ib_kernel_cap_flags {
@@ -456,6 +464,12 @@ struct ib_device_attr {
u64 max_dm_size;
/* Max entries for sgl for optimized performance per READ */
u32 max_sgl_rd;
+ /* Jobs allocated at once, not the largest JobID */
+ u32 max_job_ids;
+ /* Job keys allocated at once */
+ u32 max_job_keys;
+ /* Largest address table a single job may have */
+ u32 max_addr_entries;
};
enum ib_mtu {
diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
index 565301594634..bae6fcfa01dd 100644
--- a/include/uapi/rdma/ib_user_verbs.h
+++ b/include/uapi/rdma/ib_user_verbs.h
@@ -300,6 +300,10 @@ struct ib_uverbs_ex_query_device_resp {
__aligned_u64 max_dm_size;
__u32 xrc_odp_caps;
__u32 reserved;
+ __u32 max_job_ids; /* jobs allocated at once, not a JobID */
+ __u32 max_job_keys; /* job keys allocated at once */
+ __u32 max_addr_entries; /* largest address table of one job */
+ __u32 reserved2;
};
struct ib_uverbs_query_port {
@@ -1370,6 +1374,27 @@ enum ib_uverbs_device_cap_flags {
IB_UVERBS_DEVICE_ATOMIC_WRITE = 1ULL << 40,
/* CoCo guest with DMA bounce buffering required */
IB_UVERBS_DEVICE_CC_DMA_BOUNCE = 1ULL << 41,
+ /* Reliable unconnected QPs, jobs, job keys and address tables */
+ IB_UVERBS_DEVICE_RU = 1ULL << 42,
+ /* 64-bit immediate data */
+ IB_UVERBS_DEVICE_IMM64 = 1ULL << 43,
+ /* An MR may carry 64-bit keys as a second name beside its 32-bit ones */
+ IB_UVERBS_DEVICE_KEY64 = 1ULL << 44,
+ /* MR registration with a caller-chosen rkey */
+ IB_UVERBS_DEVICE_USER_RKEY = 1ULL << 45,
+ /*
+ * MR access restrictions, meaningful only with IB_UVERBS_DEVICE_RU.
+ * Each names what an incoming request must match, and each is its own
+ * match: supporting two does not imply their combination.
+ */
+ /* PD: any QP */
+ IB_UVERBS_DEVICE_MR_UNRESTRICTED = 1ULL << 46,
+ /* PD and JobID: any QP, one job */
+ IB_UVERBS_DEVICE_MR_JOB_RESTRICTED = 1ULL << 47,
+ /* PD and QP: the QP the MR is attached to */
+ IB_UVERBS_DEVICE_MR_QP_RESTRICTED = 1ULL << 48,
+ /* PD, QP and JobID: the attached QP, one job */
+ IB_UVERBS_DEVICE_MR_QP_JOB_RESTRICTED = 1ULL << 49,
};
enum ib_uverbs_raw_packet_caps {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 03/13] RDMA/core: report the reliable unconnected device capabilities
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
Add the capability bits a device uses to advertise reliable unconnected
support alongside additional features. The transport itself, 64-bit
immediate data, 64-bit memory keys, caller-chosen keys, and the four
memory region access restrictions.
Report the job, job key, and address table limits through the extended
query_device response so userspace can size against them.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_cmd.c | 3 +++
include/rdma/ib_verbs.h | 14 ++++++++++++++
include/uapi/rdma/ib_user_verbs.h | 25 +++++++++++++++++++++++++
3 files changed, 42 insertions(+)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index aca7c6ab55cd..ed55fbbdfc18 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -3597,6 +3597,9 @@ static int ib_uverbs_ex_query_device(struct uverbs_attr_bundle *attrs)
resp.cq_moderation_caps.max_cq_moderation_period =
attr.cq_caps.max_cq_moderation_period;
resp.max_dm_size = attr.max_dm_size;
+ resp.max_job_ids = attr.max_job_ids;
+ resp.max_job_keys = attr.max_job_keys;
+ resp.max_addr_entries = attr.max_addr_entries;
resp.response_length = uverbs_response_length(attrs, sizeof(resp));
return uverbs_response(attrs, &resp, sizeof(resp));
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 4e3a5c38cbe8..02f3312c7cf2 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -278,6 +278,14 @@ enum ib_device_cap_flags {
IB_DEVICE_FLUSH_PERSISTENT = IB_UVERBS_DEVICE_FLUSH_PERSISTENT,
IB_DEVICE_ATOMIC_WRITE = IB_UVERBS_DEVICE_ATOMIC_WRITE,
IB_DEVICE_CC_DMA_BOUNCE = IB_UVERBS_DEVICE_CC_DMA_BOUNCE,
+ IB_DEVICE_RU = IB_UVERBS_DEVICE_RU,
+ IB_DEVICE_IMM64 = IB_UVERBS_DEVICE_IMM64,
+ IB_DEVICE_KEY64 = IB_UVERBS_DEVICE_KEY64,
+ IB_DEVICE_USER_RKEY = IB_UVERBS_DEVICE_USER_RKEY,
+ IB_DEVICE_MR_UNRESTRICTED = IB_UVERBS_DEVICE_MR_UNRESTRICTED,
+ IB_DEVICE_MR_JOB_RESTRICTED = IB_UVERBS_DEVICE_MR_JOB_RESTRICTED,
+ IB_DEVICE_MR_QP_RESTRICTED = IB_UVERBS_DEVICE_MR_QP_RESTRICTED,
+ IB_DEVICE_MR_QP_JOB_RESTRICTED = IB_UVERBS_DEVICE_MR_QP_JOB_RESTRICTED,
};
enum ib_kernel_cap_flags {
@@ -456,6 +464,12 @@ struct ib_device_attr {
u64 max_dm_size;
/* Max entries for sgl for optimized performance per READ */
u32 max_sgl_rd;
+ /* Jobs allocated at once, not the largest JobID */
+ u32 max_job_ids;
+ /* Job keys allocated at once */
+ u32 max_job_keys;
+ /* Largest address table a single job may have */
+ u32 max_addr_entries;
};
enum ib_mtu {
diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
index 565301594634..bae6fcfa01dd 100644
--- a/include/uapi/rdma/ib_user_verbs.h
+++ b/include/uapi/rdma/ib_user_verbs.h
@@ -300,6 +300,10 @@ struct ib_uverbs_ex_query_device_resp {
__aligned_u64 max_dm_size;
__u32 xrc_odp_caps;
__u32 reserved;
+ __u32 max_job_ids; /* jobs allocated at once, not a JobID */
+ __u32 max_job_keys; /* job keys allocated at once */
+ __u32 max_addr_entries; /* largest address table of one job */
+ __u32 reserved2;
};
struct ib_uverbs_query_port {
@@ -1370,6 +1374,27 @@ enum ib_uverbs_device_cap_flags {
IB_UVERBS_DEVICE_ATOMIC_WRITE = 1ULL << 40,
/* CoCo guest with DMA bounce buffering required */
IB_UVERBS_DEVICE_CC_DMA_BOUNCE = 1ULL << 41,
+ /* Reliable unconnected QPs, jobs, job keys and address tables */
+ IB_UVERBS_DEVICE_RU = 1ULL << 42,
+ /* 64-bit immediate data */
+ IB_UVERBS_DEVICE_IMM64 = 1ULL << 43,
+ /* An MR may carry 64-bit keys as a second name beside its 32-bit ones */
+ IB_UVERBS_DEVICE_KEY64 = 1ULL << 44,
+ /* MR registration with a caller-chosen rkey */
+ IB_UVERBS_DEVICE_USER_RKEY = 1ULL << 45,
+ /*
+ * MR access restrictions, meaningful only with IB_UVERBS_DEVICE_RU.
+ * Each names what an incoming request must match, and each is its own
+ * match: supporting two does not imply their combination.
+ */
+ /* PD: any QP */
+ IB_UVERBS_DEVICE_MR_UNRESTRICTED = 1ULL << 46,
+ /* PD and JobID: any QP, one job */
+ IB_UVERBS_DEVICE_MR_JOB_RESTRICTED = 1ULL << 47,
+ /* PD and QP: the QP the MR is attached to */
+ IB_UVERBS_DEVICE_MR_QP_RESTRICTED = 1ULL << 48,
+ /* PD, QP and JobID: the attached QP, one job */
+ IB_UVERBS_DEVICE_MR_QP_JOB_RESTRICTED = 1ULL << 49,
};
enum ib_uverbs_raw_packet_caps {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 03/13] RDMA/core: report the reliable unconnected device capabilities
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:30 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:30 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> Add the capability bits a device uses to advertise reliable unconnected
> support alongside additional features. The transport itself, 64-bit
> immediate data, 64-bit memory keys, caller-chosen keys, and the four
> memory region access restrictions.
>
> Report the job, job key, and address table limits through the extended
> query_device response so userspace can size against them.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=3
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 04/13] RDMA/core: add the reliable unconnected queue pair type
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
Add IB_QPT_RU and its userspace counterpart, together with the field
layout of a reliable unconnected queue pair number used by UET.
Such a number uses all 32 bits, unlike the 24-bit InfiniBand and RoCE
one, so exempt it from the restrack assertion that no queue pair number
sets the top byte.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/restrack.c | 13 ++++++++++++-
include/rdma/ib_verbs.h | 1 +
include/uapi/rdma/ib_user_ioctl_verbs.h | 16 ++++++++++++++++
3 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c
index a8b838fdc7b5..ddb09b17b733 100644
--- a/drivers/infiniband/core/restrack.c
+++ b/drivers/infiniband/core/restrack.c
@@ -241,7 +241,18 @@ void rdma_restrack_add(struct rdma_restrack_entry *res)
/* Special case to ensure that LQPN points to right QP */
struct ib_qp *qp = container_of(res, struct ib_qp, res);
- WARN_ONCE(qp->qp_num >> 24 || qp->port >> 8,
+ /* An InfiniBand/RoCE QP number is 24 bits and the top byte is
+ * where the port is folded in for SMI and GSI. A reliable
+ * unconnected pair is an exception as its QP number is a
+ * full 32b wide, though of the top byte only the absolute bit
+ * may be set. An absolute number whose low bits are 0 or 1
+ * still conflicts SMI/GSI on port 128, which takes a device
+ * with both RU and MADs on that many ports.
+ */
+ WARN_ONCE(qp->port >> 8 ||
+ (qp->qp_type != IB_QPT_RU && qp->qp_num >> 24) ||
+ (qp->qp_type == IB_QPT_RU &&
+ qp->qp_num & IB_UVERBS_QPN_UET_RESERVED_MASK),
"QP number 0x%0X and port 0x%0X", qp->qp_num,
qp->port);
res->id = qp->qp_num;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 02f3312c7cf2..8d74ed4fc581 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1188,6 +1188,7 @@ enum ib_qp_type {
IB_QPT_RAW_PACKET = IB_UVERBS_QPT_RAW_PACKET,
IB_QPT_XRC_INI = IB_UVERBS_QPT_XRC_INI,
IB_QPT_XRC_TGT = IB_UVERBS_QPT_XRC_TGT,
+ IB_QPT_RU = IB_UVERBS_QPT_RU,
IB_QPT_MAX,
IB_QPT_DRIVER = IB_UVERBS_QPT_DRIVER,
/* Reserve a range for qp types internal to the low level driver.
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index ed72f48960f8..b6d46115f799 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -91,6 +91,7 @@ enum ib_uverbs_qp_type {
IB_UVERBS_QPT_RAW_PACKET = 8,
IB_UVERBS_QPT_XRC_INI,
IB_UVERBS_QPT_XRC_TGT,
+ IB_UVERBS_QPT_RU,
IB_UVERBS_QPT_DRIVER = 0xFF,
};
@@ -302,6 +303,21 @@ struct ib_uverbs_buffer_desc {
__aligned_u64 length;
};
+/* A UET reliable unconnected queue pair number uses all 32 bits:
+ *
+ * [ absolute 1b | reserved 7b | resource index 12b | PIDonFEP 12b ]
+ *
+ * An absolute pair is named by its number alone. A relative one is named
+ * within a job, so it must carry a job key when it is created. The reserved
+ * bits are zero.
+ */
+#define IB_UVERBS_QPN_UET_PID_ON_FEP_MASK 0x00000fff
+#define IB_UVERBS_QPN_UET_PID_ON_FEP_SHIFT 0
+#define IB_UVERBS_QPN_UET_RI_MASK 0x00fff000
+#define IB_UVERBS_QPN_UET_RI_SHIFT 12
+#define IB_UVERBS_QPN_UET_RESERVED_MASK 0x7f000000
+#define IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT 0x80000000
+
enum ib_uverbs_comp_cntr_entry {
IB_UVERBS_COMP_CNTR_ENTRY_COMP,
IB_UVERBS_COMP_CNTR_ENTRY_ERR,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 04/13] RDMA/core: add the reliable unconnected queue pair type
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
Add IB_QPT_RU and its userspace counterpart, together with the field
layout of a reliable unconnected queue pair number used by UET.
Such a number uses all 32 bits, unlike the 24-bit InfiniBand and RoCE
one, so exempt it from the restrack assertion that no queue pair number
sets the top byte.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/restrack.c | 13 ++++++++++++-
include/rdma/ib_verbs.h | 1 +
include/uapi/rdma/ib_user_ioctl_verbs.h | 16 ++++++++++++++++
3 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c
index a8b838fdc7b5..ddb09b17b733 100644
--- a/drivers/infiniband/core/restrack.c
+++ b/drivers/infiniband/core/restrack.c
@@ -241,7 +241,18 @@ void rdma_restrack_add(struct rdma_restrack_entry *res)
/* Special case to ensure that LQPN points to right QP */
struct ib_qp *qp = container_of(res, struct ib_qp, res);
- WARN_ONCE(qp->qp_num >> 24 || qp->port >> 8,
+ /* An InfiniBand/RoCE QP number is 24 bits and the top byte is
+ * where the port is folded in for SMI and GSI. A reliable
+ * unconnected pair is an exception as its QP number is a
+ * full 32b wide, though of the top byte only the absolute bit
+ * may be set. An absolute number whose low bits are 0 or 1
+ * still conflicts SMI/GSI on port 128, which takes a device
+ * with both RU and MADs on that many ports.
+ */
+ WARN_ONCE(qp->port >> 8 ||
+ (qp->qp_type != IB_QPT_RU && qp->qp_num >> 24) ||
+ (qp->qp_type == IB_QPT_RU &&
+ qp->qp_num & IB_UVERBS_QPN_UET_RESERVED_MASK),
"QP number 0x%0X and port 0x%0X", qp->qp_num,
qp->port);
res->id = qp->qp_num;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 02f3312c7cf2..8d74ed4fc581 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1188,6 +1188,7 @@ enum ib_qp_type {
IB_QPT_RAW_PACKET = IB_UVERBS_QPT_RAW_PACKET,
IB_QPT_XRC_INI = IB_UVERBS_QPT_XRC_INI,
IB_QPT_XRC_TGT = IB_UVERBS_QPT_XRC_TGT,
+ IB_QPT_RU = IB_UVERBS_QPT_RU,
IB_QPT_MAX,
IB_QPT_DRIVER = IB_UVERBS_QPT_DRIVER,
/* Reserve a range for qp types internal to the low level driver.
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index ed72f48960f8..b6d46115f799 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -91,6 +91,7 @@ enum ib_uverbs_qp_type {
IB_UVERBS_QPT_RAW_PACKET = 8,
IB_UVERBS_QPT_XRC_INI,
IB_UVERBS_QPT_XRC_TGT,
+ IB_UVERBS_QPT_RU,
IB_UVERBS_QPT_DRIVER = 0xFF,
};
@@ -302,6 +303,21 @@ struct ib_uverbs_buffer_desc {
__aligned_u64 length;
};
+/* A UET reliable unconnected queue pair number uses all 32 bits:
+ *
+ * [ absolute 1b | reserved 7b | resource index 12b | PIDonFEP 12b ]
+ *
+ * An absolute pair is named by its number alone. A relative one is named
+ * within a job, so it must carry a job key when it is created. The reserved
+ * bits are zero.
+ */
+#define IB_UVERBS_QPN_UET_PID_ON_FEP_MASK 0x00000fff
+#define IB_UVERBS_QPN_UET_PID_ON_FEP_SHIFT 0
+#define IB_UVERBS_QPN_UET_RI_MASK 0x00fff000
+#define IB_UVERBS_QPN_UET_RI_SHIFT 12
+#define IB_UVERBS_QPN_UET_RESERVED_MASK 0x7f000000
+#define IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT 0x80000000
+
enum ib_uverbs_comp_cntr_entry {
IB_UVERBS_COMP_CNTR_ENTRY_COMP,
IB_UVERBS_COMP_CNTR_ENTRY_ERR,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 04/13] RDMA/core: add the reliable unconnected queue pair type
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:43 ` sashiko-bot
2026-10-15 16:07 ` Eric Davis
-1 siblings, 1 reply; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:43 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] rdma: missing state transitions for IB_QPT_RU break QP modification
- [Medium] rdma: missing IB_QPT_RU validation in legacy ib_uverbs_post_send
--
commit dcd2780ae82f1b92d760ff7dc346585d88f3a3f7
Author: Eric Davis <eric.davis@broadcom.com>
RDMA/core: add the reliable unconnected queue pair type
This commit adds the IB_QPT_RU type and its userspace counterpart. It also
defines the field layout for a reliable unconnected queue pair number used by
UET, and exempts its full 32-bit width from the usual 24-bit restrack
assertion.
[ ... ]
> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index 02f3312c7cf24..8d74ed4fc581f 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
> @@ -1188,6 +1188,7 @@ enum ib_qp_type {
> IB_QPT_RAW_PACKET = IB_UVERBS_QPT_RAW_PACKET,
> IB_QPT_XRC_INI = IB_UVERBS_QPT_XRC_INI,
> IB_QPT_XRC_TGT = IB_UVERBS_QPT_XRC_TGT,
> + IB_QPT_RU = IB_UVERBS_QPT_RU,
[Severity: High]
With the addition of IB_QPT_RU, should it also be added to qp_state_table[]
in drivers/infiniband/core/verbs.c?
Currently, because IB_QPT_RU is omitted from qp_state_table[], its
configuration logic zero-evaluates. This causes the core helper
ib_modify_qp_is_ok() to block all state advancements for IB_QPT_RU QPs, as
the implicit 0 values flag any transition parameters as invalid:
drivers/infiniband/core/verbs.c:ib_modify_qp_is_ok() {
...
req_param = qp_state_table[cur_state][next_state].req_param[type];
opt_param = qp_state_table[cur_state][next_state].opt_param[type];
if ((mask & req_param) != req_param)
return false;
...
}
For example, when transitioning from IB_QPS_RESET to IB_QPS_RTS,
ib_modify_qp_is_ok() checks qp_state_table[IB_QPS_RESET][IB_QPS_RTS].valid,
reads 0, and returns false. This causes the driver to reject the configuration
with -EINVAL.
[Severity: Medium]
Since IB_QPT_RU is an unconnected queue pair type, should
ib_uverbs_post_send() in drivers/infiniband/core/uverbs_cmd.c be updated to
account for it?
Currently, ib_uverbs_post_send() only checks for IB_QPT_UD to identify
unconnected QPs:
drivers/infiniband/core/uverbs_cmd.c:ib_uverbs_post_send() {
...
is_ud = qp->qp_type == IB_QPT_UD;
...
}
If a send request is posted for an IB_QPT_RU QP via this legacy API, the core
will fall back to connected WR parsing, allocating struct ib_send_wr instead
of an unconnected WR struct with destination fields.
When this malformed WR is passed to an unconnected transport driver's
post_send op, the driver might attempt to access missing fields, which could
result in an out-of-bounds read or type confusion.
Should ib_uverbs_post_send() either parse IB_QPT_RU as an unconnected WR or
explicitly reject the operation with -EINVAL to prevent this?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=4
^ permalink raw reply [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 04/13] RDMA/core: add the reliable unconnected queue pair type
2026-10-14 22:43 ` sashiko-bot
@ 2026-10-15 16:07 ` Eric Davis
2026-10-15 16:09 ` Eric Davis
0 siblings, 1 reply; 60+ messages in thread
From: Eric Davis @ 2026-10-15 16:07 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rdma, Jason Gunthorpe, Leon Romanovsky
[-- Attachment #1: Type: text/plain, Size: 1583 bytes --]
On Wed, Oct 14, 2026 at 3:43 PM <sashiko-bot@kernel.org> wrote:
> With the addition of IB_QPT_RU, should it also be added to qp_state_table[]
> in drivers/infiniband/core/verbs.c?
No. Core never calls ib_modify_qp_is_ok(). It is a helper a driver may
call from its own modify_qp, and an RU driver validates RU state changes
itself.
I don't think a row can be added without causing a regression. The valid flag
in qp_state_table is per transition, not per QP type, so making RESET->RTS
valid for RU would make it valid for RC, UD, and every other type in
every driver that uses the helper.
What core does enforce for RU is in patch 10. The attributes a QP may be
created with are limited to STATE, PKEY_INDEX, PORT, ACCESS_FLAGS and
PATH_MTU, and the state must be RTS. The driver's modify_qp then takes
that as RESET->RTS, and the QP is destroyed if it refuses.
> Since IB_QPT_RU is an unconnected queue pair type, should
> ib_uverbs_post_send() in drivers/infiniband/core/uverbs_cmd.c be updated to
> account for it?
No. RU work requests never enter the kernel in this design; they are
posted by the provider (the rdma-core series), and there is no kernel RU
work request struct that a driver could mistake an ib_send_wr for. The
write() POST_SEND and POST_RECV commands are declared with
UAPI_DEF_METHOD_NEEDS_FN(post_send) and (post_recv), so they only
exist on a device whose driver supplies those ops.
>
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=4
Thanks,
- e
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5461 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 04/13] RDMA/core: add the reliable unconnected queue pair type
2026-10-15 16:07 ` Eric Davis
@ 2026-10-15 16:09 ` Eric Davis
2026-10-15 17:30 ` Jason Gunthorpe
0 siblings, 1 reply; 60+ messages in thread
From: Eric Davis @ 2026-10-15 16:09 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rdma, Jason Gunthorpe, Leon Romanovsky
[-- Attachment #1: Type: text/plain, Size: 1874 bytes --]
If it's preferred, v2 can make both post commands return -EOPNOTSUPP for an
IB_QPT_RU QP so that is explicit. Let me know.
Thanks,
- eric
On Thu, Oct 15, 2026 at 9:07 AM Eric Davis <eric.davis@broadcom.com> wrote:
>
> On Wed, Oct 14, 2026 at 3:43 PM <sashiko-bot@kernel.org> wrote:
> > With the addition of IB_QPT_RU, should it also be added to qp_state_table[]
> > in drivers/infiniband/core/verbs.c?
>
> No. Core never calls ib_modify_qp_is_ok(). It is a helper a driver may
> call from its own modify_qp, and an RU driver validates RU state changes
> itself.
>
> I don't think a row can be added without causing a regression. The valid flag
> in qp_state_table is per transition, not per QP type, so making RESET->RTS
> valid for RU would make it valid for RC, UD, and every other type in
> every driver that uses the helper.
>
> What core does enforce for RU is in patch 10. The attributes a QP may be
> created with are limited to STATE, PKEY_INDEX, PORT, ACCESS_FLAGS and
> PATH_MTU, and the state must be RTS. The driver's modify_qp then takes
> that as RESET->RTS, and the QP is destroyed if it refuses.
>
> > Since IB_QPT_RU is an unconnected queue pair type, should
> > ib_uverbs_post_send() in drivers/infiniband/core/uverbs_cmd.c be updated to
> > account for it?
>
> No. RU work requests never enter the kernel in this design; they are
> posted by the provider (the rdma-core series), and there is no kernel RU
> work request struct that a driver could mistake an ib_send_wr for. The
> write() POST_SEND and POST_RECV commands are declared with
> UAPI_DEF_METHOD_NEEDS_FN(post_send) and (post_recv), so they only
> exist on a device whose driver supplies those ops.
>
> >
> > --
> > Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=4
>
> Thanks,
> - e
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5461 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 04/13] RDMA/core: add the reliable unconnected queue pair type
2026-10-15 16:09 ` Eric Davis
@ 2026-10-15 17:30 ` Jason Gunthorpe
2026-10-15 18:35 ` Doug Ledford
0 siblings, 1 reply; 60+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 17:30 UTC (permalink / raw)
To: Eric Davis; +Cc: sashiko-reviews, linux-rdma, Leon Romanovsky
On Thu, Oct 15, 2026 at 09:09:20AM -0700, Eric Davis wrote:
> If it's preferred, v2 can make both post commands return -EOPNOTSUPP for an
> IB_QPT_RU QP so that is explicit. Let me know.
I didn't try to undertsand all this ai output, but as a general
comment if things are not supported please return EOPNOTSUPP rather
than pretending success.
Jason
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 04/13] RDMA/core: add the reliable unconnected queue pair type
2026-10-15 17:30 ` Jason Gunthorpe
@ 2026-10-15 18:35 ` Doug Ledford
0 siblings, 0 replies; 60+ messages in thread
From: Doug Ledford @ 2026-10-15 18:35 UTC (permalink / raw)
To: Jason Gunthorpe, Eric Davis; +Cc: sashiko-reviews, linux-rdma, Leon Romanovsky
[-- Attachment #1.1: Type: text/plain, Size: 1586 bytes --]
On 10/15/26 10:30 AM, Jason Gunthorpe wrote:
> On Thu, Oct 15, 2026 at 09:09:20AM -0700, Eric Davis wrote:
>> If it's preferred, v2 can make both post commands return -EOPNOTSUPP for an
>> IB_QPT_RU QP so that is explicit. Let me know.
>
> I didn't try to undertsand all this ai output, but as a general
> comment if things are not supported please return EOPNOTSUPP rather
> than pretending success.
>
> Jason
In this particular instance, it might be better to add the support for
RU qp types to the calls. We don't use them in real providers, but
neither are these calls used in any other hardware provider. They are
used if someone creates a software provider (rxe, siw, potentially suet
in the future maybe?). But, this does point out that we can't rely on
"real providers don't do this" because a nefarious user can download
rdma-core, make their own changes to the code, compile, and run it
against the uverbs device. Since this device is often not overly
restricted in terms of access, we can't guarantee that things won't get
called incorrectly whether by accident or on purpose. This particular
entry point is an example where not getting this right could cause the
kernel to mis-interpret the struct passed in and then it could do all
sorts of wonky things. This is the sort of things that google's
syzcaller would be used to trip up. So, I think this is probably one
that needs resolved.
--
Doug Ledford <doug.ledford@hpe.com>
GPG KeyID: B826A3330E572FDD
Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 05/13] RDMA/core: extended address handle attributes
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
An unconnected transport over UET resolves a peer by both its network
address and its queue pair number. The write() create-AH command has no
room for additional attributes. Add rdma_ah_attr_ex to carry new
attributes, reach it through rdma_ah_init_attr, and add an ioctl method
that supports it.
rdma_create_user_ah() now takes the init attributes rather than the
address alone, so callers pass the wider set.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/user_mad.c | 4 +-
drivers/infiniband/core/uverbs.h | 3 ++
drivers/infiniband/core/uverbs_cmd.c | 60 ++++++++++++++----------
drivers/infiniband/core/uverbs_std_types.c | 73 +++++++++++++++++++++++++++++-
drivers/infiniband/core/verbs.c | 43 ++++++++++--------
include/rdma/ib_verbs.h | 20 ++++++--
include/uapi/rdma/ib_user_ioctl_cmds.h | 7 +++
include/uapi/rdma/ib_user_ioctl_verbs.h | 9 ++++
8 files changed, 172 insertions(+), 47 deletions(-)
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index c3ef9f85c7eb..05e2898f0a6a 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -512,6 +512,7 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf,
struct ib_umad_packet *packet;
struct ib_mad_agent *agent;
struct rdma_ah_attr ah_attr;
+ struct rdma_ah_init_attr init_attr = {};
struct ib_ah *ah;
__be64 *tid;
int ret, hdr_len, copy_offset, rmpp_active;
@@ -569,7 +570,8 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf,
rdma_ah_set_dgid_raw(&ah_attr, packet->mad.hdr.gid);
}
- ah = rdma_create_user_ah(agent->qp->pd, &ah_attr, NULL);
+ init_attr.ah_attr = &ah_attr;
+ ah = rdma_create_user_ah(agent->qp->pd, &init_attr, NULL);
if (IS_ERR(ah)) {
ret = PTR_ERR(ah);
goto err_up;
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index c64dd6b94e10..e08c41b8f45b 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -227,6 +227,9 @@ int ib_uverbs_dealloc_xrcd(struct ib_uobject *uobject, struct ib_xrcd *xrcd,
int uverbs_dealloc_mw(struct ib_mw *mw);
void ib_uverbs_detach_umcast(struct ib_qp *qp,
struct ib_uqp_object *uobj);
+int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
+ struct rdma_ah_attr *rdma_attr,
+ const struct ib_uverbs_ah_attr *uverb_attr);
struct bundle_alloc_head {
struct_group_tagged(bundle_alloc_head_hdr, hdr,
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index ed55fbbdfc18..5cfbe863df0b 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1746,6 +1746,37 @@ static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
}
}
+/* The ib_uverbs_ah_attr counterpart of copy_ah_attr_from_uverbs() below,
+ * which takes the modify QP layout. This one also rejects an invalid port.
+ */
+int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
+ struct rdma_ah_attr *rdma_attr,
+ const struct ib_uverbs_ah_attr *uverb_attr)
+{
+ if (!rdma_is_port_valid(dev, uverb_attr->port_num))
+ return -EINVAL;
+
+ rdma_attr->type = rdma_ah_find_type(dev, uverb_attr->port_num);
+ rdma_ah_set_make_grd(rdma_attr, false);
+ rdma_ah_set_dlid(rdma_attr, uverb_attr->dlid);
+ rdma_ah_set_sl(rdma_attr, uverb_attr->sl);
+ rdma_ah_set_path_bits(rdma_attr, uverb_attr->src_path_bits);
+ rdma_ah_set_static_rate(rdma_attr, uverb_attr->static_rate);
+ rdma_ah_set_port_num(rdma_attr, uverb_attr->port_num);
+
+ if (uverb_attr->is_global) {
+ rdma_ah_set_grh(rdma_attr, NULL, uverb_attr->grh.flow_label,
+ uverb_attr->grh.sgid_index,
+ uverb_attr->grh.hop_limit,
+ uverb_attr->grh.traffic_class);
+ rdma_ah_set_dgid_raw(rdma_attr, uverb_attr->grh.dgid);
+ } else {
+ rdma_ah_set_ah_flags(rdma_attr, 0);
+ }
+
+ return 0;
+}
+
static void copy_ah_attr_from_uverbs(struct ib_device *dev,
struct rdma_ah_attr *rdma_attr,
struct ib_uverbs_qp_dest *uverb_attr)
@@ -2421,6 +2452,7 @@ static int ib_uverbs_create_ah(struct uverbs_attr_bundle *attrs)
struct ib_pd *pd;
struct ib_ah *ah;
struct rdma_ah_attr attr = {};
+ struct rdma_ah_init_attr init_attr = {};
int ret;
struct ib_device *ib_dev;
@@ -2432,36 +2464,18 @@ static int ib_uverbs_create_ah(struct uverbs_attr_bundle *attrs)
if (IS_ERR(uobj))
return PTR_ERR(uobj);
- if (!rdma_is_port_valid(ib_dev, cmd.attr.port_num)) {
- ret = -EINVAL;
- goto err;
- }
-
pd = uobj_get_obj_read(pd, UVERBS_OBJECT_PD, cmd.pd_handle, attrs);
if (IS_ERR(pd)) {
ret = PTR_ERR(pd);
goto err;
}
- attr.type = rdma_ah_find_type(ib_dev, cmd.attr.port_num);
- rdma_ah_set_make_grd(&attr, false);
- rdma_ah_set_dlid(&attr, cmd.attr.dlid);
- rdma_ah_set_sl(&attr, cmd.attr.sl);
- rdma_ah_set_path_bits(&attr, cmd.attr.src_path_bits);
- rdma_ah_set_static_rate(&attr, cmd.attr.static_rate);
- rdma_ah_set_port_num(&attr, cmd.attr.port_num);
-
- if (cmd.attr.is_global) {
- rdma_ah_set_grh(&attr, NULL, cmd.attr.grh.flow_label,
- cmd.attr.grh.sgid_index,
- cmd.attr.grh.hop_limit,
- cmd.attr.grh.traffic_class);
- rdma_ah_set_dgid_raw(&attr, cmd.attr.grh.dgid);
- } else {
- rdma_ah_set_ah_flags(&attr, 0);
- }
+ ret = copy_ah_attr_from_uverbs_ah(ib_dev, &attr, &cmd.attr);
+ if (ret)
+ goto err_put;
- ah = rdma_create_user_ah(pd, &attr, &attrs->driver_udata);
+ init_attr.ah_attr = &attr;
+ ah = rdma_create_user_ah(pd, &init_attr, &attrs->driver_udata);
if (IS_ERR(ah)) {
ret = PTR_ERR(ah);
goto err_put;
diff --git a/drivers/infiniband/core/uverbs_std_types.c b/drivers/infiniband/core/uverbs_std_types.c
index e160786e1df1..c72c9e5a2286 100644
--- a/drivers/infiniband/core/uverbs_std_types.c
+++ b/drivers/infiniband/core/uverbs_std_types.c
@@ -184,6 +184,76 @@ DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_MW,
UVERBS_TYPE_ALLOC_IDR(uverbs_free_mw),
&UVERBS_METHOD(UVERBS_METHOD_MW_DESTROY));
+/* Creating an address handle is still a write() command whose request
+ * struct cannot grow. A transport (reliable unconnected) that resolves a
+ * peer using both transport and network addresses requires additional
+ * fields in the address handle. Here the address handle create command is
+ * made available as an ioctl method, where additional attributes can be
+ * carried.
+ */
+static int UVERBS_HANDLER(UVERBS_METHOD_AH_EX_CREATE)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj = uverbs_attr_get_uobject(
+ attrs, UVERBS_ATTR_CREATE_AH_EX_HANDLE);
+ struct ib_pd *pd =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_CREATE_AH_EX_PD_HANDLE);
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_uverbs_ah_attr_ex uah;
+ struct rdma_ah_init_attr init_attr = {};
+ struct rdma_ah_attr_ex attr = {};
+ struct ib_ah *ah;
+ int ret;
+
+ if (IS_ERR(pd))
+ return PTR_ERR(pd);
+
+ /* Only a reliable unconnected device has a use for the extended
+ * attributes.
+ */
+ if (!(ib_dev->attrs.device_cap_flags & IB_DEVICE_RU))
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from(&uah, attrs,
+ UVERBS_ATTR_CREATE_AH_EX_AH_ATTR_EX);
+ if (ret)
+ return ret;
+
+ ret = copy_ah_attr_from_uverbs_ah(ib_dev, &attr.ah_attr, &uah.ah_attr);
+ if (ret)
+ return ret;
+
+ attr.remote_qpn = uah.remote_qpn;
+
+ init_attr.ah_attr = &attr.ah_attr;
+ init_attr.ah_attr_ex = &attr;
+
+ ah = rdma_create_user_ah(pd, &init_attr, &attrs->driver_udata);
+ if (IS_ERR(ah))
+ return PTR_ERR(ah);
+
+ ah->uobject = uobj;
+ uobj->object = ah;
+
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_AH_EX_CREATE,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_AH_EX_HANDLE,
+ UVERBS_OBJECT_AH,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_AH_EX_PD_HANDLE,
+ UVERBS_OBJECT_PD,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_AH_EX_AH_ATTR_EX,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ah_attr_ex,
+ remote_qpn),
+ UA_MANDATORY),
+ UVERBS_ATTR_UHW());
+
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
UVERBS_METHOD_AH_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_AH_HANDLE,
@@ -193,7 +263,8 @@ DECLARE_UVERBS_NAMED_METHOD_DESTROY(
DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_AH,
UVERBS_TYPE_ALLOC_IDR(uverbs_free_ah),
- &UVERBS_METHOD(UVERBS_METHOD_AH_DESTROY));
+ &UVERBS_METHOD(UVERBS_METHOD_AH_DESTROY),
+ &UVERBS_METHOD(UVERBS_METHOD_AH_EX_CREATE));
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
UVERBS_METHOD_FLOW_DESTROY,
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 24db88e6d3fa..8ce7e376112c 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -561,24 +561,23 @@ rdma_update_sgid_attr(struct rdma_ah_attr *ah_attr,
}
static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
- struct rdma_ah_attr *ah_attr,
- u32 flags,
- struct ib_udata *udata,
- struct net_device *xmit_slave)
+ struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata)
{
- struct rdma_ah_init_attr init_attr = {};
+ struct rdma_ah_attr *ah_attr = init_attr->ah_attr;
struct ib_device *device = pd->device;
struct ib_ah *ah;
int ret;
- might_sleep_if(flags & RDMA_CREATE_AH_SLEEPABLE);
+ might_sleep_if(init_attr->flags & RDMA_CREATE_AH_SLEEPABLE);
if (!udata && !device->ops.create_ah)
return ERR_PTR(-EOPNOTSUPP);
ah = rdma_zalloc_drv_obj_gfp(
device, ib_ah,
- (flags & RDMA_CREATE_AH_SLEEPABLE) ? GFP_KERNEL : GFP_ATOMIC);
+ (init_attr->flags & RDMA_CREATE_AH_SLEEPABLE) ? GFP_KERNEL :
+ GFP_ATOMIC);
if (!ah)
return ERR_PTR(-ENOMEM);
@@ -586,14 +585,12 @@ static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
ah->pd = pd;
ah->type = ah_attr->type;
ah->sgid_attr = rdma_update_sgid_attr(ah_attr, NULL);
- init_attr.ah_attr = ah_attr;
- init_attr.flags = flags;
- init_attr.xmit_slave = xmit_slave;
if (udata)
- ret = device->ops.create_user_ah(ah, &init_attr, udata);
+ ret = device->ops.create_user_ah(ah, init_attr, udata);
else
- ret = device->ops.create_ah(ah, &init_attr, NULL);
+ ret = device->ops.create_ah(ah, init_attr, NULL);
+
if (ret) {
if (ah->sgid_attr)
rdma_put_gid_attr(ah->sgid_attr);
@@ -619,6 +616,7 @@ static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
u32 flags)
{
+ struct rdma_ah_init_attr init_attr = {};
const struct ib_gid_attr *old_sgid_attr;
struct net_device *slave;
struct ib_ah *ah;
@@ -634,7 +632,12 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
return ERR_CAST(slave);
}
- ah = _rdma_create_ah(pd, ah_attr, flags, NULL, slave);
+
+ init_attr.ah_attr = ah_attr;
+ init_attr.flags = flags;
+ init_attr.xmit_slave = slave;
+
+ ah = _rdma_create_ah(pd, &init_attr, NULL);
rdma_lag_put_ah_roce_slave(slave);
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
return ah;
@@ -646,18 +649,21 @@ EXPORT_SYMBOL(rdma_create_ah);
* given address vector.
* It resolves destination mac address for ah attribute of RoCE type.
* @pd: The protection domain associated with the address handle.
- * @ah_attr: The attributes of the address vector.
+ * @init_attr: The address vector, and for an extended handle the
+ * attributes beyond it.
* @udata: pointer to user's input output buffer information need by
* provider driver.
*
- * It returns 0 on success and returns appropriate error code on error.
* The address handle is used to reference a local or global destination
* in all UD QP post sends.
+ *
+ * Return: the address handle, or an ERR_PTR on error.
*/
struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
- struct rdma_ah_attr *ah_attr,
+ struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata)
{
+ struct rdma_ah_attr *ah_attr = init_attr->ah_attr;
const struct ib_gid_attr *old_sgid_attr;
struct ib_ah *ah;
int err;
@@ -674,8 +680,9 @@ struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
}
}
- ah = _rdma_create_ah(pd, ah_attr, RDMA_CREATE_AH_SLEEPABLE,
- udata, NULL);
+ init_attr->flags |= RDMA_CREATE_AH_SLEEPABLE;
+
+ ah = _rdma_create_ah(pd, init_attr, udata);
out:
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 8d74ed4fc581..68f7a271cf7a 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -974,10 +974,14 @@ struct ib_mr_status {
*/
__attribute_const__ enum ib_rate mult_to_ib_rate(int mult);
+struct rdma_ah_attr_ex;
+
struct rdma_ah_init_attr {
struct rdma_ah_attr *ah_attr;
u32 flags;
struct net_device *xmit_slave;
+ /* set to name a peer by both transport+network addresses */
+ struct rdma_ah_attr_ex *ah_attr_ex;
};
enum rdma_ah_attr_type {
@@ -1016,6 +1020,11 @@ struct rdma_ah_attr {
};
};
+struct rdma_ah_attr_ex {
+ struct rdma_ah_attr ah_attr;
+ u32 remote_qpn;
+};
+
enum ib_wc_status {
IB_WC_SUCCESS,
IB_WC_LOC_LEN_ERR,
@@ -3893,16 +3902,18 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
* rdma_create_user_ah - Creates an address handle for the given address vector.
* It resolves destination mac address for ah attribute of RoCE type.
* @pd: The protection domain associated with the address handle.
- * @ah_attr: The attributes of the address vector.
+ * @init_attr: The address vector, and for an extended handle the
+ * attributes beyond it.
* @udata: pointer to user's input output buffer information need by
* provider driver.
*
- * It returns 0 on success and returns appropriate error code on error.
* The address handle is used to reference a local or global destination
* in all UD QP post sends.
+ *
+ * Return: the address handle, or an ERR_PTR on error.
*/
struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
- struct rdma_ah_attr *ah_attr,
+ struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata);
/**
* ib_get_gids_from_rdma_hdr - Get sgid and dgid from GRH or IPv4 header
@@ -4981,7 +4992,8 @@ static inline struct ib_global_route
return &attr->grh;
}
-static inline void rdma_ah_set_dgid_raw(struct rdma_ah_attr *attr, void *dgid)
+static inline void rdma_ah_set_dgid_raw(struct rdma_ah_attr *attr,
+ const void *dgid)
{
struct ib_global_route *grh = rdma_ah_retrieve_grh(attr);
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 816b3107464f..40fcd4bd1b5e 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -410,6 +410,13 @@ enum uverbs_attrs_xrcd_destroy_ids {
enum uverbs_methods_ah {
UVERBS_METHOD_AH_DESTROY,
+ UVERBS_METHOD_AH_EX_CREATE,
+};
+
+enum uverbs_attrs_create_ah_ex_cmd_attr_ids {
+ UVERBS_ATTR_CREATE_AH_EX_HANDLE,
+ UVERBS_ATTR_CREATE_AH_EX_PD_HANDLE,
+ UVERBS_ATTR_CREATE_AH_EX_AH_ATTR_EX,
};
enum uverbs_attrs_ah_destroy_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index b6d46115f799..210f3f945bd7 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -318,6 +318,15 @@ struct ib_uverbs_buffer_desc {
#define IB_UVERBS_QPN_UET_RESERVED_MASK 0x7f000000
#define IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT 0x80000000
+/* An address handle that names a peer using both transport+network addresses.
+ * Used for reliable unconnected QPs in both work request address handles and
+ * a Job address table.
+ */
+struct ib_uverbs_ah_attr_ex {
+ struct ib_uverbs_ah_attr ah_attr;
+ __u32 remote_qpn;
+};
+
enum ib_uverbs_comp_cntr_entry {
IB_UVERBS_COMP_CNTR_ENTRY_COMP,
IB_UVERBS_COMP_CNTR_ENTRY_ERR,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 05/13] RDMA/core: extended address handle attributes
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
An unconnected transport over UET resolves a peer by both its network
address and its queue pair number. The write() create-AH command has no
room for additional attributes. Add rdma_ah_attr_ex to carry new
attributes, reach it through rdma_ah_init_attr, and add an ioctl method
that supports it.
rdma_create_user_ah() now takes the init attributes rather than the
address alone, so callers pass the wider set.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/user_mad.c | 4 +-
drivers/infiniband/core/uverbs.h | 3 ++
drivers/infiniband/core/uverbs_cmd.c | 60 ++++++++++++++----------
drivers/infiniband/core/uverbs_std_types.c | 73 +++++++++++++++++++++++++++++-
drivers/infiniband/core/verbs.c | 43 ++++++++++--------
include/rdma/ib_verbs.h | 20 ++++++--
include/uapi/rdma/ib_user_ioctl_cmds.h | 7 +++
include/uapi/rdma/ib_user_ioctl_verbs.h | 9 ++++
8 files changed, 172 insertions(+), 47 deletions(-)
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index c3ef9f85c7eb..05e2898f0a6a 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -512,6 +512,7 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf,
struct ib_umad_packet *packet;
struct ib_mad_agent *agent;
struct rdma_ah_attr ah_attr;
+ struct rdma_ah_init_attr init_attr = {};
struct ib_ah *ah;
__be64 *tid;
int ret, hdr_len, copy_offset, rmpp_active;
@@ -569,7 +570,8 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf,
rdma_ah_set_dgid_raw(&ah_attr, packet->mad.hdr.gid);
}
- ah = rdma_create_user_ah(agent->qp->pd, &ah_attr, NULL);
+ init_attr.ah_attr = &ah_attr;
+ ah = rdma_create_user_ah(agent->qp->pd, &init_attr, NULL);
if (IS_ERR(ah)) {
ret = PTR_ERR(ah);
goto err_up;
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index c64dd6b94e10..e08c41b8f45b 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -227,6 +227,9 @@ int ib_uverbs_dealloc_xrcd(struct ib_uobject *uobject, struct ib_xrcd *xrcd,
int uverbs_dealloc_mw(struct ib_mw *mw);
void ib_uverbs_detach_umcast(struct ib_qp *qp,
struct ib_uqp_object *uobj);
+int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
+ struct rdma_ah_attr *rdma_attr,
+ const struct ib_uverbs_ah_attr *uverb_attr);
struct bundle_alloc_head {
struct_group_tagged(bundle_alloc_head_hdr, hdr,
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index ed55fbbdfc18..5cfbe863df0b 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1746,6 +1746,37 @@ static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
}
}
+/* The ib_uverbs_ah_attr counterpart of copy_ah_attr_from_uverbs() below,
+ * which takes the modify QP layout. This one also rejects an invalid port.
+ */
+int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
+ struct rdma_ah_attr *rdma_attr,
+ const struct ib_uverbs_ah_attr *uverb_attr)
+{
+ if (!rdma_is_port_valid(dev, uverb_attr->port_num))
+ return -EINVAL;
+
+ rdma_attr->type = rdma_ah_find_type(dev, uverb_attr->port_num);
+ rdma_ah_set_make_grd(rdma_attr, false);
+ rdma_ah_set_dlid(rdma_attr, uverb_attr->dlid);
+ rdma_ah_set_sl(rdma_attr, uverb_attr->sl);
+ rdma_ah_set_path_bits(rdma_attr, uverb_attr->src_path_bits);
+ rdma_ah_set_static_rate(rdma_attr, uverb_attr->static_rate);
+ rdma_ah_set_port_num(rdma_attr, uverb_attr->port_num);
+
+ if (uverb_attr->is_global) {
+ rdma_ah_set_grh(rdma_attr, NULL, uverb_attr->grh.flow_label,
+ uverb_attr->grh.sgid_index,
+ uverb_attr->grh.hop_limit,
+ uverb_attr->grh.traffic_class);
+ rdma_ah_set_dgid_raw(rdma_attr, uverb_attr->grh.dgid);
+ } else {
+ rdma_ah_set_ah_flags(rdma_attr, 0);
+ }
+
+ return 0;
+}
+
static void copy_ah_attr_from_uverbs(struct ib_device *dev,
struct rdma_ah_attr *rdma_attr,
struct ib_uverbs_qp_dest *uverb_attr)
@@ -2421,6 +2452,7 @@ static int ib_uverbs_create_ah(struct uverbs_attr_bundle *attrs)
struct ib_pd *pd;
struct ib_ah *ah;
struct rdma_ah_attr attr = {};
+ struct rdma_ah_init_attr init_attr = {};
int ret;
struct ib_device *ib_dev;
@@ -2432,36 +2464,18 @@ static int ib_uverbs_create_ah(struct uverbs_attr_bundle *attrs)
if (IS_ERR(uobj))
return PTR_ERR(uobj);
- if (!rdma_is_port_valid(ib_dev, cmd.attr.port_num)) {
- ret = -EINVAL;
- goto err;
- }
-
pd = uobj_get_obj_read(pd, UVERBS_OBJECT_PD, cmd.pd_handle, attrs);
if (IS_ERR(pd)) {
ret = PTR_ERR(pd);
goto err;
}
- attr.type = rdma_ah_find_type(ib_dev, cmd.attr.port_num);
- rdma_ah_set_make_grd(&attr, false);
- rdma_ah_set_dlid(&attr, cmd.attr.dlid);
- rdma_ah_set_sl(&attr, cmd.attr.sl);
- rdma_ah_set_path_bits(&attr, cmd.attr.src_path_bits);
- rdma_ah_set_static_rate(&attr, cmd.attr.static_rate);
- rdma_ah_set_port_num(&attr, cmd.attr.port_num);
-
- if (cmd.attr.is_global) {
- rdma_ah_set_grh(&attr, NULL, cmd.attr.grh.flow_label,
- cmd.attr.grh.sgid_index,
- cmd.attr.grh.hop_limit,
- cmd.attr.grh.traffic_class);
- rdma_ah_set_dgid_raw(&attr, cmd.attr.grh.dgid);
- } else {
- rdma_ah_set_ah_flags(&attr, 0);
- }
+ ret = copy_ah_attr_from_uverbs_ah(ib_dev, &attr, &cmd.attr);
+ if (ret)
+ goto err_put;
- ah = rdma_create_user_ah(pd, &attr, &attrs->driver_udata);
+ init_attr.ah_attr = &attr;
+ ah = rdma_create_user_ah(pd, &init_attr, &attrs->driver_udata);
if (IS_ERR(ah)) {
ret = PTR_ERR(ah);
goto err_put;
diff --git a/drivers/infiniband/core/uverbs_std_types.c b/drivers/infiniband/core/uverbs_std_types.c
index e160786e1df1..c72c9e5a2286 100644
--- a/drivers/infiniband/core/uverbs_std_types.c
+++ b/drivers/infiniband/core/uverbs_std_types.c
@@ -184,6 +184,76 @@ DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_MW,
UVERBS_TYPE_ALLOC_IDR(uverbs_free_mw),
&UVERBS_METHOD(UVERBS_METHOD_MW_DESTROY));
+/* Creating an address handle is still a write() command whose request
+ * struct cannot grow. A transport (reliable unconnected) that resolves a
+ * peer using both transport and network addresses requires additional
+ * fields in the address handle. Here the address handle create command is
+ * made available as an ioctl method, where additional attributes can be
+ * carried.
+ */
+static int UVERBS_HANDLER(UVERBS_METHOD_AH_EX_CREATE)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj = uverbs_attr_get_uobject(
+ attrs, UVERBS_ATTR_CREATE_AH_EX_HANDLE);
+ struct ib_pd *pd =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_CREATE_AH_EX_PD_HANDLE);
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_uverbs_ah_attr_ex uah;
+ struct rdma_ah_init_attr init_attr = {};
+ struct rdma_ah_attr_ex attr = {};
+ struct ib_ah *ah;
+ int ret;
+
+ if (IS_ERR(pd))
+ return PTR_ERR(pd);
+
+ /* Only a reliable unconnected device has a use for the extended
+ * attributes.
+ */
+ if (!(ib_dev->attrs.device_cap_flags & IB_DEVICE_RU))
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from(&uah, attrs,
+ UVERBS_ATTR_CREATE_AH_EX_AH_ATTR_EX);
+ if (ret)
+ return ret;
+
+ ret = copy_ah_attr_from_uverbs_ah(ib_dev, &attr.ah_attr, &uah.ah_attr);
+ if (ret)
+ return ret;
+
+ attr.remote_qpn = uah.remote_qpn;
+
+ init_attr.ah_attr = &attr.ah_attr;
+ init_attr.ah_attr_ex = &attr;
+
+ ah = rdma_create_user_ah(pd, &init_attr, &attrs->driver_udata);
+ if (IS_ERR(ah))
+ return PTR_ERR(ah);
+
+ ah->uobject = uobj;
+ uobj->object = ah;
+
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_AH_EX_CREATE,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_AH_EX_HANDLE,
+ UVERBS_OBJECT_AH,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_AH_EX_PD_HANDLE,
+ UVERBS_OBJECT_PD,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_AH_EX_AH_ATTR_EX,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ah_attr_ex,
+ remote_qpn),
+ UA_MANDATORY),
+ UVERBS_ATTR_UHW());
+
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
UVERBS_METHOD_AH_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_AH_HANDLE,
@@ -193,7 +263,8 @@ DECLARE_UVERBS_NAMED_METHOD_DESTROY(
DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_AH,
UVERBS_TYPE_ALLOC_IDR(uverbs_free_ah),
- &UVERBS_METHOD(UVERBS_METHOD_AH_DESTROY));
+ &UVERBS_METHOD(UVERBS_METHOD_AH_DESTROY),
+ &UVERBS_METHOD(UVERBS_METHOD_AH_EX_CREATE));
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
UVERBS_METHOD_FLOW_DESTROY,
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 24db88e6d3fa..8ce7e376112c 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -561,24 +561,23 @@ rdma_update_sgid_attr(struct rdma_ah_attr *ah_attr,
}
static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
- struct rdma_ah_attr *ah_attr,
- u32 flags,
- struct ib_udata *udata,
- struct net_device *xmit_slave)
+ struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata)
{
- struct rdma_ah_init_attr init_attr = {};
+ struct rdma_ah_attr *ah_attr = init_attr->ah_attr;
struct ib_device *device = pd->device;
struct ib_ah *ah;
int ret;
- might_sleep_if(flags & RDMA_CREATE_AH_SLEEPABLE);
+ might_sleep_if(init_attr->flags & RDMA_CREATE_AH_SLEEPABLE);
if (!udata && !device->ops.create_ah)
return ERR_PTR(-EOPNOTSUPP);
ah = rdma_zalloc_drv_obj_gfp(
device, ib_ah,
- (flags & RDMA_CREATE_AH_SLEEPABLE) ? GFP_KERNEL : GFP_ATOMIC);
+ (init_attr->flags & RDMA_CREATE_AH_SLEEPABLE) ? GFP_KERNEL :
+ GFP_ATOMIC);
if (!ah)
return ERR_PTR(-ENOMEM);
@@ -586,14 +585,12 @@ static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
ah->pd = pd;
ah->type = ah_attr->type;
ah->sgid_attr = rdma_update_sgid_attr(ah_attr, NULL);
- init_attr.ah_attr = ah_attr;
- init_attr.flags = flags;
- init_attr.xmit_slave = xmit_slave;
if (udata)
- ret = device->ops.create_user_ah(ah, &init_attr, udata);
+ ret = device->ops.create_user_ah(ah, init_attr, udata);
else
- ret = device->ops.create_ah(ah, &init_attr, NULL);
+ ret = device->ops.create_ah(ah, init_attr, NULL);
+
if (ret) {
if (ah->sgid_attr)
rdma_put_gid_attr(ah->sgid_attr);
@@ -619,6 +616,7 @@ static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
u32 flags)
{
+ struct rdma_ah_init_attr init_attr = {};
const struct ib_gid_attr *old_sgid_attr;
struct net_device *slave;
struct ib_ah *ah;
@@ -634,7 +632,12 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
return ERR_CAST(slave);
}
- ah = _rdma_create_ah(pd, ah_attr, flags, NULL, slave);
+
+ init_attr.ah_attr = ah_attr;
+ init_attr.flags = flags;
+ init_attr.xmit_slave = slave;
+
+ ah = _rdma_create_ah(pd, &init_attr, NULL);
rdma_lag_put_ah_roce_slave(slave);
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
return ah;
@@ -646,18 +649,21 @@ EXPORT_SYMBOL(rdma_create_ah);
* given address vector.
* It resolves destination mac address for ah attribute of RoCE type.
* @pd: The protection domain associated with the address handle.
- * @ah_attr: The attributes of the address vector.
+ * @init_attr: The address vector, and for an extended handle the
+ * attributes beyond it.
* @udata: pointer to user's input output buffer information need by
* provider driver.
*
- * It returns 0 on success and returns appropriate error code on error.
* The address handle is used to reference a local or global destination
* in all UD QP post sends.
+ *
+ * Return: the address handle, or an ERR_PTR on error.
*/
struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
- struct rdma_ah_attr *ah_attr,
+ struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata)
{
+ struct rdma_ah_attr *ah_attr = init_attr->ah_attr;
const struct ib_gid_attr *old_sgid_attr;
struct ib_ah *ah;
int err;
@@ -674,8 +680,9 @@ struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
}
}
- ah = _rdma_create_ah(pd, ah_attr, RDMA_CREATE_AH_SLEEPABLE,
- udata, NULL);
+ init_attr->flags |= RDMA_CREATE_AH_SLEEPABLE;
+
+ ah = _rdma_create_ah(pd, init_attr, udata);
out:
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 8d74ed4fc581..68f7a271cf7a 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -974,10 +974,14 @@ struct ib_mr_status {
*/
__attribute_const__ enum ib_rate mult_to_ib_rate(int mult);
+struct rdma_ah_attr_ex;
+
struct rdma_ah_init_attr {
struct rdma_ah_attr *ah_attr;
u32 flags;
struct net_device *xmit_slave;
+ /* set to name a peer by both transport+network addresses */
+ struct rdma_ah_attr_ex *ah_attr_ex;
};
enum rdma_ah_attr_type {
@@ -1016,6 +1020,11 @@ struct rdma_ah_attr {
};
};
+struct rdma_ah_attr_ex {
+ struct rdma_ah_attr ah_attr;
+ u32 remote_qpn;
+};
+
enum ib_wc_status {
IB_WC_SUCCESS,
IB_WC_LOC_LEN_ERR,
@@ -3893,16 +3902,18 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
* rdma_create_user_ah - Creates an address handle for the given address vector.
* It resolves destination mac address for ah attribute of RoCE type.
* @pd: The protection domain associated with the address handle.
- * @ah_attr: The attributes of the address vector.
+ * @init_attr: The address vector, and for an extended handle the
+ * attributes beyond it.
* @udata: pointer to user's input output buffer information need by
* provider driver.
*
- * It returns 0 on success and returns appropriate error code on error.
* The address handle is used to reference a local or global destination
* in all UD QP post sends.
+ *
+ * Return: the address handle, or an ERR_PTR on error.
*/
struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
- struct rdma_ah_attr *ah_attr,
+ struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata);
/**
* ib_get_gids_from_rdma_hdr - Get sgid and dgid from GRH or IPv4 header
@@ -4981,7 +4992,8 @@ static inline struct ib_global_route
return &attr->grh;
}
-static inline void rdma_ah_set_dgid_raw(struct rdma_ah_attr *attr, void *dgid)
+static inline void rdma_ah_set_dgid_raw(struct rdma_ah_attr *attr,
+ const void *dgid)
{
struct ib_global_route *grh = rdma_ah_retrieve_grh(attr);
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 816b3107464f..40fcd4bd1b5e 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -410,6 +410,13 @@ enum uverbs_attrs_xrcd_destroy_ids {
enum uverbs_methods_ah {
UVERBS_METHOD_AH_DESTROY,
+ UVERBS_METHOD_AH_EX_CREATE,
+};
+
+enum uverbs_attrs_create_ah_ex_cmd_attr_ids {
+ UVERBS_ATTR_CREATE_AH_EX_HANDLE,
+ UVERBS_ATTR_CREATE_AH_EX_PD_HANDLE,
+ UVERBS_ATTR_CREATE_AH_EX_AH_ATTR_EX,
};
enum uverbs_attrs_ah_destroy_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index b6d46115f799..210f3f945bd7 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -318,6 +318,15 @@ struct ib_uverbs_buffer_desc {
#define IB_UVERBS_QPN_UET_RESERVED_MASK 0x7f000000
#define IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT 0x80000000
+/* An address handle that names a peer using both transport+network addresses.
+ * Used for reliable unconnected QPs in both work request address handles and
+ * a Job address table.
+ */
+struct ib_uverbs_ah_attr_ex {
+ struct ib_uverbs_ah_attr ah_attr;
+ __u32 remote_qpn;
+};
+
enum ib_uverbs_comp_cntr_entry {
IB_UVERBS_COMP_CNTR_ENTRY_COMP,
IB_UVERBS_COMP_CNTR_ENTRY_ERR,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 05/13] RDMA/core: extended address handle attributes
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:50 ` sashiko-bot
2026-10-15 16:15 ` Eric Davis
-1 siblings, 1 reply; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:50 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] rdma: uverbs_copy_from() breaks forward ABI compat for zero-padded structs
--
commit 580d7ef2fa7c4af573e68362c952e7b8886ec21b
Author: Eric Davis <eric.davis@broadcom.com>
RDMA/core: extended address handle attributes
This commit adds rdma_ah_attr_ex to carry new attributes and implements
an ioctl method to support extended address handles. It also updates
rdma_create_user_ah() to accept the initialized attributes.
> diff --git a/drivers/infiniband/core/uverbs_std_types.c b/drivers/infiniband/core/uverbs_std_types.c
> index e160786e1df16..c72c9e5a22863 100644
> --- a/drivers/infiniband/core/uverbs_std_types.c
> +++ b/drivers/infiniband/core/uverbs_std_types.c
> @@ -184,6 +184,76 @@ DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_MW,
[ ... ]
> +static int UVERBS_HANDLER(UVERBS_METHOD_AH_EX_CREATE)(
> + struct uverbs_attr_bundle *attrs)
> +{
[ ... ]
> + if (!(ib_dev->attrs.device_cap_flags & IB_DEVICE_RU))
> + return -EOPNOTSUPP;
> +
> + ret = uverbs_copy_from(&uah, attrs,
> + UVERBS_ATTR_CREATE_AH_EX_AH_ATTR_EX);
[Severity: Medium]
Does using uverbs_copy_from() here break forward ABI compatibility?
UVERBS_ATTR_STRUCT sets .zero_trailing = 1, which should permit a newer
userspace to pass a larger zero-padded payload to an older kernel. However,
_uverbs_copy_from() explicitly rejects larger sizes:
include/rdma/uverbs_ioctl.h:_uverbs_copy_from() {
...
if (unlikely(size < attr->ptr_attr.len))
return -EINVAL;
...
}
If a newer userspace application passes a zero-padded payload of 40 bytes to
this kernel (where the structure size is 36), the size check in
_uverbs_copy_from() will evaluate to 36 < 40 and return -EINVAL.
Should this use uverbs_copy_from_or_zero() instead, to properly support
trailing zeros and allow future structure extensions?
> + if (ret)
> + return ret;
> +
> + ret = copy_ah_attr_from_uverbs_ah(ib_dev, &attr.ah_attr, &uah.ah_attr);
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=5
^ permalink raw reply [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 05/13] RDMA/core: extended address handle attributes
2026-10-14 22:50 ` sashiko-bot
@ 2026-10-15 16:15 ` Eric Davis
2026-10-15 17:30 ` Jason Gunthorpe
0 siblings, 1 reply; 60+ messages in thread
From: Eric Davis @ 2026-10-15 16:15 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rdma, Leon Romanovsky, Jason Gunthorpe
[-- Attachment #1: Type: text/plain, Size: 487 bytes --]
On Wed, Oct 14, 2026 at 3:50 PM <sashiko-bot@kernel.org> wrote:
> Should this use uverbs_copy_from_or_zero() instead, to properly support
> trailing zeros and allow future structure extensions?
Agreed, I will fix this in v2. Patch 7's JOB_ADDR_INSERT handler reads the same
struct ib_uverbs_ah_attr_ex the same way. v2 will fix both.
>
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=5
Thanks,
- eric
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5461 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 05/13] RDMA/core: extended address handle attributes
2026-10-15 16:15 ` Eric Davis
@ 2026-10-15 17:30 ` Jason Gunthorpe
0 siblings, 0 replies; 60+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 17:30 UTC (permalink / raw)
To: Eric Davis; +Cc: sashiko-reviews, linux-rdma, Leon Romanovsky
On Thu, Oct 15, 2026 at 09:15:56AM -0700, Eric Davis wrote:
> On Wed, Oct 14, 2026 at 3:50 PM <sashiko-bot@kernel.org> wrote:
> > Should this use uverbs_copy_from_or_zero() instead, to properly support
> > trailing zeros and allow future structure extensions?
>
> Agreed, I will fix this in v2. Patch 7's JOB_ADDR_INSERT handler reads the same
> struct ib_uverbs_ah_attr_ex the same way. v2 will fix both.
Please go through all the uapi stuff and make sure it uses new
mechanisms and helpers all over. If this was written a while ago it is
probably out of date.
Jason
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 06/13] RDMA/core: query the semantics a queue pair type guarantees
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
An unconnected transport orders far less than a connected one, and how
much varies by device. Add a device method that reports, for a queue pair
type, which operations it orders against which, the largest packet it
puts on the wire, how wide an immediate it carries, and whether a memory
region may be attached to a QP so it's only reachable over that QP.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/device.c | 1 +
drivers/infiniband/core/uverbs_std_types_device.c | 73 ++++++++++++++++++++++-
include/rdma/ib_verbs.h | 15 +++++
include/uapi/rdma/ib_user_ioctl_cmds.h | 6 ++
include/uapi/rdma/ib_user_ioctl_verbs.h | 63 +++++++++++++++++++
5 files changed, 157 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 7c65a9373701..8b1563de4726 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -3011,6 +3011,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, query_port);
SET_DEVICE_OP(dev_ops, query_port_speed);
SET_DEVICE_OP(dev_ops, query_qp);
+ SET_DEVICE_OP(dev_ops, query_qp_semantics);
SET_DEVICE_OP(dev_ops, query_srq);
SET_DEVICE_OP(dev_ops, query_ucontext);
SET_DEVICE_OP(dev_ops, rdma_netdev_get_params);
diff --git a/drivers/infiniband/core/uverbs_std_types_device.c b/drivers/infiniband/core/uverbs_std_types_device.c
index db65032e80b4..997fc82f9010 100644
--- a/drivers/infiniband/core/uverbs_std_types_device.c
+++ b/drivers/infiniband/core/uverbs_std_types_device.c
@@ -591,6 +591,76 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL));
+/* What a QP type guarantees on this device. Which operations it orders
+ * against, how large a packet it puts on the wire, how wide an immediate it
+ * carries.
+ */
+static int UVERBS_HANDLER(UVERBS_METHOD_QUERY_QP_SEMANTICS)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uverbs_qp_semantics resp = {};
+ struct ib_qp_semantics sem = {};
+ struct ib_ucontext *ucontext;
+ struct ib_device *ib_dev;
+ enum ib_qp_type qp_type;
+ int ret;
+
+ ucontext = ib_uverbs_get_ucontext(attrs);
+ if (IS_ERR(ucontext))
+ return PTR_ERR(ucontext);
+ ib_dev = ucontext->device;
+
+ if (!ib_dev->ops.query_qp_semantics)
+ return -EOPNOTSUPP;
+
+ ret = uverbs_get_const(&qp_type, attrs,
+ UVERBS_ATTR_QUERY_QP_SEMANTICS_QP_TYPE);
+ if (ret)
+ return ret;
+
+ /* only the types userspace can create */
+ switch (qp_type) {
+ case IB_UVERBS_QPT_RC:
+ case IB_UVERBS_QPT_UC:
+ case IB_UVERBS_QPT_UD:
+ case IB_UVERBS_QPT_RAW_PACKET:
+ case IB_UVERBS_QPT_XRC_INI:
+ case IB_UVERBS_QPT_XRC_TGT:
+ case IB_UVERBS_QPT_RU:
+ case IB_UVERBS_QPT_DRIVER:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ret = ib_dev->ops.query_qp_semantics(ib_dev, qp_type, &sem);
+ if (ret)
+ return ret;
+
+ resp.comp_mask = sem.comp_mask;
+ resp.msg_order = sem.msg_order;
+ resp.max_rdma_raw_size = sem.max_rdma_raw_size;
+ resp.max_rdma_war_size = sem.max_rdma_war_size;
+ resp.max_rdma_waw_size = sem.max_rdma_waw_size;
+ resp.max_pdu = sem.max_pdu;
+ resp.usage_flags = sem.usage_flags;
+ resp.imm_data_size = sem.imm_data_size;
+
+ return uverbs_copy_to_struct_or_zero(
+ attrs, UVERBS_ATTR_QUERY_QP_SEMANTICS_RESP, &resp,
+ sizeof(resp));
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_QUERY_QP_SEMANTICS,
+ UVERBS_ATTR_CONST_IN(UVERBS_ATTR_QUERY_QP_SEMANTICS_QP_TYPE,
+ enum ib_uverbs_qp_type,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_QP_SEMANTICS_RESP,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_qp_semantics,
+ imm_data_size),
+ UA_MANDATORY));
+
DECLARE_UVERBS_GLOBAL_METHODS(UVERBS_OBJECT_DEVICE,
&UVERBS_METHOD(UVERBS_METHOD_GET_CONTEXT),
&UVERBS_METHOD(UVERBS_METHOD_INVOKE_WRITE),
@@ -600,7 +670,8 @@ DECLARE_UVERBS_GLOBAL_METHODS(UVERBS_OBJECT_DEVICE,
&UVERBS_METHOD(UVERBS_METHOD_QUERY_CONTEXT),
&UVERBS_METHOD(UVERBS_METHOD_QUERY_GID_TABLE),
&UVERBS_METHOD(UVERBS_METHOD_QUERY_GID_ENTRY),
- &UVERBS_METHOD(UVERBS_METHOD_QUERY_COMP_CNTR_CAPS));
+ &UVERBS_METHOD(UVERBS_METHOD_QUERY_COMP_CNTR_CAPS),
+ &UVERBS_METHOD(UVERBS_METHOD_QUERY_QP_SEMANTICS));
const struct uapi_definition uverbs_def_obj_device[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_DEVICE),
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 68f7a271cf7a..003b70424429 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2496,6 +2496,18 @@ struct ib_counters {
atomic_t usecnt;
};
+/* what a queue pair of a given type guarantees (query_qp_semantics) */
+struct ib_qp_semantics {
+ u32 comp_mask; /* enum ib_uverbs_qp_semantics_mask */
+ u32 msg_order; /* enum ib_uverbs_qp_msg_order */
+ u32 max_rdma_raw_size;
+ u32 max_rdma_war_size;
+ u32 max_rdma_waw_size;
+ u32 max_pdu;
+ u32 usage_flags; /* enum ib_uverbs_qp_usage_flags */
+ u8 imm_data_size; /* bits */
+};
+
struct ib_counters_read_attr {
u64 *counters_buff;
u32 ncounters;
@@ -2751,6 +2763,9 @@ struct ib_device_ops {
int mr_access_flags, struct ib_pd *pd,
struct ib_udata *udata);
int (*dereg_mr)(struct ib_mr *mr, struct ib_udata *udata);
+ int (*query_qp_semantics)(struct ib_device *device,
+ enum ib_qp_type qp_type,
+ struct ib_qp_semantics *sem);
struct ib_mr *(*alloc_mr)(struct ib_pd *pd, enum ib_mr_type mr_type,
u32 max_num_sg);
struct ib_mr *(*alloc_mr_integrity)(struct ib_pd *pd,
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 40fcd4bd1b5e..fb746257eb2c 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -77,6 +77,12 @@ enum uverbs_methods_device {
UVERBS_METHOD_QUERY_GID_ENTRY,
UVERBS_METHOD_QUERY_PORT_SPEED,
UVERBS_METHOD_QUERY_COMP_CNTR_CAPS,
+ UVERBS_METHOD_QUERY_QP_SEMANTICS,
+};
+
+enum uverbs_attrs_query_qp_semantics_attr_ids {
+ UVERBS_ATTR_QUERY_QP_SEMANTICS_QP_TYPE,
+ UVERBS_ATTR_QUERY_QP_SEMANTICS_RESP,
};
enum uverbs_attrs_invoke_write_cmd_attr_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 210f3f945bd7..3d05a6c7ca8f 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -318,6 +318,50 @@ struct ib_uverbs_buffer_desc {
#define IB_UVERBS_QPN_UET_RESERVED_MASK 0x7f000000
#define IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT 0x80000000
+enum ib_uverbs_qp_semantics_mask {
+ IB_UVERBS_QP_SEMANTICS_MASK_MSG_ORDER = 1 << 0,
+ IB_UVERBS_QP_SEMANTICS_MASK_RAW = 1 << 1,
+ IB_UVERBS_QP_SEMANTICS_MASK_WAR = 1 << 2,
+ IB_UVERBS_QP_SEMANTICS_MASK_WAW = 1 << 3,
+ IB_UVERBS_QP_SEMANTICS_MASK_PDU = 1 << 4,
+ IB_UVERBS_QP_SEMANTICS_MASK_IMM = 1 << 5,
+ IB_UVERBS_QP_SEMANTICS_MASK_USAGE = 1 << 6,
+};
+
+/* Ordering guarantees between operations on one QP. "X after Y" means an
+ * X may not pass a Y posted before it. A bit that is clear means the pair
+ * does not order that combination, and an application that needs it must
+ * wait for the earlier completion.
+ */
+enum ib_uverbs_qp_msg_order {
+ /* atomic after atomic */
+ IB_UVERBS_QP_ORDER_ATOMIC_RAR = 1 << 0,
+ IB_UVERBS_QP_ORDER_ATOMIC_RAW = 1 << 1,
+ IB_UVERBS_QP_ORDER_ATOMIC_WAR = 1 << 2,
+ IB_UVERBS_QP_ORDER_ATOMIC_WAW = 1 << 3,
+ /* rma after rma */
+ IB_UVERBS_QP_ORDER_RDMA_RAR = 1 << 4,
+ IB_UVERBS_QP_ORDER_RDMA_RAW = 1 << 5,
+ IB_UVERBS_QP_ORDER_RDMA_WAR = 1 << 6,
+ IB_UVERBS_QP_ORDER_RDMA_WAW = 1 << 7,
+ /* send against atomic and rma */
+ IB_UVERBS_QP_ORDER_RAS = 1 << 8,
+ IB_UVERBS_QP_ORDER_SAR = 1 << 9,
+ IB_UVERBS_QP_ORDER_SAS = 1 << 10,
+ IB_UVERBS_QP_ORDER_SAW = 1 << 11,
+ IB_UVERBS_QP_ORDER_WAS = 1 << 12,
+ /* rma and atomic, either kind, against each other */
+ IB_UVERBS_QP_ORDER_RAR = 1 << 13,
+ IB_UVERBS_QP_ORDER_RAW = 1 << 14,
+ IB_UVERBS_QP_ORDER_WAR = 1 << 15,
+ IB_UVERBS_QP_ORDER_WAW = 1 << 16,
+};
+
+enum ib_uverbs_qp_usage_flags {
+ /* immediate data consumes a posted receive */
+ IB_UVERBS_QP_USAGE_IMM_DATA_RQ = 1 << 0,
+};
+
/* An address handle that names a peer using both transport+network addresses.
* Used for reliable unconnected QPs in both work request address handles and
* a Job address table.
@@ -327,6 +371,25 @@ struct ib_uverbs_ah_attr_ex {
__u32 remote_qpn;
};
+struct ib_uverbs_qp_semantics {
+ __u32 comp_mask; /* enum ib_uverbs_qp_semantics_mask */
+ __u32 msg_order; /* enum ib_uverbs_qp_msg_order */
+ /*
+ * Data ordering: the largest transfer, in bytes, for which the target
+ * memory reflects the order of the pair under its RDMA_ and plain
+ * msg_order bits. 0 if not provided, and those bits are then clear.
+ * There is no read-after-read size, as two reads leave memory
+ * unchanged.
+ */
+ __u32 max_rdma_raw_size; /* read after write */
+ __u32 max_rdma_war_size; /* write after read */
+ __u32 max_rdma_waw_size; /* write after write */
+ __u32 max_pdu; /* largest packet, bytes */
+ __u32 usage_flags; /* enum ib_uverbs_qp_usage_flags */
+ __u8 imm_data_size; /* widest immediate, in bits */
+ __u8 reserved[3];
+};
+
enum ib_uverbs_comp_cntr_entry {
IB_UVERBS_COMP_CNTR_ENTRY_COMP,
IB_UVERBS_COMP_CNTR_ENTRY_ERR,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 06/13] RDMA/core: query the semantics a queue pair type guarantees
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
An unconnected transport orders far less than a connected one, and how
much varies by device. Add a device method that reports, for a queue pair
type, which operations it orders against which, the largest packet it
puts on the wire, how wide an immediate it carries, and whether a memory
region may be attached to a QP so it's only reachable over that QP.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/device.c | 1 +
drivers/infiniband/core/uverbs_std_types_device.c | 73 ++++++++++++++++++++++-
include/rdma/ib_verbs.h | 15 +++++
include/uapi/rdma/ib_user_ioctl_cmds.h | 6 ++
include/uapi/rdma/ib_user_ioctl_verbs.h | 63 +++++++++++++++++++
5 files changed, 157 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 7c65a9373701..8b1563de4726 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -3011,6 +3011,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, query_port);
SET_DEVICE_OP(dev_ops, query_port_speed);
SET_DEVICE_OP(dev_ops, query_qp);
+ SET_DEVICE_OP(dev_ops, query_qp_semantics);
SET_DEVICE_OP(dev_ops, query_srq);
SET_DEVICE_OP(dev_ops, query_ucontext);
SET_DEVICE_OP(dev_ops, rdma_netdev_get_params);
diff --git a/drivers/infiniband/core/uverbs_std_types_device.c b/drivers/infiniband/core/uverbs_std_types_device.c
index db65032e80b4..997fc82f9010 100644
--- a/drivers/infiniband/core/uverbs_std_types_device.c
+++ b/drivers/infiniband/core/uverbs_std_types_device.c
@@ -591,6 +591,76 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL));
+/* What a QP type guarantees on this device. Which operations it orders
+ * against, how large a packet it puts on the wire, how wide an immediate it
+ * carries.
+ */
+static int UVERBS_HANDLER(UVERBS_METHOD_QUERY_QP_SEMANTICS)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uverbs_qp_semantics resp = {};
+ struct ib_qp_semantics sem = {};
+ struct ib_ucontext *ucontext;
+ struct ib_device *ib_dev;
+ enum ib_qp_type qp_type;
+ int ret;
+
+ ucontext = ib_uverbs_get_ucontext(attrs);
+ if (IS_ERR(ucontext))
+ return PTR_ERR(ucontext);
+ ib_dev = ucontext->device;
+
+ if (!ib_dev->ops.query_qp_semantics)
+ return -EOPNOTSUPP;
+
+ ret = uverbs_get_const(&qp_type, attrs,
+ UVERBS_ATTR_QUERY_QP_SEMANTICS_QP_TYPE);
+ if (ret)
+ return ret;
+
+ /* only the types userspace can create */
+ switch (qp_type) {
+ case IB_UVERBS_QPT_RC:
+ case IB_UVERBS_QPT_UC:
+ case IB_UVERBS_QPT_UD:
+ case IB_UVERBS_QPT_RAW_PACKET:
+ case IB_UVERBS_QPT_XRC_INI:
+ case IB_UVERBS_QPT_XRC_TGT:
+ case IB_UVERBS_QPT_RU:
+ case IB_UVERBS_QPT_DRIVER:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ret = ib_dev->ops.query_qp_semantics(ib_dev, qp_type, &sem);
+ if (ret)
+ return ret;
+
+ resp.comp_mask = sem.comp_mask;
+ resp.msg_order = sem.msg_order;
+ resp.max_rdma_raw_size = sem.max_rdma_raw_size;
+ resp.max_rdma_war_size = sem.max_rdma_war_size;
+ resp.max_rdma_waw_size = sem.max_rdma_waw_size;
+ resp.max_pdu = sem.max_pdu;
+ resp.usage_flags = sem.usage_flags;
+ resp.imm_data_size = sem.imm_data_size;
+
+ return uverbs_copy_to_struct_or_zero(
+ attrs, UVERBS_ATTR_QUERY_QP_SEMANTICS_RESP, &resp,
+ sizeof(resp));
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_QUERY_QP_SEMANTICS,
+ UVERBS_ATTR_CONST_IN(UVERBS_ATTR_QUERY_QP_SEMANTICS_QP_TYPE,
+ enum ib_uverbs_qp_type,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_QP_SEMANTICS_RESP,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_qp_semantics,
+ imm_data_size),
+ UA_MANDATORY));
+
DECLARE_UVERBS_GLOBAL_METHODS(UVERBS_OBJECT_DEVICE,
&UVERBS_METHOD(UVERBS_METHOD_GET_CONTEXT),
&UVERBS_METHOD(UVERBS_METHOD_INVOKE_WRITE),
@@ -600,7 +670,8 @@ DECLARE_UVERBS_GLOBAL_METHODS(UVERBS_OBJECT_DEVICE,
&UVERBS_METHOD(UVERBS_METHOD_QUERY_CONTEXT),
&UVERBS_METHOD(UVERBS_METHOD_QUERY_GID_TABLE),
&UVERBS_METHOD(UVERBS_METHOD_QUERY_GID_ENTRY),
- &UVERBS_METHOD(UVERBS_METHOD_QUERY_COMP_CNTR_CAPS));
+ &UVERBS_METHOD(UVERBS_METHOD_QUERY_COMP_CNTR_CAPS),
+ &UVERBS_METHOD(UVERBS_METHOD_QUERY_QP_SEMANTICS));
const struct uapi_definition uverbs_def_obj_device[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_DEVICE),
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 68f7a271cf7a..003b70424429 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2496,6 +2496,18 @@ struct ib_counters {
atomic_t usecnt;
};
+/* what a queue pair of a given type guarantees (query_qp_semantics) */
+struct ib_qp_semantics {
+ u32 comp_mask; /* enum ib_uverbs_qp_semantics_mask */
+ u32 msg_order; /* enum ib_uverbs_qp_msg_order */
+ u32 max_rdma_raw_size;
+ u32 max_rdma_war_size;
+ u32 max_rdma_waw_size;
+ u32 max_pdu;
+ u32 usage_flags; /* enum ib_uverbs_qp_usage_flags */
+ u8 imm_data_size; /* bits */
+};
+
struct ib_counters_read_attr {
u64 *counters_buff;
u32 ncounters;
@@ -2751,6 +2763,9 @@ struct ib_device_ops {
int mr_access_flags, struct ib_pd *pd,
struct ib_udata *udata);
int (*dereg_mr)(struct ib_mr *mr, struct ib_udata *udata);
+ int (*query_qp_semantics)(struct ib_device *device,
+ enum ib_qp_type qp_type,
+ struct ib_qp_semantics *sem);
struct ib_mr *(*alloc_mr)(struct ib_pd *pd, enum ib_mr_type mr_type,
u32 max_num_sg);
struct ib_mr *(*alloc_mr_integrity)(struct ib_pd *pd,
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 40fcd4bd1b5e..fb746257eb2c 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -77,6 +77,12 @@ enum uverbs_methods_device {
UVERBS_METHOD_QUERY_GID_ENTRY,
UVERBS_METHOD_QUERY_PORT_SPEED,
UVERBS_METHOD_QUERY_COMP_CNTR_CAPS,
+ UVERBS_METHOD_QUERY_QP_SEMANTICS,
+};
+
+enum uverbs_attrs_query_qp_semantics_attr_ids {
+ UVERBS_ATTR_QUERY_QP_SEMANTICS_QP_TYPE,
+ UVERBS_ATTR_QUERY_QP_SEMANTICS_RESP,
};
enum uverbs_attrs_invoke_write_cmd_attr_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 210f3f945bd7..3d05a6c7ca8f 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -318,6 +318,50 @@ struct ib_uverbs_buffer_desc {
#define IB_UVERBS_QPN_UET_RESERVED_MASK 0x7f000000
#define IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT 0x80000000
+enum ib_uverbs_qp_semantics_mask {
+ IB_UVERBS_QP_SEMANTICS_MASK_MSG_ORDER = 1 << 0,
+ IB_UVERBS_QP_SEMANTICS_MASK_RAW = 1 << 1,
+ IB_UVERBS_QP_SEMANTICS_MASK_WAR = 1 << 2,
+ IB_UVERBS_QP_SEMANTICS_MASK_WAW = 1 << 3,
+ IB_UVERBS_QP_SEMANTICS_MASK_PDU = 1 << 4,
+ IB_UVERBS_QP_SEMANTICS_MASK_IMM = 1 << 5,
+ IB_UVERBS_QP_SEMANTICS_MASK_USAGE = 1 << 6,
+};
+
+/* Ordering guarantees between operations on one QP. "X after Y" means an
+ * X may not pass a Y posted before it. A bit that is clear means the pair
+ * does not order that combination, and an application that needs it must
+ * wait for the earlier completion.
+ */
+enum ib_uverbs_qp_msg_order {
+ /* atomic after atomic */
+ IB_UVERBS_QP_ORDER_ATOMIC_RAR = 1 << 0,
+ IB_UVERBS_QP_ORDER_ATOMIC_RAW = 1 << 1,
+ IB_UVERBS_QP_ORDER_ATOMIC_WAR = 1 << 2,
+ IB_UVERBS_QP_ORDER_ATOMIC_WAW = 1 << 3,
+ /* rma after rma */
+ IB_UVERBS_QP_ORDER_RDMA_RAR = 1 << 4,
+ IB_UVERBS_QP_ORDER_RDMA_RAW = 1 << 5,
+ IB_UVERBS_QP_ORDER_RDMA_WAR = 1 << 6,
+ IB_UVERBS_QP_ORDER_RDMA_WAW = 1 << 7,
+ /* send against atomic and rma */
+ IB_UVERBS_QP_ORDER_RAS = 1 << 8,
+ IB_UVERBS_QP_ORDER_SAR = 1 << 9,
+ IB_UVERBS_QP_ORDER_SAS = 1 << 10,
+ IB_UVERBS_QP_ORDER_SAW = 1 << 11,
+ IB_UVERBS_QP_ORDER_WAS = 1 << 12,
+ /* rma and atomic, either kind, against each other */
+ IB_UVERBS_QP_ORDER_RAR = 1 << 13,
+ IB_UVERBS_QP_ORDER_RAW = 1 << 14,
+ IB_UVERBS_QP_ORDER_WAR = 1 << 15,
+ IB_UVERBS_QP_ORDER_WAW = 1 << 16,
+};
+
+enum ib_uverbs_qp_usage_flags {
+ /* immediate data consumes a posted receive */
+ IB_UVERBS_QP_USAGE_IMM_DATA_RQ = 1 << 0,
+};
+
/* An address handle that names a peer using both transport+network addresses.
* Used for reliable unconnected QPs in both work request address handles and
* a Job address table.
@@ -327,6 +371,25 @@ struct ib_uverbs_ah_attr_ex {
__u32 remote_qpn;
};
+struct ib_uverbs_qp_semantics {
+ __u32 comp_mask; /* enum ib_uverbs_qp_semantics_mask */
+ __u32 msg_order; /* enum ib_uverbs_qp_msg_order */
+ /*
+ * Data ordering: the largest transfer, in bytes, for which the target
+ * memory reflects the order of the pair under its RDMA_ and plain
+ * msg_order bits. 0 if not provided, and those bits are then clear.
+ * There is no read-after-read size, as two reads leave memory
+ * unchanged.
+ */
+ __u32 max_rdma_raw_size; /* read after write */
+ __u32 max_rdma_war_size; /* write after read */
+ __u32 max_rdma_waw_size; /* write after write */
+ __u32 max_pdu; /* largest packet, bytes */
+ __u32 usage_flags; /* enum ib_uverbs_qp_usage_flags */
+ __u8 imm_data_size; /* widest immediate, in bits */
+ __u8 reserved[3];
+};
+
enum ib_uverbs_comp_cntr_entry {
IB_UVERBS_COMP_CNTR_ENTRY_COMP,
IB_UVERBS_COMP_CNTR_ENTRY_ERR,
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 06/13] RDMA/core: query the semantics a queue pair type guarantees
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:34 ` sashiko-bot
2026-10-15 16:18 ` Eric Davis
-1 siblings, 1 reply; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:34 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] rdma: query_qp_semantics lacks MR attachment capability flag
--
commit 5f36c96aa3fcf03ebbebf03d834b66a1b7642e0a
Author: Eric Davis <eric.davis@broadcom.com>
RDMA/core: query the semantics a queue pair type guarantees
This commit adds a device method to report the guarantees of a queue pair
type, such as operations ordering, largest packet size, and immediate data
width.
> An unconnected transport orders far less than a connected one, and how
> much varies by device. Add a device method that reports, for a queue pair
> type, which operations it orders against which, the largest packet it
> puts on the wire, how wide an immediate it carries, and whether a memory
> region may be attached to a QP so it's only reachable over that QP.
[Severity: Medium]
The commit message states that query_qp_semantics reports "whether a memory
region may be attached to a QP so it's only reachable over that QP".
Is this description accurate for this patch? I notice that a later commit
reports this capability via query_device instead. Could the commit message
be updated to reflect that the MR attachment capability is not included
in query_qp_semantics?
> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index 68f7a271cf7a7..003b704244293 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
[ ... ]
> @@ -2496,6 +2496,18 @@ struct ib_counters {
> atomic_t usecnt;
> };
>
> +/* what a queue pair of a given type guarantees (query_qp_semantics) */
> +struct ib_qp_semantics {
> + u32 comp_mask; /* enum ib_uverbs_qp_semantics_mask */
> + u32 msg_order; /* enum ib_uverbs_qp_msg_order */
> + u32 max_rdma_raw_size;
> + u32 max_rdma_war_size;
> + u32 max_rdma_waw_size;
> + u32 max_pdu;
> + u32 usage_flags; /* enum ib_uverbs_qp_usage_flags */
> + u8 imm_data_size; /* bits */
> +};
[Severity: Medium]
Does struct ib_qp_semantics omit the MR attachment capability claimed by the
commit message? There doesn't appear to be a field or flag here for it.
> diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
> index 210f3f945bd73..3d05a6c7ca8f6 100644
> --- a/include/uapi/rdma/ib_user_ioctl_verbs.h
> +++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
[ ... ]
> +enum ib_uverbs_qp_usage_flags {
> + /* immediate data consumes a posted receive */
> + IB_UVERBS_QP_USAGE_IMM_DATA_RQ = 1 << 0,
> +};
[Severity: Medium]
The usage flags enum only contains IB_UVERBS_QP_USAGE_IMM_DATA_RQ. Should
there be a flag here indicating whether an MR can be attached, or is this
omission expected since it's later reported via query_device?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=6
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 07/13] RDMA/core: job and job key objects
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
An unconnected transport demultiplexes incoming traffic on a JobID rather
than on a connection. Add two objects for it: a job, which owns that ID
and a table of peer addresses, and a job key, which ties a job to a
protection domain and authorizes a queue pair to act within it.
A job can be exported to a file descriptor and imported elsewhere, so an
unprivileged process can join a job that was set up for it.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/Makefile | 3 +-
drivers/infiniband/core/device.c | 10 +
drivers/infiniband/core/nldev.c | 2 +
drivers/infiniband/core/rdma_core.h | 2 +
drivers/infiniband/core/restrack.c | 4 +
drivers/infiniband/core/uverbs.h | 7 +
drivers/infiniband/core/uverbs_main.c | 5 +
drivers/infiniband/core/uverbs_std_types_job.c | 763 +++++++++++++++++++++++++
drivers/infiniband/core/uverbs_uapi.c | 2 +
drivers/infiniband/core/verbs.c | 38 ++
include/rdma/ib_verbs.h | 70 +++
include/rdma/restrack.h | 8 +
include/rdma/uverbs_ioctl.h | 2 +-
include/uapi/rdma/ib_user_ioctl_cmds.h | 82 +++
14 files changed, 996 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index 47ef6b0afd29..f169c1bfa8b8 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -47,7 +47,8 @@ ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
uverbs_std_types_async_fd.o \
uverbs_std_types_srq.o \
uverbs_std_types_wq.o \
- uverbs_std_types_qp.o
+ uverbs_std_types_qp.o \
+ uverbs_std_types_job.o
ib_uverbs_support-y := rdma_core.o \
ucaps.o \
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 8b1563de4726..cf2a59afcf76 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -2915,6 +2915,14 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, counter_unbind_qp);
SET_DEVICE_OP(dev_ops, counter_update_stats);
SET_DEVICE_OP(dev_ops, create_ah);
+ SET_DEVICE_OP(dev_ops, alloc_job);
+ SET_DEVICE_OP(dev_ops, dealloc_job);
+ SET_DEVICE_OP(dev_ops, query_job);
+ SET_DEVICE_OP(dev_ops, job_addr_insert);
+ SET_DEVICE_OP(dev_ops, job_addr_remove);
+ SET_DEVICE_OP(dev_ops, job_addr_query);
+ SET_DEVICE_OP(dev_ops, create_jkey);
+ SET_DEVICE_OP(dev_ops, destroy_jkey);
SET_DEVICE_OP(dev_ops, create_counters);
SET_DEVICE_OP(dev_ops, create_cq);
SET_DEVICE_OP(dev_ops, create_comp_cntr);
@@ -3031,6 +3039,8 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_OBJ_SIZE(dev_ops, ib_ah);
SET_OBJ_SIZE(dev_ops, ib_counters);
+ SET_OBJ_SIZE(dev_ops, ib_job);
+ SET_OBJ_SIZE(dev_ops, ib_jkey);
SET_OBJ_SIZE(dev_ops, ib_cq);
SET_OBJ_SIZE(dev_ops, ib_comp_cntr);
SET_OBJ_SIZE(dev_ops, ib_dmah);
diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 91b7be84b15f..5313eb4849c7 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -454,6 +454,8 @@ static int fill_res_info(struct sk_buff *msg, struct ib_device *device,
[RDMA_RESTRACK_CTX] = "ctx",
[RDMA_RESTRACK_SRQ] = "srq",
[RDMA_RESTRACK_COMP_CNTR] = "comp_cntr",
+ [RDMA_RESTRACK_JOB] = "job",
+ [RDMA_RESTRACK_JKEY] = "jkey",
};
struct ib_comp_cntr_caps comp_cntr_caps = {};
diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
index 9de14e625dd3..523a6f7d4439 100644
--- a/drivers/infiniband/core/rdma_core.h
+++ b/drivers/infiniband/core/rdma_core.h
@@ -164,6 +164,8 @@ void uverbs_user_mmap_disassociate(struct ib_uverbs_file *ufile);
extern const struct uapi_definition uverbs_def_obj_async_fd[];
extern const struct uapi_definition uverbs_def_obj_counters[];
+extern const struct uapi_definition uverbs_def_obj_job[];
+extern const struct uapi_definition uverbs_def_obj_jkey[];
extern const struct uapi_definition uverbs_def_obj_comp_cntr[];
extern const struct uapi_definition uverbs_def_obj_cq[];
extern const struct uapi_definition uverbs_def_obj_device[];
diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c
index ddb09b17b733..53217f0ad4c8 100644
--- a/drivers/infiniband/core/restrack.c
+++ b/drivers/infiniband/core/restrack.c
@@ -106,6 +106,10 @@ static struct ib_device *res_to_dev(struct rdma_restrack_entry *res)
return container_of(res, struct ib_dmah, res)->device;
case RDMA_RESTRACK_COMP_CNTR:
return container_of(res, struct ib_comp_cntr, res)->device;
+ case RDMA_RESTRACK_JOB:
+ return container_of(res, struct ib_job, res)->device;
+ case RDMA_RESTRACK_JKEY:
+ return container_of(res, struct ib_jkey, res)->device;
default:
WARN_ONCE(true, "Wrong resource tracking type %u\n", res->type);
return NULL;
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index e08c41b8f45b..cecb10ffd635 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -174,6 +174,12 @@ struct ib_uxrcd_object {
atomic_t refcnt;
};
+struct ib_ujob_object {
+ struct ib_uobject uobject;
+ /* job keys created through this handle */
+ atomic_t refcnt;
+};
+
struct ib_usrq_object {
struct ib_uevent_object uevent;
struct ib_uxrcd_object *uxrcd;
@@ -230,6 +236,7 @@ void ib_uverbs_detach_umcast(struct ib_qp *qp,
int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
struct rdma_ah_attr *rdma_attr,
const struct ib_uverbs_ah_attr *uverb_attr);
+void ib_uverbs_revoke_jobs(struct ib_device *device);
struct bundle_alloc_head {
struct_group_tagged(bundle_alloc_head_hdr, hdr,
diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
index 2a046c888dfa..3d1da3083132 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -1282,6 +1282,11 @@ static void ib_uverbs_remove_one(struct ib_device *device, void *client_data)
if (wait_clients)
wait_for_completion(&uverbs_dev->comp);
+ /* Every handle is gone, only an exported job can still hold the
+ * device.
+ */
+ ib_uverbs_revoke_jobs(device);
+
put_device(&uverbs_dev->dev);
}
diff --git a/drivers/infiniband/core/uverbs_std_types_job.c b/drivers/infiniband/core/uverbs_std_types_job.c
new file mode 100644
index 000000000000..8af3e7653e2d
--- /dev/null
+++ b/drivers/infiniband/core/uverbs_std_types_job.c
@@ -0,0 +1,763 @@
+// SPDX-License-Identifier: GPL-2.0-only OR Linux-OpenIB
+
+/*
+ * Copyright (c) 2026 Broadcom. All rights reserved. The term Broadcom
+ * refers to Broadcom Inc. and/or its subsidiaries.
+ */
+
+/*
+ * The job object of a reliable unordered transport.
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
+
+#include <rdma/ib_marshall.h>
+#include <rdma/uverbs_std_types.h>
+
+#include "rdma_core.h"
+#include "restrack.h"
+#include "uverbs.h"
+
+/* Release the job once, whether from its last reference or from a revoke
+ * at device unregister. Called with job->lock held.
+ */
+static void ib_job_teardown(struct ib_job *job)
+{
+ if (job->revoked)
+ return;
+
+ rdma_restrack_del(&job->res);
+ job->device->ops.dealloc_job(job);
+ WRITE_ONCE(job->revoked, true);
+}
+
+static void ib_job_release(struct kref *ref)
+{
+ struct ib_job *job = container_of(ref, struct ib_job, ref);
+ struct ib_device *ib_dev = job->device;
+
+ mutex_lock(&job->lock);
+ ib_job_teardown(job);
+ mutex_unlock(&job->lock);
+
+ mutex_destroy(&job->lock);
+ kfree(job);
+ put_device(&ib_dev->dev);
+}
+
+/* An exported job is a file that no ufile tracks, so it can outlive the
+ * device. Once every handle is gone, release what is left while the driver
+ * is still here; the job then holds only its memory and the device's, until
+ * its last file is closed.
+ */
+void ib_uverbs_revoke_jobs(struct ib_device *ib_dev)
+{
+ struct rdma_restrack_root *rt = &ib_dev->res[RDMA_RESTRACK_JOB];
+ struct rdma_restrack_entry *res;
+ unsigned long id = 0;
+ struct ib_job *job;
+ bool live;
+
+ for (;;) {
+ xa_lock(&rt->xa);
+
+ res = xa_find(&rt->xa, &id, U32_MAX, XA_PRESENT);
+ if (!res) {
+ xa_unlock(&rt->xa);
+ break;
+ }
+
+ /* a job already on its way out removes itself */
+ job = container_of(res, struct ib_job, res);
+ live = kref_get_unless_zero(&job->ref);
+
+ xa_unlock(&rt->xa);
+
+ id++;
+
+ if (!live)
+ continue;
+
+ mutex_lock(&job->lock);
+ ib_job_teardown(job);
+ mutex_unlock(&job->lock);
+ kref_put(&job->ref, ib_job_release);
+ }
+}
+
+static int ib_job_fd_release(struct inode *inode, struct file *filp)
+{
+ kref_put(&((struct ib_job *)filp->private_data)->ref, ib_job_release);
+
+ return 0;
+}
+
+static const struct file_operations ib_job_fops = {
+ .owner = THIS_MODULE,
+ .release = ib_job_fd_release,
+};
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_EXPORT)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_EXPORT_JOB_HANDLE);
+ struct file *filp;
+ int fd, ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ fd = get_unused_fd_flags(O_CLOEXEC);
+ if (fd < 0)
+ return fd;
+
+ /* an exported job is an anonymous file holding one reference */
+ kref_get(&job->ref);
+
+ filp = anon_inode_getfile("[ib_job]", &ib_job_fops, job,
+ (O_RDWR | O_CLOEXEC));
+ if (IS_ERR(filp)) {
+ ret = PTR_ERR(filp);
+ goto err;
+ }
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_EXPORT_JOB_FD, &fd,
+ sizeof(fd));
+ if (ret) {
+ fput(filp);
+ put_unused_fd(fd);
+ return ret;
+ }
+
+ fd_install(fd, filp);
+
+ return 0;
+
+err:
+ kref_put(&job->ref, ib_job_release);
+ put_unused_fd(fd);
+
+ return ret;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_IMPORT)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj =
+ uverbs_attr_get_uobject(attrs, UVERBS_ATTR_IMPORT_JOB_HANDLE);
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_job *job;
+ struct fd f;
+ s32 fd;
+ int ret;
+
+ ret = uverbs_copy_from(&fd, attrs, UVERBS_ATTR_IMPORT_JOB_FD);
+ if (ret)
+ return ret;
+
+ f = fdget(fd);
+ if (!fd_file(f))
+ return -EBADF;
+
+ /* refuse anything that is not an exported job */
+ if (fd_file(f)->f_op != &ib_job_fops) {
+ fdput(f);
+ return -EINVAL;
+ }
+
+ job = fd_file(f)->private_data;
+
+ /* a handle names a job on the device it was allocated on */
+ if (job->device != ib_dev) {
+ fdput(f);
+ return -EINVAL;
+ }
+
+ if (READ_ONCE(job->revoked)) {
+ fdput(f);
+ return -EIO;
+ }
+
+ /* an imported job holds one reference */
+ kref_get(&job->ref);
+ fdput(f);
+
+ uobj->object = job;
+
+ return 0;
+}
+
+static int uverbs_free_job(struct ib_uobject *uobject,
+ enum rdma_remove_reason why,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_ujob_object *ujob =
+ container_of(uobject, struct ib_ujob_object, uobject);
+ struct ib_job *job = uobject->object;
+
+ /* don't free if there are still references to this job */
+ if (atomic_read(&ujob->refcnt))
+ return -EBUSY;
+
+ kref_put(&job->ref, ib_job_release);
+
+ return 0;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ALLOC)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj =
+ uverbs_attr_get_uobject(attrs, UVERBS_ATTR_ALLOC_JOB_HANDLE);
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_job_attr attr = {};
+ struct ib_job *job;
+ int ret;
+
+ if (!ib_dev->ops.alloc_job)
+ return -EOPNOTSUPP;
+
+ /* A JobID names a domain across the fabric rather than anything
+ * local to this process, so claiming one is an administrative act.
+ */
+ if (!rdma_uattrs_has_raw_cap(attrs))
+ return -EPERM;
+
+ ret = uverbs_copy_from(&attr.job_id, attrs, UVERBS_ATTR_ALLOC_JOB_ID);
+ if (ret)
+ return ret;
+
+ /* The rest are optional and an absent attribute reads as -ENOENT
+ * which IS_UVERBS_COPY_ERR() checks.
+ */
+ ret = uverbs_copy_from(&attr.max_addr_entries, attrs,
+ UVERBS_ATTR_ALLOC_JOB_MAX_ADDR_ENTRIES);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_from(&attr.flags, attrs,
+ UVERBS_ATTR_ALLOC_JOB_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_from(&attr.port_num, attrs,
+ UVERBS_ATTR_ALLOC_JOB_PORT_NUM);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_from(&attr.sgid_index, attrs,
+ UVERBS_ATTR_ALLOC_JOB_SGID_INDEX);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ /* no flag is defined yet */
+ if (attr.flags)
+ return -EINVAL;
+
+ if (attr.max_addr_entries > ib_dev->attrs.max_addr_entries)
+ return -EINVAL;
+
+ /* the port carries the job's traffic, so it has to be one we have */
+ if (!attr.port_num)
+ attr.port_num = rdma_start_port(ib_dev);
+ if (!rdma_is_port_valid(ib_dev, attr.port_num))
+ return -EINVAL;
+
+ if (attr.sgid_index >=
+ ib_dev->port_data[attr.port_num].immutable.gid_tbl_len)
+ return -EINVAL;
+
+ job = rdma_zalloc_drv_obj(ib_dev, ib_job);
+ if (!job)
+ return -ENOMEM;
+
+ job->device = ib_dev;
+ job->job_id = attr.job_id;
+ kref_init(&job->ref);
+ mutex_init(&job->lock);
+ uobj->object = job;
+
+ rdma_restrack_new(&job->res, RDMA_RESTRACK_JOB);
+ rdma_restrack_set_name(&job->res, NULL);
+
+ ret = ib_dev->ops.alloc_job(job, &attr, attrs);
+ if (ret)
+ goto err_free;
+
+ /* revoke finds a job through restrack, an untracked one can't exist */
+ rdma_restrack_add(&job->res);
+ if (!job->res.valid) {
+ ib_dev->ops.dealloc_job(job);
+ ret = -ENOMEM;
+ goto err_free;
+ }
+
+ /* an exported job can outlive unregistration, so pin the device's
+ * memory
+ */
+ get_device(&ib_dev->dev);
+
+ return 0;
+
+err_free:
+ rdma_restrack_put(&job->res);
+ mutex_destroy(&job->lock);
+ kfree(job);
+ return ret;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_QUERY)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QUERY_JOB_HANDLE);
+ struct ib_job_attr attr = {};
+ int ret;
+
+ if (!job->device->ops.query_job)
+ return -EOPNOTSUPP;
+
+ ret = job->device->ops.query_job(job, &attr);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_ID, &attr.job_id,
+ sizeof(attr.job_id));
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_MAX_ADDR_ENTRIES,
+ &attr.max_addr_entries,
+ sizeof(attr.max_addr_entries));
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_FLAGS, &attr.flags,
+ sizeof(attr.flags));
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_PORT_NUM,
+ &attr.port_num, sizeof(attr.port_num));
+ if (ret)
+ return ret;
+
+ return uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_SGID_INDEX,
+ &attr.sgid_index, sizeof(attr.sgid_index));
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ADDR_INSERT)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_JOB_ADDR_INSERT_HANDLE);
+ struct ib_uverbs_ah_attr_ex uah;
+ struct rdma_ah_attr_ex ah_attr = {};
+ u32 index, flags = 0;
+ int ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ if (!job->device->ops.job_addr_insert)
+ return -EOPNOTSUPP;
+
+ /* An address table entry names a peer on the fabric for every process
+ * in the job, so writing one redirects their traffic. An address
+ * handle, by contrast, serves only the process that made it.
+ */
+ if (!rdma_uattrs_has_raw_cap(attrs))
+ return -EPERM;
+
+ ret = uverbs_copy_from(&index, attrs,
+ UVERBS_ATTR_JOB_ADDR_INSERT_INDEX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&uah, attrs,
+ UVERBS_ATTR_JOB_ADDR_INSERT_AH_ATTR_EX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&flags, attrs,
+ UVERBS_ATTR_JOB_ADDR_INSERT_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags || index >= job->device->attrs.max_addr_entries)
+ return -EINVAL;
+
+ ret = copy_ah_attr_from_uverbs_ah(job->device, &ah_attr.ah_attr,
+ &uah.ah_attr);
+ if (ret)
+ return ret;
+
+ ah_attr.remote_qpn = uah.remote_qpn;
+
+ return rdma_job_addr_insert(job, index, &ah_attr, flags);
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ADDR_REMOVE)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_JOB_ADDR_REMOVE_HANDLE);
+ u32 index, flags = 0;
+ int ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ if (!job->device->ops.job_addr_remove)
+ return -EOPNOTSUPP;
+
+ if (!rdma_uattrs_has_raw_cap(attrs))
+ return -EPERM;
+
+ ret = uverbs_copy_from(&index, attrs,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_INDEX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&flags, attrs,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags || index >= job->device->attrs.max_addr_entries)
+ return -EINVAL;
+
+ return job->device->ops.job_addr_remove(job, index, flags);
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ADDR_QUERY)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_JOB_ADDR_QUERY_HANDLE);
+ struct ib_uverbs_ah_attr_ex uah = {};
+ struct rdma_ah_attr_ex ah_attr = {};
+ u32 index, flags = 0;
+ int ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ if (!job->device->ops.job_addr_query)
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from(&index, attrs,
+ UVERBS_ATTR_JOB_ADDR_QUERY_INDEX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&flags, attrs,
+ UVERBS_ATTR_JOB_ADDR_QUERY_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags || index >= job->device->attrs.max_addr_entries)
+ return -EINVAL;
+
+ ret = job->device->ops.job_addr_query(job, index, &ah_attr, flags);
+ if (ret)
+ return ret;
+
+ ib_copy_ah_attr_to_user(job->device, &uah.ah_attr, &ah_attr.ah_attr);
+ uah.remote_qpn = ah_attr.remote_qpn;
+
+ return uverbs_copy_to(attrs, UVERBS_ATTR_JOB_ADDR_QUERY_AH_ATTR_EX,
+ &uah, sizeof(uah));
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ADDR_INSERT,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_JOB_ADDR_INSERT_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_INSERT_INDEX,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_INSERT_AH_ATTR_EX,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ah_attr_ex,
+ remote_qpn),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_INSERT_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ADDR_REMOVE,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_JOB_ADDR_REMOVE_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_REMOVE_INDEX,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_REMOVE_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ADDR_QUERY,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_JOB_ADDR_QUERY_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_QUERY_INDEX,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_JOB_ADDR_QUERY_AH_ATTR_EX,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ah_attr_ex,
+ remote_qpn),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_QUERY_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ALLOC,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_ALLOC_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_ID,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_PORT_NUM,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_SGID_INDEX,
+ UVERBS_ATTR_TYPE(u8),
+ UA_OPTIONAL),
+ UVERBS_ATTR_UHW());
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_EXPORT,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_EXPORT_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_EXPORT_JOB_FD,
+ UVERBS_ATTR_TYPE(s32),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_IMPORT,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_IMPORT_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_IMPORT_JOB_FD,
+ UVERBS_ATTR_TYPE(s32),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD_DESTROY(
+ UVERBS_METHOD_JOB_DEALLOC,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_DEALLOC_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_DESTROY,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_QUERY,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QUERY_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_ID,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_PORT_NUM,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_SGID_INDEX,
+ UVERBS_ATTR_TYPE(u8),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_JOB,
+ UVERBS_TYPE_ALLOC_IDR_SZ(
+ sizeof(struct ib_ujob_object),
+ uverbs_free_job),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ALLOC),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_DEALLOC),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_QUERY),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ADDR_INSERT),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ADDR_REMOVE),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ADDR_QUERY),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_EXPORT),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_IMPORT));
+
+const struct uapi_definition uverbs_def_obj_job[] = {
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_JOB,
+ UAPI_DEF_OBJ_NEEDS_FN(dealloc_job)),
+ {}
+};
+
+/* drop what JKEY_CREATE took on the key's job and PD */
+static void ib_jkey_put_refs(struct ib_jkey *jkey)
+{
+ atomic_dec(&jkey->pd->usecnt);
+ atomic_dec(&container_of(jkey->job_uobject, struct ib_ujob_object,
+ uobject)->refcnt);
+ kref_put(&jkey->job->ref, ib_job_release);
+}
+
+static int uverbs_free_jkey(struct ib_uobject *uobject,
+ enum rdma_remove_reason why,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_jkey *jkey = uobject->object;
+ int ret;
+
+ /* queue pairs created with this key still hold it */
+ if (atomic_read(&jkey->usecnt))
+ return -EBUSY;
+
+ rdma_restrack_begin_del(&jkey->res);
+
+ ret = jkey->device->ops.destroy_jkey(jkey, attrs);
+ if (ret) {
+ rdma_restrack_abort_del(&jkey->res);
+ return ret;
+ }
+
+ rdma_restrack_commit_del(&jkey->res);
+
+ ib_jkey_put_refs(jkey);
+ kfree(jkey);
+
+ return 0;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JKEY_CREATE)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj =
+ uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_JKEY_HANDLE);
+ struct ib_uobject *job_uobj = uverbs_attr_get_uobject(
+ attrs, UVERBS_ATTR_CREATE_JKEY_JOB_HANDLE);
+ struct ib_pd *pd =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_CREATE_JKEY_PD_HANDLE);
+ struct ib_job *job;
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_jkey *jkey;
+ u32 flags = 0;
+ int ret;
+
+ if (!ib_dev->ops.create_jkey)
+ return -EOPNOTSUPP;
+
+ if (IS_ERR(job_uobj))
+ return PTR_ERR(job_uobj);
+
+ if (IS_ERR(pd))
+ return PTR_ERR(pd);
+
+ job = job_uobj->object;
+
+ ret = uverbs_copy_from(&flags, attrs, UVERBS_ATTR_CREATE_JKEY_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags)
+ return -EINVAL;
+
+ jkey = rdma_zalloc_drv_obj(ib_dev, ib_jkey);
+ if (!jkey)
+ return -ENOMEM;
+
+ jkey->device = ib_dev;
+ jkey->uobject = uobj;
+ jkey->job = job;
+ jkey->job_uobject = job_uobj;
+ jkey->pd = pd;
+ atomic_set(&jkey->usecnt, 0);
+ uobj->object = jkey;
+
+ rdma_restrack_new(&jkey->res, RDMA_RESTRACK_JKEY);
+ rdma_restrack_set_name(&jkey->res, NULL);
+
+ ret = ib_dev->ops.create_jkey(jkey, flags, attrs);
+ if (ret) {
+ rdma_restrack_put(&jkey->res);
+ kfree(jkey);
+ return ret;
+ }
+
+ rdma_restrack_add(&jkey->res);
+
+ /* the job and PD cannot be freed while this key references them */
+ kref_get(&job->ref);
+ atomic_inc(&container_of(job_uobj, struct ib_ujob_object,
+ uobject)->refcnt);
+ atomic_inc(&pd->usecnt);
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_JKEY_JKEY, &jkey->jkey,
+ sizeof(jkey->jkey));
+ if (ret) {
+ rdma_restrack_del(&jkey->res);
+ ib_dev->ops.destroy_jkey(jkey, attrs);
+ ib_jkey_put_refs(jkey);
+ kfree(jkey);
+ return ret;
+ }
+
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JKEY_CREATE,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_JKEY_HANDLE,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_JKEY_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_JKEY_PD_HANDLE,
+ UVERBS_OBJECT_PD,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_JKEY_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_JKEY_JKEY,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_UHW());
+
+DECLARE_UVERBS_NAMED_METHOD_DESTROY(
+ UVERBS_METHOD_JKEY_DESTROY,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_JKEY_HANDLE,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_DESTROY,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_JKEY,
+ UVERBS_TYPE_ALLOC_IDR(uverbs_free_jkey),
+ &UVERBS_METHOD(UVERBS_METHOD_JKEY_CREATE),
+ &UVERBS_METHOD(UVERBS_METHOD_JKEY_DESTROY));
+
+const struct uapi_definition uverbs_def_obj_jkey[] = {
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_JKEY,
+ UAPI_DEF_OBJ_NEEDS_FN(destroy_jkey)),
+ {}
+};
diff --git a/drivers/infiniband/core/uverbs_uapi.c b/drivers/infiniband/core/uverbs_uapi.c
index b67db3790f03..ee84be3061ea 100644
--- a/drivers/infiniband/core/uverbs_uapi.c
+++ b/drivers/infiniband/core/uverbs_uapi.c
@@ -636,6 +636,8 @@ static const struct uapi_definition uverbs_core_api[] = {
UAPI_DEF_CHAIN(uverbs_def_obj_dmah),
UAPI_DEF_CHAIN(uverbs_def_obj_flow_action),
UAPI_DEF_CHAIN(uverbs_def_obj_intf),
+ UAPI_DEF_CHAIN(uverbs_def_obj_job),
+ UAPI_DEF_CHAIN(uverbs_def_obj_jkey),
UAPI_DEF_CHAIN(uverbs_def_obj_mr),
UAPI_DEF_CHAIN(uverbs_def_obj_qp),
UAPI_DEF_CHAIN(uverbs_def_obj_srq),
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 8ce7e376112c..0a5340642abb 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -690,6 +690,44 @@ struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
}
EXPORT_SYMBOL(rdma_create_user_ah);
+/**
+ * rdma_job_addr_insert - Insert a peer into a job's address table
+ * @job: The job.
+ * @index: The entry in the job's address table.
+ * @ah_attr: The peer's address and queue pair.
+ * @flags: Insert flags.
+ *
+ * The address is resolved as rdma_create_user_ah() resolves an address
+ * handle's: the source GID attribute is filled in, and for an Ethernet
+ * address so is the destination MAC.
+ *
+ * Return: 0 on success, or a negative errno.
+ */
+int rdma_job_addr_insert(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags)
+{
+ struct rdma_ah_attr *attr = &ah_attr->ah_attr;
+ const struct ib_gid_attr *old_sgid_attr;
+ int ret;
+
+ ret = rdma_fill_sgid_attr(job->device, attr, &old_sgid_attr);
+ if (ret)
+ return ret;
+
+ if (attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ ret = ib_resolve_eth_dmac(job->device, attr);
+ if (ret)
+ goto out;
+ }
+
+ ret = job->device->ops.job_addr_insert(job, index, ah_attr, flags);
+
+out:
+ rdma_unfill_sgid_attr(attr, old_sgid_attr);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_job_addr_insert);
+
int ib_get_rdma_header_version(const union rdma_network_hdr *hdr)
{
const struct iphdr *ip4h = (struct iphdr *)&hdr->roce4grh;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 003b70424429..6aa8e7120e71 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2496,6 +2496,43 @@ struct ib_counters {
atomic_t usecnt;
};
+struct ib_job_attr {
+ u32 job_id;
+ u32 max_addr_entries;
+ u32 flags;
+ u32 port_num;
+ u8 sgid_index;
+};
+
+/* A job is a receive-side demultiplexing domain of an unconnected transport.
+ * A JobID names it on the wire, memory regions bind to it, and job keys
+ * authorize queue pairs within it.
+ */
+struct ib_job {
+ struct ib_device *device;
+ /* exports, job keys, regions, and queue pairs referencing this job */
+ struct kref ref;
+ u32 job_id;
+ struct rdma_restrack_entry res;
+ /* serializes release against revoke at device unregister */
+ struct mutex lock;
+ bool revoked;
+};
+
+/* A job key ties a job to a PD and authorizes a queue pair to act within a
+ * job. It holds a reference to its job and PD.
+ */
+struct ib_jkey {
+ struct ib_device *device;
+ struct ib_uobject *uobject;
+ struct ib_job *job;
+ struct ib_uobject *job_uobject;
+ struct ib_pd *pd;
+ u32 jkey;
+ atomic_t usecnt;
+ struct rdma_restrack_entry res;
+};
+
/* what a queue pair of a given type guarantees (query_qp_semantics) */
struct ib_qp_semantics {
u32 comp_mask; /* enum ib_uverbs_qp_semantics_mask */
@@ -2830,6 +2867,20 @@ struct ib_device_ops {
struct ib_mr *(*reg_dm_mr)(struct ib_pd *pd, struct ib_dm *dm,
struct ib_dm_mr_attr *attr,
struct uverbs_attr_bundle *attrs);
+ /* job_addr_* may run concurrently on one job; the driver serializes */
+ int (*alloc_job)(struct ib_job *job, struct ib_job_attr *attr,
+ struct uverbs_attr_bundle *attrs);
+ void (*dealloc_job)(struct ib_job *job);
+ int (*query_job)(struct ib_job *job, struct ib_job_attr *attr);
+ int (*job_addr_insert)(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags);
+ int (*job_addr_remove)(struct ib_job *job, u32 index, u32 flags);
+ int (*job_addr_query)(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags);
+ int (*create_jkey)(struct ib_jkey *jkey, u32 flags,
+ struct uverbs_attr_bundle *attrs);
+ int (*destroy_jkey)(struct ib_jkey *jkey,
+ struct uverbs_attr_bundle *attrs);
int (*create_counters)(struct ib_counters *counters,
struct uverbs_attr_bundle *attrs);
int (*destroy_counters)(struct ib_counters *counters);
@@ -2983,6 +3034,8 @@ struct ib_device_ops {
DECLARE_RDMA_OBJ_SIZE(ib_ah);
DECLARE_RDMA_OBJ_SIZE(ib_counters);
+ DECLARE_RDMA_OBJ_SIZE(ib_job);
+ DECLARE_RDMA_OBJ_SIZE(ib_jkey);
DECLARE_RDMA_OBJ_SIZE(ib_cq);
DECLARE_RDMA_OBJ_SIZE(ib_comp_cntr);
DECLARE_RDMA_OBJ_SIZE(ib_dmah);
@@ -3930,6 +3983,23 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata);
+
+/**
+ * rdma_job_addr_insert - Insert a peer into a job's address table
+ * @job: The job.
+ * @index: The entry in the job's address table.
+ * @ah_attr: The peer's address and queue pair.
+ * @flags: Insert flags.
+ *
+ * The address is resolved as rdma_create_user_ah() resolves an address
+ * handle's: the source GID attribute is filled in, and for an Ethernet
+ * address so is the destination MAC.
+ *
+ * Return: 0 on success, or a negative errno.
+ */
+int rdma_job_addr_insert(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags);
+
/**
* ib_get_gids_from_rdma_hdr - Get sgid and dgid from GRH or IPv4 header
* work completion.
diff --git a/include/rdma/restrack.h b/include/rdma/restrack.h
index 47fec3083d28..a7d53540da67 100644
--- a/include/rdma/restrack.h
+++ b/include/rdma/restrack.h
@@ -64,6 +64,14 @@ enum rdma_restrack_type {
* @RDMA_RESTRACK_COMP_CNTR: Completion Counter
*/
RDMA_RESTRACK_COMP_CNTR,
+ /**
+ * @RDMA_RESTRACK_JOB: Transport job
+ */
+ RDMA_RESTRACK_JOB,
+ /**
+ * @RDMA_RESTRACK_JKEY: Job key
+ */
+ RDMA_RESTRACK_JKEY,
/**
* @RDMA_RESTRACK_MAX: Last entry, used for array dclarations
*/
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index f4d93bc3625a..f19ed5e956a6 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -147,7 +147,7 @@ enum uapi_radix_data {
UVERBS_API_OBJ_KEY_BITS = 6,
UVERBS_API_OBJ_KEY_SHIFT =
UVERBS_API_METHOD_KEY_BITS + UVERBS_API_METHOD_KEY_SHIFT,
- UVERBS_API_OBJ_KEY_NUM_CORE = 20,
+ UVERBS_API_OBJ_KEY_NUM_CORE = 22,
UVERBS_API_OBJ_KEY_NUM_DRIVER =
(1 << UVERBS_API_OBJ_KEY_BITS) - UVERBS_API_OBJ_KEY_NUM_CORE,
UVERBS_API_OBJ_KEY_MASK = GENMASK(31, UVERBS_API_OBJ_KEY_SHIFT),
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index fb746257eb2c..29abfbd227bc 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -58,6 +58,8 @@ enum uverbs_default_objects {
UVERBS_OBJECT_DMAH,
UVERBS_OBJECT_DMABUF,
UVERBS_OBJECT_COMP_CNTR,
+ UVERBS_OBJECT_JOB,
+ UVERBS_OBJECT_JKEY,
};
enum {
@@ -495,4 +497,84 @@ enum uverbs_attrs_read_comp_cntr_cmd_attr_ids {
UVERBS_ATTR_READ_COMP_CNTR_RESP_VALUE,
};
+enum uverbs_methods_job {
+ UVERBS_METHOD_JOB_ALLOC,
+ UVERBS_METHOD_JOB_DEALLOC,
+ UVERBS_METHOD_JOB_QUERY,
+ UVERBS_METHOD_JOB_ADDR_INSERT,
+ UVERBS_METHOD_JOB_ADDR_REMOVE,
+ UVERBS_METHOD_JOB_ADDR_QUERY,
+ UVERBS_METHOD_JOB_EXPORT,
+ UVERBS_METHOD_JOB_IMPORT,
+};
+
+enum uverbs_attrs_alloc_job_cmd_attr_ids {
+ UVERBS_ATTR_ALLOC_JOB_HANDLE,
+ UVERBS_ATTR_ALLOC_JOB_ID,
+ UVERBS_ATTR_ALLOC_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_ALLOC_JOB_FLAGS,
+ UVERBS_ATTR_ALLOC_JOB_PORT_NUM,
+ UVERBS_ATTR_ALLOC_JOB_SGID_INDEX,
+};
+
+enum uverbs_attrs_dealloc_job_cmd_attr_ids {
+ UVERBS_ATTR_DEALLOC_JOB_HANDLE,
+};
+
+enum uverbs_attrs_query_job_cmd_attr_ids {
+ UVERBS_ATTR_QUERY_JOB_HANDLE,
+ UVERBS_ATTR_QUERY_JOB_ID,
+ UVERBS_ATTR_QUERY_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_QUERY_JOB_FLAGS,
+ UVERBS_ATTR_QUERY_JOB_PORT_NUM,
+ UVERBS_ATTR_QUERY_JOB_SGID_INDEX,
+};
+
+enum uverbs_attrs_job_addr_insert_cmd_attr_ids {
+ UVERBS_ATTR_JOB_ADDR_INSERT_HANDLE,
+ UVERBS_ATTR_JOB_ADDR_INSERT_INDEX,
+ UVERBS_ATTR_JOB_ADDR_INSERT_AH_ATTR_EX,
+ UVERBS_ATTR_JOB_ADDR_INSERT_FLAGS,
+};
+
+enum uverbs_attrs_job_addr_remove_cmd_attr_ids {
+ UVERBS_ATTR_JOB_ADDR_REMOVE_HANDLE,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_INDEX,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_FLAGS,
+};
+
+enum uverbs_attrs_job_addr_query_cmd_attr_ids {
+ UVERBS_ATTR_JOB_ADDR_QUERY_HANDLE,
+ UVERBS_ATTR_JOB_ADDR_QUERY_INDEX,
+ UVERBS_ATTR_JOB_ADDR_QUERY_AH_ATTR_EX,
+ UVERBS_ATTR_JOB_ADDR_QUERY_FLAGS,
+};
+
+enum uverbs_attrs_export_job_cmd_attr_ids {
+ UVERBS_ATTR_EXPORT_JOB_HANDLE,
+ UVERBS_ATTR_EXPORT_JOB_FD,
+};
+
+enum uverbs_attrs_import_job_cmd_attr_ids {
+ UVERBS_ATTR_IMPORT_JOB_HANDLE,
+ UVERBS_ATTR_IMPORT_JOB_FD,
+};
+
+enum uverbs_methods_jkey {
+ UVERBS_METHOD_JKEY_CREATE,
+ UVERBS_METHOD_JKEY_DESTROY,
+};
+
+enum uverbs_attrs_create_jkey_cmd_attr_ids {
+ UVERBS_ATTR_CREATE_JKEY_HANDLE,
+ UVERBS_ATTR_CREATE_JKEY_JOB_HANDLE,
+ UVERBS_ATTR_CREATE_JKEY_PD_HANDLE,
+ UVERBS_ATTR_CREATE_JKEY_FLAGS,
+ UVERBS_ATTR_CREATE_JKEY_JKEY,
+};
+
+enum uverbs_attrs_destroy_jkey_cmd_attr_ids {
+ UVERBS_ATTR_DESTROY_JKEY_HANDLE,
+};
+
#endif
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 07/13] RDMA/core: job and job key objects
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
An unconnected transport demultiplexes incoming traffic on a JobID rather
than on a connection. Add two objects for it: a job, which owns that ID
and a table of peer addresses, and a job key, which ties a job to a
protection domain and authorizes a queue pair to act within it.
A job can be exported to a file descriptor and imported elsewhere, so an
unprivileged process can join a job that was set up for it.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/Makefile | 3 +-
drivers/infiniband/core/device.c | 10 +
drivers/infiniband/core/nldev.c | 2 +
drivers/infiniband/core/rdma_core.h | 2 +
drivers/infiniband/core/restrack.c | 4 +
drivers/infiniband/core/uverbs.h | 7 +
drivers/infiniband/core/uverbs_main.c | 5 +
drivers/infiniband/core/uverbs_std_types_job.c | 763 +++++++++++++++++++++++++
drivers/infiniband/core/uverbs_uapi.c | 2 +
drivers/infiniband/core/verbs.c | 38 ++
include/rdma/ib_verbs.h | 70 +++
include/rdma/restrack.h | 8 +
include/rdma/uverbs_ioctl.h | 2 +-
include/uapi/rdma/ib_user_ioctl_cmds.h | 82 +++
14 files changed, 996 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index 47ef6b0afd29..f169c1bfa8b8 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -47,7 +47,8 @@ ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
uverbs_std_types_async_fd.o \
uverbs_std_types_srq.o \
uverbs_std_types_wq.o \
- uverbs_std_types_qp.o
+ uverbs_std_types_qp.o \
+ uverbs_std_types_job.o
ib_uverbs_support-y := rdma_core.o \
ucaps.o \
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 8b1563de4726..cf2a59afcf76 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -2915,6 +2915,14 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, counter_unbind_qp);
SET_DEVICE_OP(dev_ops, counter_update_stats);
SET_DEVICE_OP(dev_ops, create_ah);
+ SET_DEVICE_OP(dev_ops, alloc_job);
+ SET_DEVICE_OP(dev_ops, dealloc_job);
+ SET_DEVICE_OP(dev_ops, query_job);
+ SET_DEVICE_OP(dev_ops, job_addr_insert);
+ SET_DEVICE_OP(dev_ops, job_addr_remove);
+ SET_DEVICE_OP(dev_ops, job_addr_query);
+ SET_DEVICE_OP(dev_ops, create_jkey);
+ SET_DEVICE_OP(dev_ops, destroy_jkey);
SET_DEVICE_OP(dev_ops, create_counters);
SET_DEVICE_OP(dev_ops, create_cq);
SET_DEVICE_OP(dev_ops, create_comp_cntr);
@@ -3031,6 +3039,8 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_OBJ_SIZE(dev_ops, ib_ah);
SET_OBJ_SIZE(dev_ops, ib_counters);
+ SET_OBJ_SIZE(dev_ops, ib_job);
+ SET_OBJ_SIZE(dev_ops, ib_jkey);
SET_OBJ_SIZE(dev_ops, ib_cq);
SET_OBJ_SIZE(dev_ops, ib_comp_cntr);
SET_OBJ_SIZE(dev_ops, ib_dmah);
diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 91b7be84b15f..5313eb4849c7 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -454,6 +454,8 @@ static int fill_res_info(struct sk_buff *msg, struct ib_device *device,
[RDMA_RESTRACK_CTX] = "ctx",
[RDMA_RESTRACK_SRQ] = "srq",
[RDMA_RESTRACK_COMP_CNTR] = "comp_cntr",
+ [RDMA_RESTRACK_JOB] = "job",
+ [RDMA_RESTRACK_JKEY] = "jkey",
};
struct ib_comp_cntr_caps comp_cntr_caps = {};
diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
index 9de14e625dd3..523a6f7d4439 100644
--- a/drivers/infiniband/core/rdma_core.h
+++ b/drivers/infiniband/core/rdma_core.h
@@ -164,6 +164,8 @@ void uverbs_user_mmap_disassociate(struct ib_uverbs_file *ufile);
extern const struct uapi_definition uverbs_def_obj_async_fd[];
extern const struct uapi_definition uverbs_def_obj_counters[];
+extern const struct uapi_definition uverbs_def_obj_job[];
+extern const struct uapi_definition uverbs_def_obj_jkey[];
extern const struct uapi_definition uverbs_def_obj_comp_cntr[];
extern const struct uapi_definition uverbs_def_obj_cq[];
extern const struct uapi_definition uverbs_def_obj_device[];
diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c
index ddb09b17b733..53217f0ad4c8 100644
--- a/drivers/infiniband/core/restrack.c
+++ b/drivers/infiniband/core/restrack.c
@@ -106,6 +106,10 @@ static struct ib_device *res_to_dev(struct rdma_restrack_entry *res)
return container_of(res, struct ib_dmah, res)->device;
case RDMA_RESTRACK_COMP_CNTR:
return container_of(res, struct ib_comp_cntr, res)->device;
+ case RDMA_RESTRACK_JOB:
+ return container_of(res, struct ib_job, res)->device;
+ case RDMA_RESTRACK_JKEY:
+ return container_of(res, struct ib_jkey, res)->device;
default:
WARN_ONCE(true, "Wrong resource tracking type %u\n", res->type);
return NULL;
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index e08c41b8f45b..cecb10ffd635 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -174,6 +174,12 @@ struct ib_uxrcd_object {
atomic_t refcnt;
};
+struct ib_ujob_object {
+ struct ib_uobject uobject;
+ /* job keys created through this handle */
+ atomic_t refcnt;
+};
+
struct ib_usrq_object {
struct ib_uevent_object uevent;
struct ib_uxrcd_object *uxrcd;
@@ -230,6 +236,7 @@ void ib_uverbs_detach_umcast(struct ib_qp *qp,
int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
struct rdma_ah_attr *rdma_attr,
const struct ib_uverbs_ah_attr *uverb_attr);
+void ib_uverbs_revoke_jobs(struct ib_device *device);
struct bundle_alloc_head {
struct_group_tagged(bundle_alloc_head_hdr, hdr,
diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
index 2a046c888dfa..3d1da3083132 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -1282,6 +1282,11 @@ static void ib_uverbs_remove_one(struct ib_device *device, void *client_data)
if (wait_clients)
wait_for_completion(&uverbs_dev->comp);
+ /* Every handle is gone, only an exported job can still hold the
+ * device.
+ */
+ ib_uverbs_revoke_jobs(device);
+
put_device(&uverbs_dev->dev);
}
diff --git a/drivers/infiniband/core/uverbs_std_types_job.c b/drivers/infiniband/core/uverbs_std_types_job.c
new file mode 100644
index 000000000000..8af3e7653e2d
--- /dev/null
+++ b/drivers/infiniband/core/uverbs_std_types_job.c
@@ -0,0 +1,763 @@
+// SPDX-License-Identifier: GPL-2.0-only OR Linux-OpenIB
+
+/*
+ * Copyright (c) 2026 Broadcom. All rights reserved. The term Broadcom
+ * refers to Broadcom Inc. and/or its subsidiaries.
+ */
+
+/*
+ * The job object of a reliable unordered transport.
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
+
+#include <rdma/ib_marshall.h>
+#include <rdma/uverbs_std_types.h>
+
+#include "rdma_core.h"
+#include "restrack.h"
+#include "uverbs.h"
+
+/* Release the job once, whether from its last reference or from a revoke
+ * at device unregister. Called with job->lock held.
+ */
+static void ib_job_teardown(struct ib_job *job)
+{
+ if (job->revoked)
+ return;
+
+ rdma_restrack_del(&job->res);
+ job->device->ops.dealloc_job(job);
+ WRITE_ONCE(job->revoked, true);
+}
+
+static void ib_job_release(struct kref *ref)
+{
+ struct ib_job *job = container_of(ref, struct ib_job, ref);
+ struct ib_device *ib_dev = job->device;
+
+ mutex_lock(&job->lock);
+ ib_job_teardown(job);
+ mutex_unlock(&job->lock);
+
+ mutex_destroy(&job->lock);
+ kfree(job);
+ put_device(&ib_dev->dev);
+}
+
+/* An exported job is a file that no ufile tracks, so it can outlive the
+ * device. Once every handle is gone, release what is left while the driver
+ * is still here; the job then holds only its memory and the device's, until
+ * its last file is closed.
+ */
+void ib_uverbs_revoke_jobs(struct ib_device *ib_dev)
+{
+ struct rdma_restrack_root *rt = &ib_dev->res[RDMA_RESTRACK_JOB];
+ struct rdma_restrack_entry *res;
+ unsigned long id = 0;
+ struct ib_job *job;
+ bool live;
+
+ for (;;) {
+ xa_lock(&rt->xa);
+
+ res = xa_find(&rt->xa, &id, U32_MAX, XA_PRESENT);
+ if (!res) {
+ xa_unlock(&rt->xa);
+ break;
+ }
+
+ /* a job already on its way out removes itself */
+ job = container_of(res, struct ib_job, res);
+ live = kref_get_unless_zero(&job->ref);
+
+ xa_unlock(&rt->xa);
+
+ id++;
+
+ if (!live)
+ continue;
+
+ mutex_lock(&job->lock);
+ ib_job_teardown(job);
+ mutex_unlock(&job->lock);
+ kref_put(&job->ref, ib_job_release);
+ }
+}
+
+static int ib_job_fd_release(struct inode *inode, struct file *filp)
+{
+ kref_put(&((struct ib_job *)filp->private_data)->ref, ib_job_release);
+
+ return 0;
+}
+
+static const struct file_operations ib_job_fops = {
+ .owner = THIS_MODULE,
+ .release = ib_job_fd_release,
+};
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_EXPORT)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_EXPORT_JOB_HANDLE);
+ struct file *filp;
+ int fd, ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ fd = get_unused_fd_flags(O_CLOEXEC);
+ if (fd < 0)
+ return fd;
+
+ /* an exported job is an anonymous file holding one reference */
+ kref_get(&job->ref);
+
+ filp = anon_inode_getfile("[ib_job]", &ib_job_fops, job,
+ (O_RDWR | O_CLOEXEC));
+ if (IS_ERR(filp)) {
+ ret = PTR_ERR(filp);
+ goto err;
+ }
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_EXPORT_JOB_FD, &fd,
+ sizeof(fd));
+ if (ret) {
+ fput(filp);
+ put_unused_fd(fd);
+ return ret;
+ }
+
+ fd_install(fd, filp);
+
+ return 0;
+
+err:
+ kref_put(&job->ref, ib_job_release);
+ put_unused_fd(fd);
+
+ return ret;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_IMPORT)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj =
+ uverbs_attr_get_uobject(attrs, UVERBS_ATTR_IMPORT_JOB_HANDLE);
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_job *job;
+ struct fd f;
+ s32 fd;
+ int ret;
+
+ ret = uverbs_copy_from(&fd, attrs, UVERBS_ATTR_IMPORT_JOB_FD);
+ if (ret)
+ return ret;
+
+ f = fdget(fd);
+ if (!fd_file(f))
+ return -EBADF;
+
+ /* refuse anything that is not an exported job */
+ if (fd_file(f)->f_op != &ib_job_fops) {
+ fdput(f);
+ return -EINVAL;
+ }
+
+ job = fd_file(f)->private_data;
+
+ /* a handle names a job on the device it was allocated on */
+ if (job->device != ib_dev) {
+ fdput(f);
+ return -EINVAL;
+ }
+
+ if (READ_ONCE(job->revoked)) {
+ fdput(f);
+ return -EIO;
+ }
+
+ /* an imported job holds one reference */
+ kref_get(&job->ref);
+ fdput(f);
+
+ uobj->object = job;
+
+ return 0;
+}
+
+static int uverbs_free_job(struct ib_uobject *uobject,
+ enum rdma_remove_reason why,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_ujob_object *ujob =
+ container_of(uobject, struct ib_ujob_object, uobject);
+ struct ib_job *job = uobject->object;
+
+ /* don't free if there are still references to this job */
+ if (atomic_read(&ujob->refcnt))
+ return -EBUSY;
+
+ kref_put(&job->ref, ib_job_release);
+
+ return 0;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ALLOC)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj =
+ uverbs_attr_get_uobject(attrs, UVERBS_ATTR_ALLOC_JOB_HANDLE);
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_job_attr attr = {};
+ struct ib_job *job;
+ int ret;
+
+ if (!ib_dev->ops.alloc_job)
+ return -EOPNOTSUPP;
+
+ /* A JobID names a domain across the fabric rather than anything
+ * local to this process, so claiming one is an administrative act.
+ */
+ if (!rdma_uattrs_has_raw_cap(attrs))
+ return -EPERM;
+
+ ret = uverbs_copy_from(&attr.job_id, attrs, UVERBS_ATTR_ALLOC_JOB_ID);
+ if (ret)
+ return ret;
+
+ /* The rest are optional and an absent attribute reads as -ENOENT
+ * which IS_UVERBS_COPY_ERR() checks.
+ */
+ ret = uverbs_copy_from(&attr.max_addr_entries, attrs,
+ UVERBS_ATTR_ALLOC_JOB_MAX_ADDR_ENTRIES);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_from(&attr.flags, attrs,
+ UVERBS_ATTR_ALLOC_JOB_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_from(&attr.port_num, attrs,
+ UVERBS_ATTR_ALLOC_JOB_PORT_NUM);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_from(&attr.sgid_index, attrs,
+ UVERBS_ATTR_ALLOC_JOB_SGID_INDEX);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ /* no flag is defined yet */
+ if (attr.flags)
+ return -EINVAL;
+
+ if (attr.max_addr_entries > ib_dev->attrs.max_addr_entries)
+ return -EINVAL;
+
+ /* the port carries the job's traffic, so it has to be one we have */
+ if (!attr.port_num)
+ attr.port_num = rdma_start_port(ib_dev);
+ if (!rdma_is_port_valid(ib_dev, attr.port_num))
+ return -EINVAL;
+
+ if (attr.sgid_index >=
+ ib_dev->port_data[attr.port_num].immutable.gid_tbl_len)
+ return -EINVAL;
+
+ job = rdma_zalloc_drv_obj(ib_dev, ib_job);
+ if (!job)
+ return -ENOMEM;
+
+ job->device = ib_dev;
+ job->job_id = attr.job_id;
+ kref_init(&job->ref);
+ mutex_init(&job->lock);
+ uobj->object = job;
+
+ rdma_restrack_new(&job->res, RDMA_RESTRACK_JOB);
+ rdma_restrack_set_name(&job->res, NULL);
+
+ ret = ib_dev->ops.alloc_job(job, &attr, attrs);
+ if (ret)
+ goto err_free;
+
+ /* revoke finds a job through restrack, an untracked one can't exist */
+ rdma_restrack_add(&job->res);
+ if (!job->res.valid) {
+ ib_dev->ops.dealloc_job(job);
+ ret = -ENOMEM;
+ goto err_free;
+ }
+
+ /* an exported job can outlive unregistration, so pin the device's
+ * memory
+ */
+ get_device(&ib_dev->dev);
+
+ return 0;
+
+err_free:
+ rdma_restrack_put(&job->res);
+ mutex_destroy(&job->lock);
+ kfree(job);
+ return ret;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_QUERY)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QUERY_JOB_HANDLE);
+ struct ib_job_attr attr = {};
+ int ret;
+
+ if (!job->device->ops.query_job)
+ return -EOPNOTSUPP;
+
+ ret = job->device->ops.query_job(job, &attr);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_ID, &attr.job_id,
+ sizeof(attr.job_id));
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_MAX_ADDR_ENTRIES,
+ &attr.max_addr_entries,
+ sizeof(attr.max_addr_entries));
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_FLAGS, &attr.flags,
+ sizeof(attr.flags));
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_PORT_NUM,
+ &attr.port_num, sizeof(attr.port_num));
+ if (ret)
+ return ret;
+
+ return uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_JOB_SGID_INDEX,
+ &attr.sgid_index, sizeof(attr.sgid_index));
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ADDR_INSERT)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_JOB_ADDR_INSERT_HANDLE);
+ struct ib_uverbs_ah_attr_ex uah;
+ struct rdma_ah_attr_ex ah_attr = {};
+ u32 index, flags = 0;
+ int ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ if (!job->device->ops.job_addr_insert)
+ return -EOPNOTSUPP;
+
+ /* An address table entry names a peer on the fabric for every process
+ * in the job, so writing one redirects their traffic. An address
+ * handle, by contrast, serves only the process that made it.
+ */
+ if (!rdma_uattrs_has_raw_cap(attrs))
+ return -EPERM;
+
+ ret = uverbs_copy_from(&index, attrs,
+ UVERBS_ATTR_JOB_ADDR_INSERT_INDEX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&uah, attrs,
+ UVERBS_ATTR_JOB_ADDR_INSERT_AH_ATTR_EX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&flags, attrs,
+ UVERBS_ATTR_JOB_ADDR_INSERT_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags || index >= job->device->attrs.max_addr_entries)
+ return -EINVAL;
+
+ ret = copy_ah_attr_from_uverbs_ah(job->device, &ah_attr.ah_attr,
+ &uah.ah_attr);
+ if (ret)
+ return ret;
+
+ ah_attr.remote_qpn = uah.remote_qpn;
+
+ return rdma_job_addr_insert(job, index, &ah_attr, flags);
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ADDR_REMOVE)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_JOB_ADDR_REMOVE_HANDLE);
+ u32 index, flags = 0;
+ int ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ if (!job->device->ops.job_addr_remove)
+ return -EOPNOTSUPP;
+
+ if (!rdma_uattrs_has_raw_cap(attrs))
+ return -EPERM;
+
+ ret = uverbs_copy_from(&index, attrs,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_INDEX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&flags, attrs,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags || index >= job->device->attrs.max_addr_entries)
+ return -EINVAL;
+
+ return job->device->ops.job_addr_remove(job, index, flags);
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JOB_ADDR_QUERY)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_job *job = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_JOB_ADDR_QUERY_HANDLE);
+ struct ib_uverbs_ah_attr_ex uah = {};
+ struct rdma_ah_attr_ex ah_attr = {};
+ u32 index, flags = 0;
+ int ret;
+
+ if (IS_ERR(job))
+ return PTR_ERR(job);
+
+ if (!job->device->ops.job_addr_query)
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from(&index, attrs,
+ UVERBS_ATTR_JOB_ADDR_QUERY_INDEX);
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_from(&flags, attrs,
+ UVERBS_ATTR_JOB_ADDR_QUERY_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags || index >= job->device->attrs.max_addr_entries)
+ return -EINVAL;
+
+ ret = job->device->ops.job_addr_query(job, index, &ah_attr, flags);
+ if (ret)
+ return ret;
+
+ ib_copy_ah_attr_to_user(job->device, &uah.ah_attr, &ah_attr.ah_attr);
+ uah.remote_qpn = ah_attr.remote_qpn;
+
+ return uverbs_copy_to(attrs, UVERBS_ATTR_JOB_ADDR_QUERY_AH_ATTR_EX,
+ &uah, sizeof(uah));
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ADDR_INSERT,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_JOB_ADDR_INSERT_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_INSERT_INDEX,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_INSERT_AH_ATTR_EX,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ah_attr_ex,
+ remote_qpn),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_INSERT_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ADDR_REMOVE,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_JOB_ADDR_REMOVE_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_REMOVE_INDEX,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_REMOVE_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ADDR_QUERY,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_JOB_ADDR_QUERY_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_QUERY_INDEX,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_JOB_ADDR_QUERY_AH_ATTR_EX,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ah_attr_ex,
+ remote_qpn),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_JOB_ADDR_QUERY_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_ALLOC,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_ALLOC_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_ID,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_PORT_NUM,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_JOB_SGID_INDEX,
+ UVERBS_ATTR_TYPE(u8),
+ UA_OPTIONAL),
+ UVERBS_ATTR_UHW());
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_EXPORT,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_EXPORT_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_EXPORT_JOB_FD,
+ UVERBS_ATTR_TYPE(s32),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_IMPORT,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_IMPORT_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_IMPORT_JOB_FD,
+ UVERBS_ATTR_TYPE(s32),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD_DESTROY(
+ UVERBS_METHOD_JOB_DEALLOC,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_DEALLOC_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_DESTROY,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JOB_QUERY,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QUERY_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_ID,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_PORT_NUM,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_JOB_SGID_INDEX,
+ UVERBS_ATTR_TYPE(u8),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_JOB,
+ UVERBS_TYPE_ALLOC_IDR_SZ(
+ sizeof(struct ib_ujob_object),
+ uverbs_free_job),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ALLOC),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_DEALLOC),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_QUERY),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ADDR_INSERT),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ADDR_REMOVE),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_ADDR_QUERY),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_EXPORT),
+ &UVERBS_METHOD(UVERBS_METHOD_JOB_IMPORT));
+
+const struct uapi_definition uverbs_def_obj_job[] = {
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_JOB,
+ UAPI_DEF_OBJ_NEEDS_FN(dealloc_job)),
+ {}
+};
+
+/* drop what JKEY_CREATE took on the key's job and PD */
+static void ib_jkey_put_refs(struct ib_jkey *jkey)
+{
+ atomic_dec(&jkey->pd->usecnt);
+ atomic_dec(&container_of(jkey->job_uobject, struct ib_ujob_object,
+ uobject)->refcnt);
+ kref_put(&jkey->job->ref, ib_job_release);
+}
+
+static int uverbs_free_jkey(struct ib_uobject *uobject,
+ enum rdma_remove_reason why,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_jkey *jkey = uobject->object;
+ int ret;
+
+ /* queue pairs created with this key still hold it */
+ if (atomic_read(&jkey->usecnt))
+ return -EBUSY;
+
+ rdma_restrack_begin_del(&jkey->res);
+
+ ret = jkey->device->ops.destroy_jkey(jkey, attrs);
+ if (ret) {
+ rdma_restrack_abort_del(&jkey->res);
+ return ret;
+ }
+
+ rdma_restrack_commit_del(&jkey->res);
+
+ ib_jkey_put_refs(jkey);
+ kfree(jkey);
+
+ return 0;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_JKEY_CREATE)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj =
+ uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_JKEY_HANDLE);
+ struct ib_uobject *job_uobj = uverbs_attr_get_uobject(
+ attrs, UVERBS_ATTR_CREATE_JKEY_JOB_HANDLE);
+ struct ib_pd *pd =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_CREATE_JKEY_PD_HANDLE);
+ struct ib_job *job;
+ struct ib_device *ib_dev = attrs->context->device;
+ struct ib_jkey *jkey;
+ u32 flags = 0;
+ int ret;
+
+ if (!ib_dev->ops.create_jkey)
+ return -EOPNOTSUPP;
+
+ if (IS_ERR(job_uobj))
+ return PTR_ERR(job_uobj);
+
+ if (IS_ERR(pd))
+ return PTR_ERR(pd);
+
+ job = job_uobj->object;
+
+ ret = uverbs_copy_from(&flags, attrs, UVERBS_ATTR_CREATE_JKEY_FLAGS);
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ if (flags)
+ return -EINVAL;
+
+ jkey = rdma_zalloc_drv_obj(ib_dev, ib_jkey);
+ if (!jkey)
+ return -ENOMEM;
+
+ jkey->device = ib_dev;
+ jkey->uobject = uobj;
+ jkey->job = job;
+ jkey->job_uobject = job_uobj;
+ jkey->pd = pd;
+ atomic_set(&jkey->usecnt, 0);
+ uobj->object = jkey;
+
+ rdma_restrack_new(&jkey->res, RDMA_RESTRACK_JKEY);
+ rdma_restrack_set_name(&jkey->res, NULL);
+
+ ret = ib_dev->ops.create_jkey(jkey, flags, attrs);
+ if (ret) {
+ rdma_restrack_put(&jkey->res);
+ kfree(jkey);
+ return ret;
+ }
+
+ rdma_restrack_add(&jkey->res);
+
+ /* the job and PD cannot be freed while this key references them */
+ kref_get(&job->ref);
+ atomic_inc(&container_of(job_uobj, struct ib_ujob_object,
+ uobject)->refcnt);
+ atomic_inc(&pd->usecnt);
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_JKEY_JKEY, &jkey->jkey,
+ sizeof(jkey->jkey));
+ if (ret) {
+ rdma_restrack_del(&jkey->res);
+ ib_dev->ops.destroy_jkey(jkey, attrs);
+ ib_jkey_put_refs(jkey);
+ kfree(jkey);
+ return ret;
+ }
+
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_JKEY_CREATE,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_JKEY_HANDLE,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_JKEY_JOB_HANDLE,
+ UVERBS_OBJECT_JOB,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_JKEY_PD_HANDLE,
+ UVERBS_OBJECT_PD,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_JKEY_FLAGS,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_JKEY_JKEY,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_UHW());
+
+DECLARE_UVERBS_NAMED_METHOD_DESTROY(
+ UVERBS_METHOD_JKEY_DESTROY,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_JKEY_HANDLE,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_DESTROY,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_JKEY,
+ UVERBS_TYPE_ALLOC_IDR(uverbs_free_jkey),
+ &UVERBS_METHOD(UVERBS_METHOD_JKEY_CREATE),
+ &UVERBS_METHOD(UVERBS_METHOD_JKEY_DESTROY));
+
+const struct uapi_definition uverbs_def_obj_jkey[] = {
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_JKEY,
+ UAPI_DEF_OBJ_NEEDS_FN(destroy_jkey)),
+ {}
+};
diff --git a/drivers/infiniband/core/uverbs_uapi.c b/drivers/infiniband/core/uverbs_uapi.c
index b67db3790f03..ee84be3061ea 100644
--- a/drivers/infiniband/core/uverbs_uapi.c
+++ b/drivers/infiniband/core/uverbs_uapi.c
@@ -636,6 +636,8 @@ static const struct uapi_definition uverbs_core_api[] = {
UAPI_DEF_CHAIN(uverbs_def_obj_dmah),
UAPI_DEF_CHAIN(uverbs_def_obj_flow_action),
UAPI_DEF_CHAIN(uverbs_def_obj_intf),
+ UAPI_DEF_CHAIN(uverbs_def_obj_job),
+ UAPI_DEF_CHAIN(uverbs_def_obj_jkey),
UAPI_DEF_CHAIN(uverbs_def_obj_mr),
UAPI_DEF_CHAIN(uverbs_def_obj_qp),
UAPI_DEF_CHAIN(uverbs_def_obj_srq),
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 8ce7e376112c..0a5340642abb 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -690,6 +690,44 @@ struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
}
EXPORT_SYMBOL(rdma_create_user_ah);
+/**
+ * rdma_job_addr_insert - Insert a peer into a job's address table
+ * @job: The job.
+ * @index: The entry in the job's address table.
+ * @ah_attr: The peer's address and queue pair.
+ * @flags: Insert flags.
+ *
+ * The address is resolved as rdma_create_user_ah() resolves an address
+ * handle's: the source GID attribute is filled in, and for an Ethernet
+ * address so is the destination MAC.
+ *
+ * Return: 0 on success, or a negative errno.
+ */
+int rdma_job_addr_insert(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags)
+{
+ struct rdma_ah_attr *attr = &ah_attr->ah_attr;
+ const struct ib_gid_attr *old_sgid_attr;
+ int ret;
+
+ ret = rdma_fill_sgid_attr(job->device, attr, &old_sgid_attr);
+ if (ret)
+ return ret;
+
+ if (attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ ret = ib_resolve_eth_dmac(job->device, attr);
+ if (ret)
+ goto out;
+ }
+
+ ret = job->device->ops.job_addr_insert(job, index, ah_attr, flags);
+
+out:
+ rdma_unfill_sgid_attr(attr, old_sgid_attr);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_job_addr_insert);
+
int ib_get_rdma_header_version(const union rdma_network_hdr *hdr)
{
const struct iphdr *ip4h = (struct iphdr *)&hdr->roce4grh;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 003b70424429..6aa8e7120e71 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2496,6 +2496,43 @@ struct ib_counters {
atomic_t usecnt;
};
+struct ib_job_attr {
+ u32 job_id;
+ u32 max_addr_entries;
+ u32 flags;
+ u32 port_num;
+ u8 sgid_index;
+};
+
+/* A job is a receive-side demultiplexing domain of an unconnected transport.
+ * A JobID names it on the wire, memory regions bind to it, and job keys
+ * authorize queue pairs within it.
+ */
+struct ib_job {
+ struct ib_device *device;
+ /* exports, job keys, regions, and queue pairs referencing this job */
+ struct kref ref;
+ u32 job_id;
+ struct rdma_restrack_entry res;
+ /* serializes release against revoke at device unregister */
+ struct mutex lock;
+ bool revoked;
+};
+
+/* A job key ties a job to a PD and authorizes a queue pair to act within a
+ * job. It holds a reference to its job and PD.
+ */
+struct ib_jkey {
+ struct ib_device *device;
+ struct ib_uobject *uobject;
+ struct ib_job *job;
+ struct ib_uobject *job_uobject;
+ struct ib_pd *pd;
+ u32 jkey;
+ atomic_t usecnt;
+ struct rdma_restrack_entry res;
+};
+
/* what a queue pair of a given type guarantees (query_qp_semantics) */
struct ib_qp_semantics {
u32 comp_mask; /* enum ib_uverbs_qp_semantics_mask */
@@ -2830,6 +2867,20 @@ struct ib_device_ops {
struct ib_mr *(*reg_dm_mr)(struct ib_pd *pd, struct ib_dm *dm,
struct ib_dm_mr_attr *attr,
struct uverbs_attr_bundle *attrs);
+ /* job_addr_* may run concurrently on one job; the driver serializes */
+ int (*alloc_job)(struct ib_job *job, struct ib_job_attr *attr,
+ struct uverbs_attr_bundle *attrs);
+ void (*dealloc_job)(struct ib_job *job);
+ int (*query_job)(struct ib_job *job, struct ib_job_attr *attr);
+ int (*job_addr_insert)(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags);
+ int (*job_addr_remove)(struct ib_job *job, u32 index, u32 flags);
+ int (*job_addr_query)(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags);
+ int (*create_jkey)(struct ib_jkey *jkey, u32 flags,
+ struct uverbs_attr_bundle *attrs);
+ int (*destroy_jkey)(struct ib_jkey *jkey,
+ struct uverbs_attr_bundle *attrs);
int (*create_counters)(struct ib_counters *counters,
struct uverbs_attr_bundle *attrs);
int (*destroy_counters)(struct ib_counters *counters);
@@ -2983,6 +3034,8 @@ struct ib_device_ops {
DECLARE_RDMA_OBJ_SIZE(ib_ah);
DECLARE_RDMA_OBJ_SIZE(ib_counters);
+ DECLARE_RDMA_OBJ_SIZE(ib_job);
+ DECLARE_RDMA_OBJ_SIZE(ib_jkey);
DECLARE_RDMA_OBJ_SIZE(ib_cq);
DECLARE_RDMA_OBJ_SIZE(ib_comp_cntr);
DECLARE_RDMA_OBJ_SIZE(ib_dmah);
@@ -3930,6 +3983,23 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata);
+
+/**
+ * rdma_job_addr_insert - Insert a peer into a job's address table
+ * @job: The job.
+ * @index: The entry in the job's address table.
+ * @ah_attr: The peer's address and queue pair.
+ * @flags: Insert flags.
+ *
+ * The address is resolved as rdma_create_user_ah() resolves an address
+ * handle's: the source GID attribute is filled in, and for an Ethernet
+ * address so is the destination MAC.
+ *
+ * Return: 0 on success, or a negative errno.
+ */
+int rdma_job_addr_insert(struct ib_job *job, u32 index,
+ struct rdma_ah_attr_ex *ah_attr, u32 flags);
+
/**
* ib_get_gids_from_rdma_hdr - Get sgid and dgid from GRH or IPv4 header
* work completion.
diff --git a/include/rdma/restrack.h b/include/rdma/restrack.h
index 47fec3083d28..a7d53540da67 100644
--- a/include/rdma/restrack.h
+++ b/include/rdma/restrack.h
@@ -64,6 +64,14 @@ enum rdma_restrack_type {
* @RDMA_RESTRACK_COMP_CNTR: Completion Counter
*/
RDMA_RESTRACK_COMP_CNTR,
+ /**
+ * @RDMA_RESTRACK_JOB: Transport job
+ */
+ RDMA_RESTRACK_JOB,
+ /**
+ * @RDMA_RESTRACK_JKEY: Job key
+ */
+ RDMA_RESTRACK_JKEY,
/**
* @RDMA_RESTRACK_MAX: Last entry, used for array dclarations
*/
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index f4d93bc3625a..f19ed5e956a6 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -147,7 +147,7 @@ enum uapi_radix_data {
UVERBS_API_OBJ_KEY_BITS = 6,
UVERBS_API_OBJ_KEY_SHIFT =
UVERBS_API_METHOD_KEY_BITS + UVERBS_API_METHOD_KEY_SHIFT,
- UVERBS_API_OBJ_KEY_NUM_CORE = 20,
+ UVERBS_API_OBJ_KEY_NUM_CORE = 22,
UVERBS_API_OBJ_KEY_NUM_DRIVER =
(1 << UVERBS_API_OBJ_KEY_BITS) - UVERBS_API_OBJ_KEY_NUM_CORE,
UVERBS_API_OBJ_KEY_MASK = GENMASK(31, UVERBS_API_OBJ_KEY_SHIFT),
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index fb746257eb2c..29abfbd227bc 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -58,6 +58,8 @@ enum uverbs_default_objects {
UVERBS_OBJECT_DMAH,
UVERBS_OBJECT_DMABUF,
UVERBS_OBJECT_COMP_CNTR,
+ UVERBS_OBJECT_JOB,
+ UVERBS_OBJECT_JKEY,
};
enum {
@@ -495,4 +497,84 @@ enum uverbs_attrs_read_comp_cntr_cmd_attr_ids {
UVERBS_ATTR_READ_COMP_CNTR_RESP_VALUE,
};
+enum uverbs_methods_job {
+ UVERBS_METHOD_JOB_ALLOC,
+ UVERBS_METHOD_JOB_DEALLOC,
+ UVERBS_METHOD_JOB_QUERY,
+ UVERBS_METHOD_JOB_ADDR_INSERT,
+ UVERBS_METHOD_JOB_ADDR_REMOVE,
+ UVERBS_METHOD_JOB_ADDR_QUERY,
+ UVERBS_METHOD_JOB_EXPORT,
+ UVERBS_METHOD_JOB_IMPORT,
+};
+
+enum uverbs_attrs_alloc_job_cmd_attr_ids {
+ UVERBS_ATTR_ALLOC_JOB_HANDLE,
+ UVERBS_ATTR_ALLOC_JOB_ID,
+ UVERBS_ATTR_ALLOC_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_ALLOC_JOB_FLAGS,
+ UVERBS_ATTR_ALLOC_JOB_PORT_NUM,
+ UVERBS_ATTR_ALLOC_JOB_SGID_INDEX,
+};
+
+enum uverbs_attrs_dealloc_job_cmd_attr_ids {
+ UVERBS_ATTR_DEALLOC_JOB_HANDLE,
+};
+
+enum uverbs_attrs_query_job_cmd_attr_ids {
+ UVERBS_ATTR_QUERY_JOB_HANDLE,
+ UVERBS_ATTR_QUERY_JOB_ID,
+ UVERBS_ATTR_QUERY_JOB_MAX_ADDR_ENTRIES,
+ UVERBS_ATTR_QUERY_JOB_FLAGS,
+ UVERBS_ATTR_QUERY_JOB_PORT_NUM,
+ UVERBS_ATTR_QUERY_JOB_SGID_INDEX,
+};
+
+enum uverbs_attrs_job_addr_insert_cmd_attr_ids {
+ UVERBS_ATTR_JOB_ADDR_INSERT_HANDLE,
+ UVERBS_ATTR_JOB_ADDR_INSERT_INDEX,
+ UVERBS_ATTR_JOB_ADDR_INSERT_AH_ATTR_EX,
+ UVERBS_ATTR_JOB_ADDR_INSERT_FLAGS,
+};
+
+enum uverbs_attrs_job_addr_remove_cmd_attr_ids {
+ UVERBS_ATTR_JOB_ADDR_REMOVE_HANDLE,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_INDEX,
+ UVERBS_ATTR_JOB_ADDR_REMOVE_FLAGS,
+};
+
+enum uverbs_attrs_job_addr_query_cmd_attr_ids {
+ UVERBS_ATTR_JOB_ADDR_QUERY_HANDLE,
+ UVERBS_ATTR_JOB_ADDR_QUERY_INDEX,
+ UVERBS_ATTR_JOB_ADDR_QUERY_AH_ATTR_EX,
+ UVERBS_ATTR_JOB_ADDR_QUERY_FLAGS,
+};
+
+enum uverbs_attrs_export_job_cmd_attr_ids {
+ UVERBS_ATTR_EXPORT_JOB_HANDLE,
+ UVERBS_ATTR_EXPORT_JOB_FD,
+};
+
+enum uverbs_attrs_import_job_cmd_attr_ids {
+ UVERBS_ATTR_IMPORT_JOB_HANDLE,
+ UVERBS_ATTR_IMPORT_JOB_FD,
+};
+
+enum uverbs_methods_jkey {
+ UVERBS_METHOD_JKEY_CREATE,
+ UVERBS_METHOD_JKEY_DESTROY,
+};
+
+enum uverbs_attrs_create_jkey_cmd_attr_ids {
+ UVERBS_ATTR_CREATE_JKEY_HANDLE,
+ UVERBS_ATTR_CREATE_JKEY_JOB_HANDLE,
+ UVERBS_ATTR_CREATE_JKEY_PD_HANDLE,
+ UVERBS_ATTR_CREATE_JKEY_FLAGS,
+ UVERBS_ATTR_CREATE_JKEY_JKEY,
+};
+
+enum uverbs_attrs_destroy_jkey_cmd_attr_ids {
+ UVERBS_ATTR_DESTROY_JKEY_HANDLE,
+};
+
#endif
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 07/13] RDMA/core: job and job key objects
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:36 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:36 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> An unconnected transport demultiplexes incoming traffic on a JobID rather
> than on a connection. Add two objects for it: a job, which owns that ID
> and a table of peer addresses, and a job key, which ties a job to a
> protection domain and authorizes a queue pair to act within it.
>
> A job can be exported to a file descriptor and imported elsewhere, so an
> unprivileged process can join a job that was set up for it.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=7
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 08/13] RDMA/uverbs: factor out the queue pair attribute copy
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
modify_qp() validates a set of attributes that came from userspace and
copies the ones its mask names into their kernel form. Queue pair
creation needs the same work for a transport that is ready to send as
soon as the pair exists, so move it into ib_uverbs_copy_qp_attr().
No functional change.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs.h | 6 ++
drivers/infiniband/core/uverbs_cmd.c | 191 ++++++++++++++++++-----------------
2 files changed, 105 insertions(+), 92 deletions(-)
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index cecb10ffd635..8aebceb2c642 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -311,6 +311,12 @@ struct ib_uverbs_flow_spec {
};
};
+int ib_uverbs_copy_qp_attr(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device,
+ struct ib_qp_attr *attr,
+ const struct ib_uverbs_ex_modify_qp *src,
+ u32 mask);
+
int ib_uverbs_kern_spec_to_ib_spec_filter(enum ib_flow_spec_type type,
const void *kern_spec_mask,
const void *kern_spec_val,
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 5cfbe863df0b..88212465d8a0 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1779,7 +1779,7 @@ int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
static void copy_ah_attr_from_uverbs(struct ib_device *dev,
struct rdma_ah_attr *rdma_attr,
- struct ib_uverbs_qp_dest *uverb_attr)
+ const struct ib_uverbs_qp_dest *uverb_attr)
{
rdma_attr->type = rdma_ah_find_type(dev, uverb_attr->port_num);
if (uverb_attr->is_global) {
@@ -1800,6 +1800,101 @@ static void copy_ah_attr_from_uverbs(struct ib_device *dev,
rdma_ah_set_make_grd(rdma_attr, false);
}
+/* Validate a set of QP attributes that came from userspace and copy the ones
+ * the mask names into their kernel form. Shared by the modify command, and by
+ * queue pair creation which takes the same set of attributes for a transport
+ * that is ready to send as soon as the QP is created.
+ */
+int ib_uverbs_copy_qp_attr(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device,
+ struct ib_qp_attr *attr,
+ const struct ib_uverbs_ex_modify_qp *src,
+ u32 mask)
+{
+ /* Last bit is reserved for extending the attr_mask by using
+ * another field.
+ */
+ if (mask & ~(IB_QP_ATTR_STANDARD_BITS | IB_QP_RATE_LIMIT))
+ return -EOPNOTSUPP;
+
+ if ((mask & IB_QP_PORT) &&
+ !rdma_is_port_valid(device, src->base.port_num))
+ return -EINVAL;
+
+ if ((mask & IB_QP_AV) &&
+ !rdma_is_port_valid(device, src->base.dest.port_num))
+ return -EINVAL;
+
+ if ((mask & IB_QP_ALT_PATH) &&
+ (!rdma_is_port_valid(device, src->base.alt_port_num) ||
+ !rdma_is_port_valid(device, src->base.alt_dest.port_num) ||
+ src->base.alt_port_num != src->base.alt_dest.port_num))
+ return -EINVAL;
+
+ if (((mask & IB_QP_CUR_STATE) && src->base.cur_qp_state > IB_QPS_ERR) ||
+ ((mask & IB_QP_STATE) && src->base.qp_state > IB_QPS_ERR))
+ return -EINVAL;
+
+ if (mask & IB_QP_STATE)
+ attr->qp_state = src->base.qp_state;
+ if (mask & IB_QP_CUR_STATE)
+ attr->cur_qp_state = src->base.cur_qp_state;
+ if (mask & IB_QP_PATH_MTU)
+ attr->path_mtu = src->base.path_mtu;
+ if (mask & IB_QP_PATH_MIG_STATE)
+ attr->path_mig_state = src->base.path_mig_state;
+ if (mask & IB_QP_QKEY) {
+ if (src->base.qkey & IB_QP_SET_QKEY &&
+ !(rdma_nl_get_privileged_qkey() ||
+ rdma_uattrs_has_raw_cap(attrs)))
+ return -EPERM;
+ attr->qkey = src->base.qkey;
+ }
+ if (mask & IB_QP_RQ_PSN)
+ attr->rq_psn = src->base.rq_psn;
+ if (mask & IB_QP_SQ_PSN)
+ attr->sq_psn = src->base.sq_psn;
+ if (mask & IB_QP_DEST_QPN)
+ attr->dest_qp_num = src->base.dest_qp_num;
+ if (mask & IB_QP_ACCESS_FLAGS)
+ attr->qp_access_flags = src->base.qp_access_flags;
+ if (mask & IB_QP_PKEY_INDEX)
+ attr->pkey_index = src->base.pkey_index;
+ if (mask & IB_QP_EN_SQD_ASYNC_NOTIFY)
+ attr->en_sqd_async_notify = src->base.en_sqd_async_notify;
+ if (mask & IB_QP_MAX_QP_RD_ATOMIC)
+ attr->max_rd_atomic = src->base.max_rd_atomic;
+ if (mask & IB_QP_MAX_DEST_RD_ATOMIC)
+ attr->max_dest_rd_atomic = src->base.max_dest_rd_atomic;
+ if (mask & IB_QP_MIN_RNR_TIMER)
+ attr->min_rnr_timer = src->base.min_rnr_timer;
+ if (mask & IB_QP_PORT)
+ attr->port_num = src->base.port_num;
+ if (mask & IB_QP_TIMEOUT)
+ attr->timeout = src->base.timeout;
+ if (mask & IB_QP_RETRY_CNT)
+ attr->retry_cnt = src->base.retry_cnt;
+ if (mask & IB_QP_RNR_RETRY)
+ attr->rnr_retry = src->base.rnr_retry;
+ if (mask & IB_QP_ALT_PATH) {
+ attr->alt_port_num = src->base.alt_port_num;
+ attr->alt_timeout = src->base.alt_timeout;
+ attr->alt_pkey_index = src->base.alt_pkey_index;
+ }
+ if (mask & IB_QP_RATE_LIMIT)
+ attr->rate_limit = src->rate_limit;
+
+ if (mask & IB_QP_AV)
+ copy_ah_attr_from_uverbs(device, &attr->ah_attr,
+ &src->base.dest);
+
+ if (mask & IB_QP_ALT_PATH)
+ copy_ah_attr_from_uverbs(device, &attr->alt_ah_attr,
+ &src->base.alt_dest);
+
+ return 0;
+}
+
static int modify_qp(struct uverbs_attr_bundle *attrs,
struct ib_uverbs_ex_modify_qp *cmd)
{
@@ -1818,18 +1913,7 @@ static int modify_qp(struct uverbs_attr_bundle *attrs,
goto out;
}
- if ((cmd->base.attr_mask & IB_QP_PORT) &&
- !rdma_is_port_valid(qp->device, cmd->base.port_num)) {
- ret = -EINVAL;
- goto release_qp;
- }
-
if ((cmd->base.attr_mask & IB_QP_AV)) {
- if (!rdma_is_port_valid(qp->device, cmd->base.dest.port_num)) {
- ret = -EINVAL;
- goto release_qp;
- }
-
if (cmd->base.attr_mask & IB_QP_STATE &&
cmd->base.qp_state == IB_QPS_RTR) {
/* We are in INIT->RTR TRANSITION (if we are not,
@@ -1878,80 +1962,10 @@ static int modify_qp(struct uverbs_attr_bundle *attrs,
}
}
- if ((cmd->base.attr_mask & IB_QP_ALT_PATH) &&
- (!rdma_is_port_valid(qp->device, cmd->base.alt_port_num) ||
- !rdma_is_port_valid(qp->device, cmd->base.alt_dest.port_num) ||
- cmd->base.alt_port_num != cmd->base.alt_dest.port_num)) {
- ret = -EINVAL;
- goto release_qp;
- }
-
- if ((cmd->base.attr_mask & IB_QP_CUR_STATE &&
- cmd->base.cur_qp_state > IB_QPS_ERR) ||
- (cmd->base.attr_mask & IB_QP_STATE &&
- cmd->base.qp_state > IB_QPS_ERR)) {
- ret = -EINVAL;
+ ret = ib_uverbs_copy_qp_attr(attrs, qp->device, attr, cmd,
+ cmd->base.attr_mask);
+ if (ret)
goto release_qp;
- }
-
- if (cmd->base.attr_mask & IB_QP_STATE)
- attr->qp_state = cmd->base.qp_state;
- if (cmd->base.attr_mask & IB_QP_CUR_STATE)
- attr->cur_qp_state = cmd->base.cur_qp_state;
- if (cmd->base.attr_mask & IB_QP_PATH_MTU)
- attr->path_mtu = cmd->base.path_mtu;
- if (cmd->base.attr_mask & IB_QP_PATH_MIG_STATE)
- attr->path_mig_state = cmd->base.path_mig_state;
- if (cmd->base.attr_mask & IB_QP_QKEY) {
- if (cmd->base.qkey & IB_QP_SET_QKEY &&
- !(rdma_nl_get_privileged_qkey() ||
- rdma_uattrs_has_raw_cap(attrs))) {
- ret = -EPERM;
- goto release_qp;
- }
- attr->qkey = cmd->base.qkey;
- }
- if (cmd->base.attr_mask & IB_QP_RQ_PSN)
- attr->rq_psn = cmd->base.rq_psn;
- if (cmd->base.attr_mask & IB_QP_SQ_PSN)
- attr->sq_psn = cmd->base.sq_psn;
- if (cmd->base.attr_mask & IB_QP_DEST_QPN)
- attr->dest_qp_num = cmd->base.dest_qp_num;
- if (cmd->base.attr_mask & IB_QP_ACCESS_FLAGS)
- attr->qp_access_flags = cmd->base.qp_access_flags;
- if (cmd->base.attr_mask & IB_QP_PKEY_INDEX)
- attr->pkey_index = cmd->base.pkey_index;
- if (cmd->base.attr_mask & IB_QP_EN_SQD_ASYNC_NOTIFY)
- attr->en_sqd_async_notify = cmd->base.en_sqd_async_notify;
- if (cmd->base.attr_mask & IB_QP_MAX_QP_RD_ATOMIC)
- attr->max_rd_atomic = cmd->base.max_rd_atomic;
- if (cmd->base.attr_mask & IB_QP_MAX_DEST_RD_ATOMIC)
- attr->max_dest_rd_atomic = cmd->base.max_dest_rd_atomic;
- if (cmd->base.attr_mask & IB_QP_MIN_RNR_TIMER)
- attr->min_rnr_timer = cmd->base.min_rnr_timer;
- if (cmd->base.attr_mask & IB_QP_PORT)
- attr->port_num = cmd->base.port_num;
- if (cmd->base.attr_mask & IB_QP_TIMEOUT)
- attr->timeout = cmd->base.timeout;
- if (cmd->base.attr_mask & IB_QP_RETRY_CNT)
- attr->retry_cnt = cmd->base.retry_cnt;
- if (cmd->base.attr_mask & IB_QP_RNR_RETRY)
- attr->rnr_retry = cmd->base.rnr_retry;
- if (cmd->base.attr_mask & IB_QP_ALT_PATH) {
- attr->alt_port_num = cmd->base.alt_port_num;
- attr->alt_timeout = cmd->base.alt_timeout;
- attr->alt_pkey_index = cmd->base.alt_pkey_index;
- }
- if (cmd->base.attr_mask & IB_QP_RATE_LIMIT)
- attr->rate_limit = cmd->rate_limit;
-
- if (cmd->base.attr_mask & IB_QP_AV)
- copy_ah_attr_from_uverbs(qp->device, &attr->ah_attr,
- &cmd->base.dest);
-
- if (cmd->base.attr_mask & IB_QP_ALT_PATH)
- copy_ah_attr_from_uverbs(qp->device, &attr->alt_ah_attr,
- &cmd->base.alt_dest);
ret = ib_modify_qp_with_udata(qp, attr,
modify_qp_mask(qp->qp_type,
@@ -1994,13 +2008,6 @@ static int ib_uverbs_ex_modify_qp(struct uverbs_attr_bundle *attrs)
if (ret)
return ret;
- /*
- * Last bit is reserved for extending the attr_mask by
- * using another field.
- */
- if (cmd.base.attr_mask & ~(IB_QP_ATTR_STANDARD_BITS | IB_QP_RATE_LIMIT))
- return -EOPNOTSUPP;
-
ret = modify_qp(attrs, &cmd);
if (ret)
return ret;
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 08/13] RDMA/uverbs: factor out the queue pair attribute copy
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
modify_qp() validates a set of attributes that came from userspace and
copies the ones its mask names into their kernel form. Queue pair
creation needs the same work for a transport that is ready to send as
soon as the pair exists, so move it into ib_uverbs_copy_qp_attr().
No functional change.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs.h | 6 ++
drivers/infiniband/core/uverbs_cmd.c | 191 ++++++++++++++++++-----------------
2 files changed, 105 insertions(+), 92 deletions(-)
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index cecb10ffd635..8aebceb2c642 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -311,6 +311,12 @@ struct ib_uverbs_flow_spec {
};
};
+int ib_uverbs_copy_qp_attr(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device,
+ struct ib_qp_attr *attr,
+ const struct ib_uverbs_ex_modify_qp *src,
+ u32 mask);
+
int ib_uverbs_kern_spec_to_ib_spec_filter(enum ib_flow_spec_type type,
const void *kern_spec_mask,
const void *kern_spec_val,
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 5cfbe863df0b..88212465d8a0 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1779,7 +1779,7 @@ int copy_ah_attr_from_uverbs_ah(struct ib_device *dev,
static void copy_ah_attr_from_uverbs(struct ib_device *dev,
struct rdma_ah_attr *rdma_attr,
- struct ib_uverbs_qp_dest *uverb_attr)
+ const struct ib_uverbs_qp_dest *uverb_attr)
{
rdma_attr->type = rdma_ah_find_type(dev, uverb_attr->port_num);
if (uverb_attr->is_global) {
@@ -1800,6 +1800,101 @@ static void copy_ah_attr_from_uverbs(struct ib_device *dev,
rdma_ah_set_make_grd(rdma_attr, false);
}
+/* Validate a set of QP attributes that came from userspace and copy the ones
+ * the mask names into their kernel form. Shared by the modify command, and by
+ * queue pair creation which takes the same set of attributes for a transport
+ * that is ready to send as soon as the QP is created.
+ */
+int ib_uverbs_copy_qp_attr(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device,
+ struct ib_qp_attr *attr,
+ const struct ib_uverbs_ex_modify_qp *src,
+ u32 mask)
+{
+ /* Last bit is reserved for extending the attr_mask by using
+ * another field.
+ */
+ if (mask & ~(IB_QP_ATTR_STANDARD_BITS | IB_QP_RATE_LIMIT))
+ return -EOPNOTSUPP;
+
+ if ((mask & IB_QP_PORT) &&
+ !rdma_is_port_valid(device, src->base.port_num))
+ return -EINVAL;
+
+ if ((mask & IB_QP_AV) &&
+ !rdma_is_port_valid(device, src->base.dest.port_num))
+ return -EINVAL;
+
+ if ((mask & IB_QP_ALT_PATH) &&
+ (!rdma_is_port_valid(device, src->base.alt_port_num) ||
+ !rdma_is_port_valid(device, src->base.alt_dest.port_num) ||
+ src->base.alt_port_num != src->base.alt_dest.port_num))
+ return -EINVAL;
+
+ if (((mask & IB_QP_CUR_STATE) && src->base.cur_qp_state > IB_QPS_ERR) ||
+ ((mask & IB_QP_STATE) && src->base.qp_state > IB_QPS_ERR))
+ return -EINVAL;
+
+ if (mask & IB_QP_STATE)
+ attr->qp_state = src->base.qp_state;
+ if (mask & IB_QP_CUR_STATE)
+ attr->cur_qp_state = src->base.cur_qp_state;
+ if (mask & IB_QP_PATH_MTU)
+ attr->path_mtu = src->base.path_mtu;
+ if (mask & IB_QP_PATH_MIG_STATE)
+ attr->path_mig_state = src->base.path_mig_state;
+ if (mask & IB_QP_QKEY) {
+ if (src->base.qkey & IB_QP_SET_QKEY &&
+ !(rdma_nl_get_privileged_qkey() ||
+ rdma_uattrs_has_raw_cap(attrs)))
+ return -EPERM;
+ attr->qkey = src->base.qkey;
+ }
+ if (mask & IB_QP_RQ_PSN)
+ attr->rq_psn = src->base.rq_psn;
+ if (mask & IB_QP_SQ_PSN)
+ attr->sq_psn = src->base.sq_psn;
+ if (mask & IB_QP_DEST_QPN)
+ attr->dest_qp_num = src->base.dest_qp_num;
+ if (mask & IB_QP_ACCESS_FLAGS)
+ attr->qp_access_flags = src->base.qp_access_flags;
+ if (mask & IB_QP_PKEY_INDEX)
+ attr->pkey_index = src->base.pkey_index;
+ if (mask & IB_QP_EN_SQD_ASYNC_NOTIFY)
+ attr->en_sqd_async_notify = src->base.en_sqd_async_notify;
+ if (mask & IB_QP_MAX_QP_RD_ATOMIC)
+ attr->max_rd_atomic = src->base.max_rd_atomic;
+ if (mask & IB_QP_MAX_DEST_RD_ATOMIC)
+ attr->max_dest_rd_atomic = src->base.max_dest_rd_atomic;
+ if (mask & IB_QP_MIN_RNR_TIMER)
+ attr->min_rnr_timer = src->base.min_rnr_timer;
+ if (mask & IB_QP_PORT)
+ attr->port_num = src->base.port_num;
+ if (mask & IB_QP_TIMEOUT)
+ attr->timeout = src->base.timeout;
+ if (mask & IB_QP_RETRY_CNT)
+ attr->retry_cnt = src->base.retry_cnt;
+ if (mask & IB_QP_RNR_RETRY)
+ attr->rnr_retry = src->base.rnr_retry;
+ if (mask & IB_QP_ALT_PATH) {
+ attr->alt_port_num = src->base.alt_port_num;
+ attr->alt_timeout = src->base.alt_timeout;
+ attr->alt_pkey_index = src->base.alt_pkey_index;
+ }
+ if (mask & IB_QP_RATE_LIMIT)
+ attr->rate_limit = src->rate_limit;
+
+ if (mask & IB_QP_AV)
+ copy_ah_attr_from_uverbs(device, &attr->ah_attr,
+ &src->base.dest);
+
+ if (mask & IB_QP_ALT_PATH)
+ copy_ah_attr_from_uverbs(device, &attr->alt_ah_attr,
+ &src->base.alt_dest);
+
+ return 0;
+}
+
static int modify_qp(struct uverbs_attr_bundle *attrs,
struct ib_uverbs_ex_modify_qp *cmd)
{
@@ -1818,18 +1913,7 @@ static int modify_qp(struct uverbs_attr_bundle *attrs,
goto out;
}
- if ((cmd->base.attr_mask & IB_QP_PORT) &&
- !rdma_is_port_valid(qp->device, cmd->base.port_num)) {
- ret = -EINVAL;
- goto release_qp;
- }
-
if ((cmd->base.attr_mask & IB_QP_AV)) {
- if (!rdma_is_port_valid(qp->device, cmd->base.dest.port_num)) {
- ret = -EINVAL;
- goto release_qp;
- }
-
if (cmd->base.attr_mask & IB_QP_STATE &&
cmd->base.qp_state == IB_QPS_RTR) {
/* We are in INIT->RTR TRANSITION (if we are not,
@@ -1878,80 +1962,10 @@ static int modify_qp(struct uverbs_attr_bundle *attrs,
}
}
- if ((cmd->base.attr_mask & IB_QP_ALT_PATH) &&
- (!rdma_is_port_valid(qp->device, cmd->base.alt_port_num) ||
- !rdma_is_port_valid(qp->device, cmd->base.alt_dest.port_num) ||
- cmd->base.alt_port_num != cmd->base.alt_dest.port_num)) {
- ret = -EINVAL;
- goto release_qp;
- }
-
- if ((cmd->base.attr_mask & IB_QP_CUR_STATE &&
- cmd->base.cur_qp_state > IB_QPS_ERR) ||
- (cmd->base.attr_mask & IB_QP_STATE &&
- cmd->base.qp_state > IB_QPS_ERR)) {
- ret = -EINVAL;
+ ret = ib_uverbs_copy_qp_attr(attrs, qp->device, attr, cmd,
+ cmd->base.attr_mask);
+ if (ret)
goto release_qp;
- }
-
- if (cmd->base.attr_mask & IB_QP_STATE)
- attr->qp_state = cmd->base.qp_state;
- if (cmd->base.attr_mask & IB_QP_CUR_STATE)
- attr->cur_qp_state = cmd->base.cur_qp_state;
- if (cmd->base.attr_mask & IB_QP_PATH_MTU)
- attr->path_mtu = cmd->base.path_mtu;
- if (cmd->base.attr_mask & IB_QP_PATH_MIG_STATE)
- attr->path_mig_state = cmd->base.path_mig_state;
- if (cmd->base.attr_mask & IB_QP_QKEY) {
- if (cmd->base.qkey & IB_QP_SET_QKEY &&
- !(rdma_nl_get_privileged_qkey() ||
- rdma_uattrs_has_raw_cap(attrs))) {
- ret = -EPERM;
- goto release_qp;
- }
- attr->qkey = cmd->base.qkey;
- }
- if (cmd->base.attr_mask & IB_QP_RQ_PSN)
- attr->rq_psn = cmd->base.rq_psn;
- if (cmd->base.attr_mask & IB_QP_SQ_PSN)
- attr->sq_psn = cmd->base.sq_psn;
- if (cmd->base.attr_mask & IB_QP_DEST_QPN)
- attr->dest_qp_num = cmd->base.dest_qp_num;
- if (cmd->base.attr_mask & IB_QP_ACCESS_FLAGS)
- attr->qp_access_flags = cmd->base.qp_access_flags;
- if (cmd->base.attr_mask & IB_QP_PKEY_INDEX)
- attr->pkey_index = cmd->base.pkey_index;
- if (cmd->base.attr_mask & IB_QP_EN_SQD_ASYNC_NOTIFY)
- attr->en_sqd_async_notify = cmd->base.en_sqd_async_notify;
- if (cmd->base.attr_mask & IB_QP_MAX_QP_RD_ATOMIC)
- attr->max_rd_atomic = cmd->base.max_rd_atomic;
- if (cmd->base.attr_mask & IB_QP_MAX_DEST_RD_ATOMIC)
- attr->max_dest_rd_atomic = cmd->base.max_dest_rd_atomic;
- if (cmd->base.attr_mask & IB_QP_MIN_RNR_TIMER)
- attr->min_rnr_timer = cmd->base.min_rnr_timer;
- if (cmd->base.attr_mask & IB_QP_PORT)
- attr->port_num = cmd->base.port_num;
- if (cmd->base.attr_mask & IB_QP_TIMEOUT)
- attr->timeout = cmd->base.timeout;
- if (cmd->base.attr_mask & IB_QP_RETRY_CNT)
- attr->retry_cnt = cmd->base.retry_cnt;
- if (cmd->base.attr_mask & IB_QP_RNR_RETRY)
- attr->rnr_retry = cmd->base.rnr_retry;
- if (cmd->base.attr_mask & IB_QP_ALT_PATH) {
- attr->alt_port_num = cmd->base.alt_port_num;
- attr->alt_timeout = cmd->base.alt_timeout;
- attr->alt_pkey_index = cmd->base.alt_pkey_index;
- }
- if (cmd->base.attr_mask & IB_QP_RATE_LIMIT)
- attr->rate_limit = cmd->rate_limit;
-
- if (cmd->base.attr_mask & IB_QP_AV)
- copy_ah_attr_from_uverbs(qp->device, &attr->ah_attr,
- &cmd->base.dest);
-
- if (cmd->base.attr_mask & IB_QP_ALT_PATH)
- copy_ah_attr_from_uverbs(qp->device, &attr->alt_ah_attr,
- &cmd->base.alt_dest);
ret = ib_modify_qp_with_udata(qp, attr,
modify_qp_mask(qp->qp_type,
@@ -1994,13 +2008,6 @@ static int ib_uverbs_ex_modify_qp(struct uverbs_attr_bundle *attrs)
if (ret)
return ret;
- /*
- * Last bit is reserved for extending the attr_mask by
- * using another field.
- */
- if (cmd.base.attr_mask & ~(IB_QP_ATTR_STANDARD_BITS | IB_QP_RATE_LIMIT))
- return -EOPNOTSUPP;
-
ret = modify_qp(attrs, &cmd);
if (ret)
return ret;
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 08/13] RDMA/uverbs: factor out the queue pair attribute copy
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:30 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:30 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> modify_qp() validates a set of attributes that came from userspace and
> copies the ones its mask names into their kernel form. Queue pair
> creation needs the same work for a transport that is ready to send as
> soon as the pair exists, so move it into ib_uverbs_copy_qp_attr().
>
> No functional change.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=8
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 09/13] RDMA/core: create a queue pair ready to send
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
Some transports have nothing to negotiate, so a queue pair can be ready
to send the moment it exists and the modify sequence carries no useful
information. Let a reliable unconnected queue pair be created with the
attributes that sequence would have supplied, in the same form the
modify command uses, and apply them as that modify to RTS once the
queue pair exists, so its checks run. A queue pair whose modify fails
is destroyed.
Note that the QP handle field is refused, since the create is what makes
the handle.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_std_types_qp.c | 80 +++++++++++++++++++++++++++
include/uapi/rdma/ib_user_ioctl_cmds.h | 1 +
2 files changed, 81 insertions(+)
diff --git a/drivers/infiniband/core/uverbs_std_types_qp.c b/drivers/infiniband/core/uverbs_std_types_qp.c
index 30fc20fb251f..c6225156fe30 100644
--- a/drivers/infiniband/core/uverbs_std_types_qp.c
+++ b/drivers/infiniband/core/uverbs_std_types_qp.c
@@ -79,6 +79,61 @@ static void set_caps(struct ib_qp_init_attr *attr,
}
}
+/* what a reliable unconnected QP may be created ready to send with */
+#define CREATE_QP_ATTR_RU_MASK (IB_QP_STATE | IB_QP_PKEY_INDEX | \
+ IB_QP_PORT | IB_QP_ACCESS_FLAGS | \
+ IB_QP_PATH_MTU)
+
+/* The attributes the QP would be modified with via modify_qp calls. Used for
+ * implementations that support having QPs be in RTS immediately after QP
+ * creation. These attributes arrive in the same format the modify command
+ * uses so that converter serves both, and they are applied as that modify
+ * to RTS, once the QP exists.
+ */
+static int create_qp_attr_set(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device,
+ struct ib_qp_init_attr *attr,
+ struct ib_qp_attr *qp_attr, u32 *mask)
+{
+ struct ib_uverbs_ex_modify_qp uqpa = {};
+ int ret;
+
+ *mask = 0;
+
+ if (!uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_ATTR))
+ return 0;
+
+ if (attr->qp_type != IB_QPT_RU ||
+ !(device->attrs.device_cap_flags & IB_DEVICE_RU))
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from_or_zero(&uqpa, attrs,
+ UVERBS_ATTR_CREATE_QP_ATTR);
+ if (ret)
+ return ret;
+
+ /* the create is what makes the handle, can't be one specified */
+ if (uqpa.base.qp_handle)
+ return -EINVAL;
+
+ if (uqpa.reserved || uqpa.base.reserved[0] || uqpa.base.reserved[1])
+ return -EINVAL;
+
+ ret = ib_uverbs_copy_qp_attr(attrs, device, qp_attr, &uqpa,
+ uqpa.base.attr_mask);
+ if (ret)
+ return ret;
+
+ if ((uqpa.base.attr_mask & ~CREATE_QP_ATTR_RU_MASK) ||
+ ((uqpa.base.attr_mask & IB_QP_STATE) &&
+ uqpa.base.qp_state != IB_QPS_RTS))
+ return -EINVAL;
+
+ *mask = uqpa.base.attr_mask;
+
+ return 0;
+}
+
static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
struct uverbs_attr_bundle *attrs)
{
@@ -86,6 +141,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_QP_HANDLE),
typeof(*obj), uevent.uobject);
struct ib_qp_init_attr attr = {};
+ struct ib_qp_attr qp_attr = {};
+ u32 qp_attr_mask;
struct ib_uverbs_qp_cap cap = {};
struct ib_rwq_ind_table *rwq_ind_tbl = NULL;
struct ib_qp *qp;
@@ -214,6 +271,11 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
attr.create_flags |= IB_QP_CREATE_SOURCE_QPN;
}
+ ret = create_qp_attr_set(attrs, device, &attr, &qp_attr,
+ &qp_attr_mask);
+ if (ret)
+ return ret;
+
srq = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_SRQ_HANDLE);
if (!IS_ERR(srq)) {
@@ -264,6 +326,20 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
obj->uevent.uobject.object = qp;
uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_CREATE_QP_HANDLE);
+ /* A reliable unconnected QP is ready to send once created. What the
+ * caller passed is applied as the modify to RTS it stands in for, so
+ * the modify's checks run. A failure here destroys the QP.
+ */
+ if (attr.qp_type == IB_QPT_RU) {
+ qp_attr.qp_state = IB_QPS_RTS;
+ /* the driver's data was for create, there is none for this */
+ ret = ib_modify_qp_with_udata(qp, &qp_attr,
+ qp_attr_mask | IB_QP_STATE,
+ uverbs_get_cleared_udata(attrs));
+ if (ret)
+ return ret;
+ }
+
set_caps(&attr, &cap, false);
ret = uverbs_copy_to_struct_or_zero(attrs,
UVERBS_ATTR_CREATE_QP_RESP_CAP, &cap,
@@ -325,6 +401,10 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_CREATE_QP_FLAGS,
enum ib_uverbs_qp_create_flags,
UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_ATTR,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ex_modify_qp,
+ reserved),
+ UA_OPTIONAL),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_SOURCE_QPN,
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL),
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 29abfbd227bc..6a0cd11e7b31 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -178,6 +178,7 @@ enum uverbs_attrs_create_qp_cmd_attr_ids {
UVERBS_ATTR_CREATE_QP_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_RQ_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_SQ_BUF_UMEM,
+ UVERBS_ATTR_CREATE_QP_ATTR,
};
enum uverbs_attrs_destroy_qp_cmd_attr_ids {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 09/13] RDMA/core: create a queue pair ready to send
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
Some transports have nothing to negotiate, so a queue pair can be ready
to send the moment it exists and the modify sequence carries no useful
information. Let a reliable unconnected queue pair be created with the
attributes that sequence would have supplied, in the same form the
modify command uses, and apply them as that modify to RTS once the
queue pair exists, so its checks run. A queue pair whose modify fails
is destroyed.
Note that the QP handle field is refused, since the create is what makes
the handle.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_std_types_qp.c | 80 +++++++++++++++++++++++++++
include/uapi/rdma/ib_user_ioctl_cmds.h | 1 +
2 files changed, 81 insertions(+)
diff --git a/drivers/infiniband/core/uverbs_std_types_qp.c b/drivers/infiniband/core/uverbs_std_types_qp.c
index 30fc20fb251f..c6225156fe30 100644
--- a/drivers/infiniband/core/uverbs_std_types_qp.c
+++ b/drivers/infiniband/core/uverbs_std_types_qp.c
@@ -79,6 +79,61 @@ static void set_caps(struct ib_qp_init_attr *attr,
}
}
+/* what a reliable unconnected QP may be created ready to send with */
+#define CREATE_QP_ATTR_RU_MASK (IB_QP_STATE | IB_QP_PKEY_INDEX | \
+ IB_QP_PORT | IB_QP_ACCESS_FLAGS | \
+ IB_QP_PATH_MTU)
+
+/* The attributes the QP would be modified with via modify_qp calls. Used for
+ * implementations that support having QPs be in RTS immediately after QP
+ * creation. These attributes arrive in the same format the modify command
+ * uses so that converter serves both, and they are applied as that modify
+ * to RTS, once the QP exists.
+ */
+static int create_qp_attr_set(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device,
+ struct ib_qp_init_attr *attr,
+ struct ib_qp_attr *qp_attr, u32 *mask)
+{
+ struct ib_uverbs_ex_modify_qp uqpa = {};
+ int ret;
+
+ *mask = 0;
+
+ if (!uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_ATTR))
+ return 0;
+
+ if (attr->qp_type != IB_QPT_RU ||
+ !(device->attrs.device_cap_flags & IB_DEVICE_RU))
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from_or_zero(&uqpa, attrs,
+ UVERBS_ATTR_CREATE_QP_ATTR);
+ if (ret)
+ return ret;
+
+ /* the create is what makes the handle, can't be one specified */
+ if (uqpa.base.qp_handle)
+ return -EINVAL;
+
+ if (uqpa.reserved || uqpa.base.reserved[0] || uqpa.base.reserved[1])
+ return -EINVAL;
+
+ ret = ib_uverbs_copy_qp_attr(attrs, device, qp_attr, &uqpa,
+ uqpa.base.attr_mask);
+ if (ret)
+ return ret;
+
+ if ((uqpa.base.attr_mask & ~CREATE_QP_ATTR_RU_MASK) ||
+ ((uqpa.base.attr_mask & IB_QP_STATE) &&
+ uqpa.base.qp_state != IB_QPS_RTS))
+ return -EINVAL;
+
+ *mask = uqpa.base.attr_mask;
+
+ return 0;
+}
+
static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
struct uverbs_attr_bundle *attrs)
{
@@ -86,6 +141,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_QP_HANDLE),
typeof(*obj), uevent.uobject);
struct ib_qp_init_attr attr = {};
+ struct ib_qp_attr qp_attr = {};
+ u32 qp_attr_mask;
struct ib_uverbs_qp_cap cap = {};
struct ib_rwq_ind_table *rwq_ind_tbl = NULL;
struct ib_qp *qp;
@@ -214,6 +271,11 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
attr.create_flags |= IB_QP_CREATE_SOURCE_QPN;
}
+ ret = create_qp_attr_set(attrs, device, &attr, &qp_attr,
+ &qp_attr_mask);
+ if (ret)
+ return ret;
+
srq = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_SRQ_HANDLE);
if (!IS_ERR(srq)) {
@@ -264,6 +326,20 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
obj->uevent.uobject.object = qp;
uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_CREATE_QP_HANDLE);
+ /* A reliable unconnected QP is ready to send once created. What the
+ * caller passed is applied as the modify to RTS it stands in for, so
+ * the modify's checks run. A failure here destroys the QP.
+ */
+ if (attr.qp_type == IB_QPT_RU) {
+ qp_attr.qp_state = IB_QPS_RTS;
+ /* the driver's data was for create, there is none for this */
+ ret = ib_modify_qp_with_udata(qp, &qp_attr,
+ qp_attr_mask | IB_QP_STATE,
+ uverbs_get_cleared_udata(attrs));
+ if (ret)
+ return ret;
+ }
+
set_caps(&attr, &cap, false);
ret = uverbs_copy_to_struct_or_zero(attrs,
UVERBS_ATTR_CREATE_QP_RESP_CAP, &cap,
@@ -325,6 +401,10 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_CREATE_QP_FLAGS,
enum ib_uverbs_qp_create_flags,
UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_ATTR,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_ex_modify_qp,
+ reserved),
+ UA_OPTIONAL),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_SOURCE_QPN,
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL),
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 29abfbd227bc..6a0cd11e7b31 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -178,6 +178,7 @@ enum uverbs_attrs_create_qp_cmd_attr_ids {
UVERBS_ATTR_CREATE_QP_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_RQ_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_SQ_BUF_UMEM,
+ UVERBS_ATTR_CREATE_QP_ATTR,
};
enum uverbs_attrs_destroy_qp_cmd_attr_ids {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 09/13] RDMA/core: create a queue pair ready to send
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:39 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:39 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> Some transports have nothing to negotiate, so a queue pair can be ready
> to send the moment it exists and the modify sequence carries no useful
> information. Let a reliable unconnected queue pair be created with the
> attributes that sequence would have supplied, in the same form the
> modify command uses, and apply them as that modify to RTS once the
> queue pair exists, so its checks run. A queue pair whose modify fails
> is destroyed.
>
> Note that the QP handle field is refused, since the create is what makes
> the handle.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=9
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 10/13] RDMA/core: queue pair attributes for unconnected transports
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
A queue pair on an unconnected transport is created with more than the
connected attributes. A job key authorizing it to act within a job, the
initiator identity it sends under, and the ordering guarantees the caller
wants it to keep.
The QP holds a reference to its job key for as long as it lives, so the
job cannot go away underneath it.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_std_types_qp.c | 150 ++++++++++++++++++++++++++
drivers/infiniband/core/verbs.c | 5 +
include/rdma/ib_verbs.h | 14 +++
include/uapi/rdma/ib_user_ioctl_cmds.h | 3 +
4 files changed, 172 insertions(+)
diff --git a/drivers/infiniband/core/uverbs_std_types_qp.c b/drivers/infiniband/core/uverbs_std_types_qp.c
index c6225156fe30..47ca994f9ac9 100644
--- a/drivers/infiniband/core/uverbs_std_types_qp.c
+++ b/drivers/infiniband/core/uverbs_std_types_qp.c
@@ -79,6 +79,139 @@ static void set_caps(struct ib_qp_init_attr *attr,
}
}
+/* the semantics fields, ordering bits and usage flags uverbs knows of */
+#define QP_SEMANTICS_MASK_ALL ((IB_UVERBS_QP_SEMANTICS_MASK_USAGE << 1) - 1)
+#define QP_ORDER_ALL ((IB_UVERBS_QP_ORDER_WAW << 1) - 1)
+#define QP_USAGE_ALL IB_UVERBS_QP_USAGE_IMM_DATA_RQ
+
+/* A QP may relax what its type guarantees on this device, but not ask for
+ * more than the device advertises for it.
+ */
+static int create_qp_semantics_check(struct ib_device *device,
+ const struct ib_qp_semantics *req)
+{
+ struct ib_qp_semantics adv = {};
+ u32 m = req->comp_mask;
+ int ret;
+
+ if (!device->ops.query_qp_semantics)
+ return -EOPNOTSUPP;
+
+ ret = device->ops.query_qp_semantics(device, IB_QPT_RU, &adv);
+ if (ret)
+ return ret;
+
+ if (m & ~adv.comp_mask)
+ return -EINVAL;
+
+ if (((m & IB_UVERBS_QP_SEMANTICS_MASK_MSG_ORDER) &&
+ (req->msg_order & ~adv.msg_order)) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_RAW) &&
+ req->max_rdma_raw_size > adv.max_rdma_raw_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_WAR) &&
+ req->max_rdma_war_size > adv.max_rdma_war_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_WAW) &&
+ req->max_rdma_waw_size > adv.max_rdma_waw_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_PDU) &&
+ req->max_pdu > adv.max_pdu) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_IMM) &&
+ req->imm_data_size > adv.imm_data_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_USAGE) &&
+ (req->usage_flags & ~adv.usage_flags)))
+ return -EINVAL;
+
+ return 0;
+}
+
+/* Process the optional creation attributes. A job to act within, a source
+ * identity, a set of ordering guarantees to ask for. They and the checks on
+ * a requested QP number belong to a reliable unconnected QP.
+ */
+static int create_qp_opt_attrs(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device, struct ib_pd *pd,
+ struct ib_qp_init_attr *attr,
+ struct ib_qp_semantics *sem)
+{
+ struct ib_uverbs_qp_semantics usem = {};
+ struct ib_jkey *jkey;
+ bool has_semantics;
+ int ret;
+
+ has_semantics = uverbs_attr_is_valid(attrs,
+ UVERBS_ATTR_CREATE_QP_SEMANTICS);
+
+ if (attr->qp_type != IB_QPT_RU ||
+ !(device->attrs.device_cap_flags & IB_DEVICE_RU)) {
+ if (has_semantics ||
+ uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_SRC_ID) ||
+ uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_JOB_KEY))
+ return -EOPNOTSUPP;
+
+ return 0;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_SRC_ID)) {
+ ret = uverbs_copy_from(&attr->src_id, attrs,
+ UVERBS_ATTR_CREATE_QP_SRC_ID);
+ if (ret)
+ return ret;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_JOB_KEY)) {
+ jkey = uverbs_attr_get_obj(attrs,
+ UVERBS_ATTR_CREATE_QP_JOB_KEY);
+ if (IS_ERR(jkey))
+ return PTR_ERR(jkey);
+
+ /* a key acts within the PD it was made on */
+ if (jkey->pd != pd)
+ return -EINVAL;
+
+ attr->jkey = jkey;
+ }
+
+ if (attr->create_flags & IB_QP_CREATE_SOURCE_QPN) {
+ if (attr->source_qpn & IB_UVERBS_QPN_UET_RESERVED_MASK)
+ return -EINVAL;
+
+ /* a relative QP is named within a job, so it needs a key */
+ if (!(attr->source_qpn &
+ IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT) &&
+ !attr->jkey)
+ return -EINVAL;
+ }
+
+ if (has_semantics) {
+ ret = uverbs_copy_from_or_zero(&usem, attrs,
+ UVERBS_ATTR_CREATE_QP_SEMANTICS);
+ if (ret)
+ return ret;
+
+ if ((usem.comp_mask & ~QP_SEMANTICS_MASK_ALL) ||
+ (usem.msg_order & ~QP_ORDER_ALL) ||
+ (usem.usage_flags & ~QP_USAGE_ALL) ||
+ memchr_inv(usem.reserved, 0, sizeof(usem.reserved)))
+ return -EINVAL;
+
+ sem->comp_mask = usem.comp_mask;
+ sem->msg_order = usem.msg_order;
+ sem->max_rdma_raw_size = usem.max_rdma_raw_size;
+ sem->max_rdma_war_size = usem.max_rdma_war_size;
+ sem->max_rdma_waw_size = usem.max_rdma_waw_size;
+ sem->max_pdu = usem.max_pdu;
+ sem->usage_flags = usem.usage_flags;
+ sem->imm_data_size = usem.imm_data_size;
+
+ ret = create_qp_semantics_check(device, sem);
+ if (ret)
+ return ret;
+
+ attr->qp_semantics = sem;
+ }
+
+ return 0;
+}
+
/* what a reliable unconnected QP may be created ready to send with */
#define CREATE_QP_ATTR_RU_MASK (IB_QP_STATE | IB_QP_PKEY_INDEX | \
IB_QP_PORT | IB_QP_ACCESS_FLAGS | \
@@ -143,6 +276,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
struct ib_qp_init_attr attr = {};
struct ib_qp_attr qp_attr = {};
u32 qp_attr_mask;
+ struct ib_qp_semantics qp_semantics = {};
struct ib_uverbs_qp_cap cap = {};
struct ib_rwq_ind_table *rwq_ind_tbl = NULL;
struct ib_qp *qp;
@@ -197,6 +331,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
case IB_UVERBS_QPT_UC:
case IB_UVERBS_QPT_UD:
case IB_UVERBS_QPT_XRC_INI:
+ case IB_UVERBS_QPT_RU:
case IB_UVERBS_QPT_DRIVER:
if (uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_XRCD_HANDLE) ||
@@ -271,6 +406,10 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
attr.create_flags |= IB_QP_CREATE_SOURCE_QPN;
}
+ ret = create_qp_opt_attrs(attrs, device, pd, &attr, &qp_semantics);
+ if (ret)
+ return ret;
+
ret = create_qp_attr_set(attrs, device, &attr, &qp_attr,
&qp_attr_mask);
if (ret)
@@ -401,6 +540,17 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_CREATE_QP_FLAGS,
enum ib_uverbs_qp_create_flags,
UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_SRC_ID,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_SEMANTICS,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_qp_semantics,
+ imm_data_size),
+ UA_OPTIONAL),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_JOB_KEY,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_READ,
+ UA_OPTIONAL),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_ATTR,
UVERBS_ATTR_STRUCT(struct ib_uverbs_ex_modify_qp,
reserved),
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 0a5340642abb..3446aaa16ef4 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -1343,6 +1343,7 @@ static struct ib_qp *create_qp(struct ib_device *dev, struct ib_pd *pd,
qp->qp_type = attr->qp_type;
qp->rwq_ind_tbl = attr->rwq_ind_tbl;
qp->srq = attr->srq;
+ qp->jkey = attr->jkey;
qp->event_handler = __ib_qp_event_handler;
qp->registered_event_handler = attr->event_handler;
qp->port = attr->port_num;
@@ -1438,11 +1439,15 @@ void ib_qp_usecnt_inc(struct ib_qp *qp)
atomic_inc(&qp->srq->usecnt);
if (qp->rwq_ind_tbl)
atomic_inc(&qp->rwq_ind_tbl->usecnt);
+ if (qp->jkey)
+ atomic_inc(&qp->jkey->usecnt);
}
EXPORT_SYMBOL(ib_qp_usecnt_inc);
void ib_qp_usecnt_dec(struct ib_qp *qp)
{
+ if (qp->jkey)
+ atomic_dec(&qp->jkey->usecnt);
if (qp->rwq_ind_tbl)
atomic_dec(&qp->rwq_ind_tbl->usecnt);
if (qp->srq)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 6aa8e7120e71..3e43d0b8b495 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1243,6 +1243,9 @@ enum ib_qp_create_flags {
* callback to destroy the passed in QP.
*/
+struct ib_jkey;
+struct ib_qp_semantics;
+
struct ib_qp_init_attr {
/* This callback occurs in workqueue context */
void (*event_handler)(struct ib_event *, void *);
@@ -1263,6 +1266,15 @@ struct ib_qp_init_attr {
u32 port_num;
struct ib_rwq_ind_table *rwq_ind_tbl;
u32 source_qpn;
+
+ /* The initiator ID this QP sends with, the caller's rank in its job.
+ * UET lets the caller choose it; the JobID is what is trusted.
+ */
+ u32 src_id;
+ /* the job this QP belongs to */
+ struct ib_jkey *jkey;
+ /* what the caller asked the QP to guarantee, NULL if it didn't ask */
+ struct ib_qp_semantics *qp_semantics;
};
struct ib_qp_open_attr {
@@ -2009,6 +2021,8 @@ struct ib_qp {
enum ib_qp_type qp_type;
struct ib_rwq_ind_table *rwq_ind_tbl;
struct ib_qp_security *qp_sec;
+ /* the job key this pair was created within, NULL if none */
+ struct ib_jkey *jkey;
u32 port;
bool integrity_en;
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 6a0cd11e7b31..e1ffe3a662a0 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -178,6 +178,9 @@ enum uverbs_attrs_create_qp_cmd_attr_ids {
UVERBS_ATTR_CREATE_QP_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_RQ_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_SQ_BUF_UMEM,
+ UVERBS_ATTR_CREATE_QP_SRC_ID,
+ UVERBS_ATTR_CREATE_QP_SEMANTICS,
+ UVERBS_ATTR_CREATE_QP_JOB_KEY,
UVERBS_ATTR_CREATE_QP_ATTR,
};
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 10/13] RDMA/core: queue pair attributes for unconnected transports
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
A queue pair on an unconnected transport is created with more than the
connected attributes. A job key authorizing it to act within a job, the
initiator identity it sends under, and the ordering guarantees the caller
wants it to keep.
The QP holds a reference to its job key for as long as it lives, so the
job cannot go away underneath it.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_std_types_qp.c | 150 ++++++++++++++++++++++++++
drivers/infiniband/core/verbs.c | 5 +
include/rdma/ib_verbs.h | 14 +++
include/uapi/rdma/ib_user_ioctl_cmds.h | 3 +
4 files changed, 172 insertions(+)
diff --git a/drivers/infiniband/core/uverbs_std_types_qp.c b/drivers/infiniband/core/uverbs_std_types_qp.c
index c6225156fe30..47ca994f9ac9 100644
--- a/drivers/infiniband/core/uverbs_std_types_qp.c
+++ b/drivers/infiniband/core/uverbs_std_types_qp.c
@@ -79,6 +79,139 @@ static void set_caps(struct ib_qp_init_attr *attr,
}
}
+/* the semantics fields, ordering bits and usage flags uverbs knows of */
+#define QP_SEMANTICS_MASK_ALL ((IB_UVERBS_QP_SEMANTICS_MASK_USAGE << 1) - 1)
+#define QP_ORDER_ALL ((IB_UVERBS_QP_ORDER_WAW << 1) - 1)
+#define QP_USAGE_ALL IB_UVERBS_QP_USAGE_IMM_DATA_RQ
+
+/* A QP may relax what its type guarantees on this device, but not ask for
+ * more than the device advertises for it.
+ */
+static int create_qp_semantics_check(struct ib_device *device,
+ const struct ib_qp_semantics *req)
+{
+ struct ib_qp_semantics adv = {};
+ u32 m = req->comp_mask;
+ int ret;
+
+ if (!device->ops.query_qp_semantics)
+ return -EOPNOTSUPP;
+
+ ret = device->ops.query_qp_semantics(device, IB_QPT_RU, &adv);
+ if (ret)
+ return ret;
+
+ if (m & ~adv.comp_mask)
+ return -EINVAL;
+
+ if (((m & IB_UVERBS_QP_SEMANTICS_MASK_MSG_ORDER) &&
+ (req->msg_order & ~adv.msg_order)) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_RAW) &&
+ req->max_rdma_raw_size > adv.max_rdma_raw_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_WAR) &&
+ req->max_rdma_war_size > adv.max_rdma_war_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_WAW) &&
+ req->max_rdma_waw_size > adv.max_rdma_waw_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_PDU) &&
+ req->max_pdu > adv.max_pdu) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_IMM) &&
+ req->imm_data_size > adv.imm_data_size) ||
+ ((m & IB_UVERBS_QP_SEMANTICS_MASK_USAGE) &&
+ (req->usage_flags & ~adv.usage_flags)))
+ return -EINVAL;
+
+ return 0;
+}
+
+/* Process the optional creation attributes. A job to act within, a source
+ * identity, a set of ordering guarantees to ask for. They and the checks on
+ * a requested QP number belong to a reliable unconnected QP.
+ */
+static int create_qp_opt_attrs(struct uverbs_attr_bundle *attrs,
+ struct ib_device *device, struct ib_pd *pd,
+ struct ib_qp_init_attr *attr,
+ struct ib_qp_semantics *sem)
+{
+ struct ib_uverbs_qp_semantics usem = {};
+ struct ib_jkey *jkey;
+ bool has_semantics;
+ int ret;
+
+ has_semantics = uverbs_attr_is_valid(attrs,
+ UVERBS_ATTR_CREATE_QP_SEMANTICS);
+
+ if (attr->qp_type != IB_QPT_RU ||
+ !(device->attrs.device_cap_flags & IB_DEVICE_RU)) {
+ if (has_semantics ||
+ uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_SRC_ID) ||
+ uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_JOB_KEY))
+ return -EOPNOTSUPP;
+
+ return 0;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_SRC_ID)) {
+ ret = uverbs_copy_from(&attr->src_id, attrs,
+ UVERBS_ATTR_CREATE_QP_SRC_ID);
+ if (ret)
+ return ret;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_QP_JOB_KEY)) {
+ jkey = uverbs_attr_get_obj(attrs,
+ UVERBS_ATTR_CREATE_QP_JOB_KEY);
+ if (IS_ERR(jkey))
+ return PTR_ERR(jkey);
+
+ /* a key acts within the PD it was made on */
+ if (jkey->pd != pd)
+ return -EINVAL;
+
+ attr->jkey = jkey;
+ }
+
+ if (attr->create_flags & IB_QP_CREATE_SOURCE_QPN) {
+ if (attr->source_qpn & IB_UVERBS_QPN_UET_RESERVED_MASK)
+ return -EINVAL;
+
+ /* a relative QP is named within a job, so it needs a key */
+ if (!(attr->source_qpn &
+ IB_UVERBS_QPN_UET_ABSOLUTE_ADDR_BIT) &&
+ !attr->jkey)
+ return -EINVAL;
+ }
+
+ if (has_semantics) {
+ ret = uverbs_copy_from_or_zero(&usem, attrs,
+ UVERBS_ATTR_CREATE_QP_SEMANTICS);
+ if (ret)
+ return ret;
+
+ if ((usem.comp_mask & ~QP_SEMANTICS_MASK_ALL) ||
+ (usem.msg_order & ~QP_ORDER_ALL) ||
+ (usem.usage_flags & ~QP_USAGE_ALL) ||
+ memchr_inv(usem.reserved, 0, sizeof(usem.reserved)))
+ return -EINVAL;
+
+ sem->comp_mask = usem.comp_mask;
+ sem->msg_order = usem.msg_order;
+ sem->max_rdma_raw_size = usem.max_rdma_raw_size;
+ sem->max_rdma_war_size = usem.max_rdma_war_size;
+ sem->max_rdma_waw_size = usem.max_rdma_waw_size;
+ sem->max_pdu = usem.max_pdu;
+ sem->usage_flags = usem.usage_flags;
+ sem->imm_data_size = usem.imm_data_size;
+
+ ret = create_qp_semantics_check(device, sem);
+ if (ret)
+ return ret;
+
+ attr->qp_semantics = sem;
+ }
+
+ return 0;
+}
+
/* what a reliable unconnected QP may be created ready to send with */
#define CREATE_QP_ATTR_RU_MASK (IB_QP_STATE | IB_QP_PKEY_INDEX | \
IB_QP_PORT | IB_QP_ACCESS_FLAGS | \
@@ -143,6 +276,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
struct ib_qp_init_attr attr = {};
struct ib_qp_attr qp_attr = {};
u32 qp_attr_mask;
+ struct ib_qp_semantics qp_semantics = {};
struct ib_uverbs_qp_cap cap = {};
struct ib_rwq_ind_table *rwq_ind_tbl = NULL;
struct ib_qp *qp;
@@ -197,6 +331,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
case IB_UVERBS_QPT_UC:
case IB_UVERBS_QPT_UD:
case IB_UVERBS_QPT_XRC_INI:
+ case IB_UVERBS_QPT_RU:
case IB_UVERBS_QPT_DRIVER:
if (uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_XRCD_HANDLE) ||
@@ -271,6 +406,10 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
attr.create_flags |= IB_QP_CREATE_SOURCE_QPN;
}
+ ret = create_qp_opt_attrs(attrs, device, pd, &attr, &qp_semantics);
+ if (ret)
+ return ret;
+
ret = create_qp_attr_set(attrs, device, &attr, &qp_attr,
&qp_attr_mask);
if (ret)
@@ -401,6 +540,17 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_CREATE_QP_FLAGS,
enum ib_uverbs_qp_create_flags,
UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_SRC_ID,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_SEMANTICS,
+ UVERBS_ATTR_STRUCT(struct ib_uverbs_qp_semantics,
+ imm_data_size),
+ UA_OPTIONAL),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_JOB_KEY,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_READ,
+ UA_OPTIONAL),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_ATTR,
UVERBS_ATTR_STRUCT(struct ib_uverbs_ex_modify_qp,
reserved),
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 0a5340642abb..3446aaa16ef4 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -1343,6 +1343,7 @@ static struct ib_qp *create_qp(struct ib_device *dev, struct ib_pd *pd,
qp->qp_type = attr->qp_type;
qp->rwq_ind_tbl = attr->rwq_ind_tbl;
qp->srq = attr->srq;
+ qp->jkey = attr->jkey;
qp->event_handler = __ib_qp_event_handler;
qp->registered_event_handler = attr->event_handler;
qp->port = attr->port_num;
@@ -1438,11 +1439,15 @@ void ib_qp_usecnt_inc(struct ib_qp *qp)
atomic_inc(&qp->srq->usecnt);
if (qp->rwq_ind_tbl)
atomic_inc(&qp->rwq_ind_tbl->usecnt);
+ if (qp->jkey)
+ atomic_inc(&qp->jkey->usecnt);
}
EXPORT_SYMBOL(ib_qp_usecnt_inc);
void ib_qp_usecnt_dec(struct ib_qp *qp)
{
+ if (qp->jkey)
+ atomic_dec(&qp->jkey->usecnt);
if (qp->rwq_ind_tbl)
atomic_dec(&qp->rwq_ind_tbl->usecnt);
if (qp->srq)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 6aa8e7120e71..3e43d0b8b495 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1243,6 +1243,9 @@ enum ib_qp_create_flags {
* callback to destroy the passed in QP.
*/
+struct ib_jkey;
+struct ib_qp_semantics;
+
struct ib_qp_init_attr {
/* This callback occurs in workqueue context */
void (*event_handler)(struct ib_event *, void *);
@@ -1263,6 +1266,15 @@ struct ib_qp_init_attr {
u32 port_num;
struct ib_rwq_ind_table *rwq_ind_tbl;
u32 source_qpn;
+
+ /* The initiator ID this QP sends with, the caller's rank in its job.
+ * UET lets the caller choose it; the JobID is what is trusted.
+ */
+ u32 src_id;
+ /* the job this QP belongs to */
+ struct ib_jkey *jkey;
+ /* what the caller asked the QP to guarantee, NULL if it didn't ask */
+ struct ib_qp_semantics *qp_semantics;
};
struct ib_qp_open_attr {
@@ -2009,6 +2021,8 @@ struct ib_qp {
enum ib_qp_type qp_type;
struct ib_rwq_ind_table *rwq_ind_tbl;
struct ib_qp_security *qp_sec;
+ /* the job key this pair was created within, NULL if none */
+ struct ib_jkey *jkey;
u32 port;
bool integrity_en;
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 6a0cd11e7b31..e1ffe3a662a0 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -178,6 +178,9 @@ enum uverbs_attrs_create_qp_cmd_attr_ids {
UVERBS_ATTR_CREATE_QP_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_RQ_BUF_UMEM,
UVERBS_ATTR_CREATE_QP_SQ_BUF_UMEM,
+ UVERBS_ATTR_CREATE_QP_SRC_ID,
+ UVERBS_ATTR_CREATE_QP_SEMANTICS,
+ UVERBS_ATTR_CREATE_QP_JOB_KEY,
UVERBS_ATTR_CREATE_QP_ATTR,
};
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 10/13] RDMA/core: queue pair attributes for unconnected transports
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:41 ` sashiko-bot
2026-10-15 16:21 ` Eric Davis
-1 siblings, 1 reply; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:41 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] RDMA: missing IB_QPT_RU state machine validation in qp_state_table
--
commit e32578d06dac38500b6cb1196755de6b03c64308
Author: Eric Davis <eric.davis@broadcom.com>
Subject: RDMA/core: queue pair attributes for unconnected transports
This commit adds creation attributes for queue pairs on unconnected transports.
It introduces job keys to authorize actions, initiator identities, and
semantics checks for ordering guarantees.
> diff --git a/drivers/infiniband/core/uverbs_std_types_qp.c b/drivers/infiniband/core/uverbs_std_types_qp.c
> index c6225156fe30e..47ca994f9ac96 100644
> --- a/drivers/infiniband/core/uverbs_std_types_qp.c
> +++ b/drivers/infiniband/core/uverbs_std_types_qp.c
[ ... ]
> @@ -197,6 +331,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
> case IB_UVERBS_QPT_UC:
> case IB_UVERBS_QPT_UD:
> case IB_UVERBS_QPT_XRC_INI:
> + case IB_UVERBS_QPT_RU:
[Severity: Medium]
Will this newly added transport fail state machine validation during creation?
When UVERBS_METHOD_QP_CREATE() creates an IB_QPT_RU QP, the core immediately
forces a transition from IB_QPS_RESET to IB_QPS_RTS via
ib_modify_qp_with_udata():
drivers/infiniband/core/uverbs_std_types_qp.c:
UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)() {
...
if (attr.qp_type == IB_QPT_RU) {
qp_attr.qp_state = IB_QPS_RTS;
ret = ib_modify_qp_with_udata(qp, &qp_attr,
qp_attr_mask | IB_QP_STATE,
uverbs_get_cleared_udata(attrs));
...
}
However, IB_QPT_RU appears to be missing from the qp_state_table in
drivers/infiniband/core/verbs.c.
When a driver invokes ib_modify_qp_is_ok(), the missing table entries mean
the req_param and opt_param default to 0, and the transition is not marked
as valid. Doesn't this cause ib_modify_qp_is_ok() to unconditionally return
false and reject the QP modification for any driver attempting to support
this new transport?
> case IB_UVERBS_QPT_DRIVER:
> if (uverbs_attr_is_valid(attrs,
> UVERBS_ATTR_CREATE_QP_XRCD_HANDLE) ||
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=10
^ permalink raw reply [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 10/13] RDMA/core: queue pair attributes for unconnected transports
2026-10-14 22:41 ` sashiko-bot
@ 2026-10-15 16:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-15 16:21 UTC (permalink / raw)
To: sashiko-reviews; +Cc: linux-rdma, Jason Gunthorpe, Leon Romanovsky
[-- Attachment #1: Type: text/plain, Size: 689 bytes --]
On Wed, Oct 14, 2026 at 3:41 PM <sashiko-bot@kernel.org> wrote:
> However, IB_QPT_RU appears to be missing from the qp_state_table in
> drivers/infiniband/core/verbs.c.
Same point as on patch 4, no change is planned. ib_modify_qp_with_udata()
does not consult the qp_state_table. Only a driver that calls
ib_modify_qp_is_ok() itself does and an RU driver validates RESET->RTS
for RU in its own modify_qp. Core limits what that modify may carry
(CREATE_QP_ATTR_RU_MASK, state RTS only) and destroys the QP if the
driver refuses it.
>
> --
> Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=10
Thanks,
- eric
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5461 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 11/13] RDMA/core: extended memory registration
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
Memory registration for an unconnected transport may take more than an
address and an access mask. A job to bind the region to, a key the caller
chose, or a region carved out of another. None of it fits the write()
command, so add a device method that takes the wider set of attributes
and reach it when the ioctl carries any of these new fields.
A region bound to a job holds a reference to its job key until it is
deregistered.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/device.c | 1 +
drivers/infiniband/core/uverbs_cmd.c | 7 +-
drivers/infiniband/core/uverbs_std_types_mr.c | 182 +++++++++++++++++++++++++-
drivers/infiniband/core/verbs.c | 9 ++
include/rdma/ib_verbs.h | 43 ++++++
include/uapi/rdma/ib_user_ioctl_cmds.h | 5 +
include/uapi/rdma/ib_user_ioctl_verbs.h | 10 ++
7 files changed, 252 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index cf2a59afcf76..fd45decf9c4f 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -3028,6 +3028,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, reg_dm_mr);
SET_DEVICE_OP(dev_ops, reg_user_mr);
SET_DEVICE_OP(dev_ops, reg_user_mr_dmabuf);
+ SET_DEVICE_OP(dev_ops, reg_user_mr_ex);
SET_DEVICE_OP(dev_ops, req_notify_cq);
SET_DEVICE_OP(dev_ops, rereg_user_mr);
SET_DEVICE_OP(dev_ops, resize_user_cq);
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 88212465d8a0..10630abc9ff1 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -731,6 +731,7 @@ static int ib_uverbs_reg_mr(struct uverbs_attr_bundle *attrs)
mr->type = IB_MR_TYPE_USER;
mr->dm = NULL;
mr->sig_attrs = NULL;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
mr->iova = cmd.hca_va;
@@ -790,7 +791,10 @@ static int ib_uverbs_rereg_mr(struct uverbs_attr_bundle *attrs)
mr = uobj->object;
- if (mr->dm) {
+ /* A region bound to a job or to another region is re-created rather
+ * than re-registered, so those bindings can't be lost or bypassed.
+ */
+ if (mr->dm || mr->jkey || mr->parent || atomic_read(&mr->usecnt)) {
ret = -EINVAL;
goto put_uobjs;
}
@@ -838,6 +842,7 @@ static int ib_uverbs_rereg_mr(struct uverbs_attr_bundle *attrs)
new_mr->device = new_pd->device;
new_mr->pd = new_pd;
new_mr->type = IB_MR_TYPE_USER;
+ ib_mr_init_core_refs(new_mr);
new_mr->uobject = uobj;
atomic_inc(&new_pd->usecnt);
new_uobj->object = new_mr;
diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index 0c72f801e0d3..035efba54dcb 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -40,8 +40,13 @@ static int uverbs_free_mr(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
- return ib_dereg_mr_user((struct ib_mr *)uobject->object,
- &attrs->driver_udata);
+ struct ib_mr *mr = uobject->object;
+
+ /* regions derived from this one still use it */
+ if (atomic_read(&mr->usecnt))
+ return -EBUSY;
+
+ return ib_dereg_mr_user(mr, &attrs->driver_udata);
}
static int UVERBS_HANDLER(UVERBS_METHOD_ADVISE_MR)(
@@ -132,6 +137,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_DM_MR_REG)(
mr->pd = pd;
mr->type = IB_MR_TYPE_DM;
mr->dm = dm;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
atomic_inc(&dm->usecnt);
@@ -246,6 +252,9 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_DMABUF_MR)(
mr->device = pd->device;
mr->pd = pd;
mr->type = IB_MR_TYPE_USER;
+ mr->iova = iova;
+ mr->length = length;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
@@ -266,6 +275,116 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_DMABUF_MR)(
return ret;
}
+/* Attributes that an extended MR registration uses. The 64b wide keys are
+ * not in the list as they are not a property of how the MR was registered
+ * and instead a device property which could be used by an ordinary register.
+ */
+static const u16 reg_mr_ext_attrs[] = {
+ UVERBS_ATTR_REG_MR_FLAGS,
+ UVERBS_ATTR_REG_MR_JKEY_HANDLE,
+ UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
+ UVERBS_ATTR_REG_MR_DERIVE_CNT,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64,
+};
+
+static bool reg_mr_has_ext_attrs(struct uverbs_attr_bundle *attrs)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(reg_mr_ext_attrs); i++)
+ if (uverbs_attr_is_valid(attrs, reg_mr_ext_attrs[i]))
+ return true;
+
+ return false;
+}
+
+static int reg_mr_ex_attr(struct uverbs_attr_bundle *attrs, struct ib_pd *pd,
+ struct ib_mr_ex_attr *ex)
+{
+ u64 caps = pd->device->attrs.device_cap_flags;
+ int ret;
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_FLAGS)) {
+ ret = uverbs_get_flags32(&ex->flags, attrs,
+ UVERBS_ATTR_REG_MR_FLAGS,
+ (IB_UVERBS_REG_MR_USER_RKEY |
+ IB_UVERBS_REG_MR_IDEMPOTENT_SAFE));
+ if (ret)
+ return ret;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_JKEY_HANDLE)) {
+ struct ib_jkey *jkey = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_REG_MR_JKEY_HANDLE);
+
+ if (IS_ERR(jkey))
+ return PTR_ERR(jkey);
+
+ /* a key acts within the PD it was made on */
+ if (jkey->pd != pd)
+ return -EINVAL;
+
+ if (!(caps & (IB_DEVICE_MR_JOB_RESTRICTED |
+ IB_DEVICE_MR_QP_JOB_RESTRICTED)))
+ return -EOPNOTSUPP;
+
+ ex->jkey = jkey;
+ }
+
+ /* a region carved out of another, execute the range check */
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE)) {
+ struct ib_mr *parent = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE);
+
+ if (IS_ERR(parent))
+ return PTR_ERR(parent);
+
+ if (parent->pd != pd || parent->type != IB_MR_TYPE_USER)
+ return -EINVAL;
+
+ if ((ex->iova < parent->iova) ||
+ (ex->length > parent->length) ||
+ ((ex->iova - parent->iova) >
+ (parent->length - ex->length)))
+ return -EINVAL;
+
+ ex->parent_mr = parent;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_DERIVE_CNT)) {
+ ret = uverbs_copy_from(&ex->derive_cnt, attrs,
+ UVERBS_ATTR_REG_MR_DERIVE_CNT);
+ if (ret)
+ return ret;
+ }
+
+ if (ex->flags & IB_UVERBS_REG_MR_USER_RKEY) {
+ if (!(caps & IB_DEVICE_USER_RKEY))
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from(&ex->req_rkey64, attrs,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64);
+ if (ret)
+ return ret;
+
+ if (ex->req_rkey64 > U32_MAX && !(caps & IB_DEVICE_KEY64))
+ return -EOPNOTSUPP;
+ } else if (uverbs_attr_is_valid(attrs,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64)) {
+ /* a key with no flag to say it is the caller's */
+ return -EINVAL;
+ }
+
+ /* Idempotent-safe is a property of the key so it can only be asked
+ * for on a key the caller chose.
+ */
+ if ((ex->flags & IB_UVERBS_REG_MR_IDEMPOTENT_SAFE) &&
+ !(ex->flags & IB_UVERBS_REG_MR_USER_RKEY))
+ return -EINVAL;
+
+ return 0;
+}
+
static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
struct uverbs_attr_bundle *attrs)
{
@@ -276,10 +395,12 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
u32 valid_access_flags = IB_ACCESS_SUPPORTED;
u64 length, iova, fd_offset = 0, addr = 0;
struct ib_device *ib_dev = pd->device;
+ struct ib_mr_ex_attr ex = { .fd = -1 };
struct ib_dmah *dmah = NULL;
bool has_fd_offset = false;
bool has_addr = false;
bool has_fd = false;
+ bool has_ext;
u32 access_flags;
struct ib_mr *mr;
int fd;
@@ -293,6 +414,10 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
if (ret)
return ret;
+ has_ext = reg_mr_has_ext_attrs(attrs);
+ if (has_ext && !ib_dev->ops.reg_user_mr_ex)
+ return -EOPNOTSUPP;
+
if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_ADDR)) {
ret = uverbs_copy_from(&addr, attrs,
UVERBS_ATTR_REG_MR_ADDR);
@@ -318,7 +443,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
}
if (has_fd) {
- if (!ib_dev->ops.reg_user_mr_dmabuf)
+ /* an extended registration hands the fd to reg_user_mr_ex */
+ if (!has_ext && !ib_dev->ops.reg_user_mr_dmabuf)
return -EOPNOTSUPP;
/* FD requires offset and can't come with addr */
@@ -358,7 +484,24 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
if (ret)
return ret;
- if (has_fd)
+ if (has_ext) {
+ ex.addr = addr;
+ ex.length = length;
+ ex.iova = iova;
+ ex.access_flags = access_flags;
+ ex.dmah = dmah;
+ if (has_fd) {
+ ex.fd = fd;
+ ex.fd_offset = fd_offset;
+ }
+
+ ret = reg_mr_ex_attr(attrs, pd, &ex);
+ if (ret)
+ return ret;
+
+ mr = pd->device->ops.reg_user_mr_ex(pd, &ex,
+ &attrs->driver_udata);
+ } else if (has_fd)
mr = pd->device->ops.reg_user_mr_dmabuf(pd, fd_offset, length,
iova, fd, access_flags,
dmah, attrs);
@@ -373,12 +516,26 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
mr->device = pd->device;
mr->pd = pd;
mr->type = IB_MR_TYPE_USER;
+ mr->iova = iova;
+ mr->length = length;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
if (dmah) {
mr->dmah = dmah;
atomic_inc(&dmah->usecnt);
}
+
+ /* hold a reference to the job key, deregistration will drop it */
+ mr->jkey = ex.jkey;
+ if (ex.jkey)
+ atomic_inc(&ex.jkey->usecnt);
+
+ /* and to the region this one was derived from */
+ mr->parent = ex.parent_mr;
+ if (ex.parent_mr)
+ atomic_inc(&ex.parent_mr->usecnt);
+
rdma_restrack_new(&mr->res, RDMA_RESTRACK_MR);
rdma_restrack_set_name(&mr->res, NULL);
rdma_restrack_add(&mr->res);
@@ -529,6 +686,23 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_RKEY,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
+ UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_REG_MR_FLAGS,
+ enum ib_uverbs_reg_mr_flags,
+ UA_OPTIONAL),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_REG_MR_JKEY_HANDLE,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_READ,
+ UA_OPTIONAL),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
+ UVERBS_OBJECT_MR,
+ UVERBS_ACCESS_READ,
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_DERIVE_CNT,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_REQ_RKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
UVERBS_ATTR_UHW());
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 3446aaa16ef4..87dc48ecb77a 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -2358,6 +2358,7 @@ struct ib_mr *ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mr->type = IB_MR_TYPE_USER;
mr->pd = pd;
mr->dm = NULL;
+ ib_mr_init_core_refs(mr);
atomic_inc(&pd->usecnt);
mr->iova = virt_addr;
mr->length = length;
@@ -2389,6 +2390,8 @@ int ib_dereg_mr_user(struct ib_mr *mr, struct ib_udata *udata)
struct ib_pd *pd = mr->pd;
struct ib_dm *dm = mr->dm;
struct ib_dmah *dmah = mr->dmah;
+ struct ib_jkey *jkey = mr->jkey;
+ struct ib_mr *parent = mr->parent;
struct ib_sig_attrs *sig_attrs = mr->sig_attrs;
int ret;
@@ -2401,6 +2404,10 @@ int ib_dereg_mr_user(struct ib_mr *mr, struct ib_udata *udata)
atomic_dec(&dm->usecnt);
if (dmah)
atomic_dec(&dmah->usecnt);
+ if (jkey)
+ atomic_dec(&jkey->usecnt);
+ if (parent)
+ atomic_dec(&parent->usecnt);
kfree(sig_attrs);
}
@@ -2448,6 +2455,7 @@ struct ib_mr *ib_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type,
mr->need_inval = false;
mr->type = mr_type;
mr->sig_attrs = NULL;
+ ib_mr_init_core_refs(mr);
rdma_restrack_new(&mr->res, RDMA_RESTRACK_MR);
rdma_restrack_parent_name(&mr->res, &pd->res);
@@ -2509,6 +2517,7 @@ struct ib_mr *ib_alloc_mr_integrity(struct ib_pd *pd,
mr->need_inval = false;
mr->type = IB_MR_TYPE_INTEGRITY;
mr->sig_attrs = sig_attrs;
+ ib_mr_init_core_refs(mr);
rdma_restrack_new(&mr->res, RDMA_RESTRACK_MR);
rdma_restrack_parent_name(&mr->res, &pd->res);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 3e43d0b8b495..6a0d3a928d97 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2084,6 +2084,13 @@ struct ib_mr {
struct list_head qp_entry; /* FR */
};
+ /* the job this region is restricted to, NULL if it is unrestricted */
+ struct ib_jkey *jkey;
+ /* the region this one was derived from, NULL if registered fresh */
+ struct ib_mr *parent;
+ /* regions derived from this one that are still registered */
+ atomic_t usecnt;
+
struct ib_dm *dm;
struct ib_sig_attrs *sig_attrs; /* only for IB_MR_TYPE_INTEGRITY MRs */
struct ib_dmah *dmah;
@@ -2098,6 +2105,25 @@ struct ib_mr {
struct rdma_restrack_entry res;
};
+struct ib_mr_ex_attr {
+ u64 addr;
+ u64 length;
+ u64 iova;
+ u32 access_flags; /* enum ib_access_flags */
+ u32 flags; /* enum ib_uverbs_reg_mr_flags */
+ int fd; /* -1 = host memory addr */
+ u64 fd_offset;
+ struct ib_dmah *dmah;
+ /* bind the region to a job, NULL leaves it unrestricted */
+ struct ib_jkey *jkey;
+ /* carve/derive the region out of this one, NULL registers fresh */
+ struct ib_mr *parent_mr;
+ /* how many regions the user expects to derive from this one */
+ u32 derive_cnt;
+ /* the key the user chose, valid with IB_UVERBS_REG_MR_USER_RKEY */
+ u64 req_rkey64;
+};
+
struct ib_mw {
struct ib_device *device;
struct ib_pd *pd;
@@ -2813,6 +2839,9 @@ struct ib_device_ops {
u64 length, u64 virt_addr,
int mr_access_flags, struct ib_pd *pd,
struct ib_udata *udata);
+ struct ib_mr *(*reg_user_mr_ex)(struct ib_pd *pd,
+ struct ib_mr_ex_attr *attr,
+ struct ib_udata *udata);
int (*dereg_mr)(struct ib_mr *mr, struct ib_udata *udata);
int (*query_qp_semantics)(struct ib_device *device,
enum ib_qp_type qp_type,
@@ -4823,6 +4852,20 @@ static inline u32 ib_inc_rkey(u32 rkey)
return ((rkey + 1) & mask) | (rkey & ~mask);
}
+/**
+ * ib_mr_init_core_refs - clears what core reads back when an MR is
+ * deregistered.
+ * @mr: struct ib_mr pointer to initialize.
+ *
+ * A driver's MR need not be zeroed, so every path that makes an MR calls this.
+ */
+static inline void ib_mr_init_core_refs(struct ib_mr *mr)
+{
+ mr->jkey = NULL;
+ mr->parent = NULL;
+ atomic_set(&mr->usecnt, 0);
+}
+
/**
* ib_attach_mcast - Attaches the specified QP to a multicast group.
* @qp: QP to attach to the multicast group. The QP must be type
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index e1ffe3a662a0..e4f819ccc288 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -368,6 +368,11 @@ enum uverbs_attrs_reg_mr_cmd_attr_ids {
UVERBS_ATTR_REG_MR_FD_OFFSET,
UVERBS_ATTR_REG_MR_RESP_LKEY,
UVERBS_ATTR_REG_MR_RESP_RKEY,
+ UVERBS_ATTR_REG_MR_FLAGS,
+ UVERBS_ATTR_REG_MR_JKEY_HANDLE,
+ UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
+ UVERBS_ATTR_REG_MR_DERIVE_CNT,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64,
};
enum uverbs_attrs_create_counters_cmd_attr_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 3d05a6c7ca8f..75cdfdbdb2f7 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -303,6 +303,16 @@ struct ib_uverbs_buffer_desc {
__aligned_u64 length;
};
+/* Extended memory registration, for devices whose memory keys are 64 bits
+ * wide and may be bound to a job. See UVERBS_ATTR_REG_MR_FLAGS.
+ */
+enum ib_uverbs_reg_mr_flags {
+ /* REQ_RKEY64 carries the key, rather than the device assigning one */
+ IB_UVERBS_REG_MR_USER_RKEY = 1 << 0,
+ /* the region tolerates a replayed idempotent operation */
+ IB_UVERBS_REG_MR_IDEMPOTENT_SAFE = 1 << 1,
+};
+
/* A UET reliable unconnected queue pair number uses all 32 bits:
*
* [ absolute 1b | reserved 7b | resource index 12b | PIDonFEP 12b ]
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 11/13] RDMA/core: extended memory registration
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
Memory registration for an unconnected transport may take more than an
address and an access mask. A job to bind the region to, a key the caller
chose, or a region carved out of another. None of it fits the write()
command, so add a device method that takes the wider set of attributes
and reach it when the ioctl carries any of these new fields.
A region bound to a job holds a reference to its job key until it is
deregistered.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/device.c | 1 +
drivers/infiniband/core/uverbs_cmd.c | 7 +-
drivers/infiniband/core/uverbs_std_types_mr.c | 182 +++++++++++++++++++++++++-
drivers/infiniband/core/verbs.c | 9 ++
include/rdma/ib_verbs.h | 43 ++++++
include/uapi/rdma/ib_user_ioctl_cmds.h | 5 +
include/uapi/rdma/ib_user_ioctl_verbs.h | 10 ++
7 files changed, 252 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index cf2a59afcf76..fd45decf9c4f 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -3028,6 +3028,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, reg_dm_mr);
SET_DEVICE_OP(dev_ops, reg_user_mr);
SET_DEVICE_OP(dev_ops, reg_user_mr_dmabuf);
+ SET_DEVICE_OP(dev_ops, reg_user_mr_ex);
SET_DEVICE_OP(dev_ops, req_notify_cq);
SET_DEVICE_OP(dev_ops, rereg_user_mr);
SET_DEVICE_OP(dev_ops, resize_user_cq);
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 88212465d8a0..10630abc9ff1 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -731,6 +731,7 @@ static int ib_uverbs_reg_mr(struct uverbs_attr_bundle *attrs)
mr->type = IB_MR_TYPE_USER;
mr->dm = NULL;
mr->sig_attrs = NULL;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
mr->iova = cmd.hca_va;
@@ -790,7 +791,10 @@ static int ib_uverbs_rereg_mr(struct uverbs_attr_bundle *attrs)
mr = uobj->object;
- if (mr->dm) {
+ /* A region bound to a job or to another region is re-created rather
+ * than re-registered, so those bindings can't be lost or bypassed.
+ */
+ if (mr->dm || mr->jkey || mr->parent || atomic_read(&mr->usecnt)) {
ret = -EINVAL;
goto put_uobjs;
}
@@ -838,6 +842,7 @@ static int ib_uverbs_rereg_mr(struct uverbs_attr_bundle *attrs)
new_mr->device = new_pd->device;
new_mr->pd = new_pd;
new_mr->type = IB_MR_TYPE_USER;
+ ib_mr_init_core_refs(new_mr);
new_mr->uobject = uobj;
atomic_inc(&new_pd->usecnt);
new_uobj->object = new_mr;
diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index 0c72f801e0d3..035efba54dcb 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -40,8 +40,13 @@ static int uverbs_free_mr(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
- return ib_dereg_mr_user((struct ib_mr *)uobject->object,
- &attrs->driver_udata);
+ struct ib_mr *mr = uobject->object;
+
+ /* regions derived from this one still use it */
+ if (atomic_read(&mr->usecnt))
+ return -EBUSY;
+
+ return ib_dereg_mr_user(mr, &attrs->driver_udata);
}
static int UVERBS_HANDLER(UVERBS_METHOD_ADVISE_MR)(
@@ -132,6 +137,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_DM_MR_REG)(
mr->pd = pd;
mr->type = IB_MR_TYPE_DM;
mr->dm = dm;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
atomic_inc(&dm->usecnt);
@@ -246,6 +252,9 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_DMABUF_MR)(
mr->device = pd->device;
mr->pd = pd;
mr->type = IB_MR_TYPE_USER;
+ mr->iova = iova;
+ mr->length = length;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
@@ -266,6 +275,116 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_DMABUF_MR)(
return ret;
}
+/* Attributes that an extended MR registration uses. The 64b wide keys are
+ * not in the list as they are not a property of how the MR was registered
+ * and instead a device property which could be used by an ordinary register.
+ */
+static const u16 reg_mr_ext_attrs[] = {
+ UVERBS_ATTR_REG_MR_FLAGS,
+ UVERBS_ATTR_REG_MR_JKEY_HANDLE,
+ UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
+ UVERBS_ATTR_REG_MR_DERIVE_CNT,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64,
+};
+
+static bool reg_mr_has_ext_attrs(struct uverbs_attr_bundle *attrs)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(reg_mr_ext_attrs); i++)
+ if (uverbs_attr_is_valid(attrs, reg_mr_ext_attrs[i]))
+ return true;
+
+ return false;
+}
+
+static int reg_mr_ex_attr(struct uverbs_attr_bundle *attrs, struct ib_pd *pd,
+ struct ib_mr_ex_attr *ex)
+{
+ u64 caps = pd->device->attrs.device_cap_flags;
+ int ret;
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_FLAGS)) {
+ ret = uverbs_get_flags32(&ex->flags, attrs,
+ UVERBS_ATTR_REG_MR_FLAGS,
+ (IB_UVERBS_REG_MR_USER_RKEY |
+ IB_UVERBS_REG_MR_IDEMPOTENT_SAFE));
+ if (ret)
+ return ret;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_JKEY_HANDLE)) {
+ struct ib_jkey *jkey = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_REG_MR_JKEY_HANDLE);
+
+ if (IS_ERR(jkey))
+ return PTR_ERR(jkey);
+
+ /* a key acts within the PD it was made on */
+ if (jkey->pd != pd)
+ return -EINVAL;
+
+ if (!(caps & (IB_DEVICE_MR_JOB_RESTRICTED |
+ IB_DEVICE_MR_QP_JOB_RESTRICTED)))
+ return -EOPNOTSUPP;
+
+ ex->jkey = jkey;
+ }
+
+ /* a region carved out of another, execute the range check */
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE)) {
+ struct ib_mr *parent = uverbs_attr_get_obj(
+ attrs, UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE);
+
+ if (IS_ERR(parent))
+ return PTR_ERR(parent);
+
+ if (parent->pd != pd || parent->type != IB_MR_TYPE_USER)
+ return -EINVAL;
+
+ if ((ex->iova < parent->iova) ||
+ (ex->length > parent->length) ||
+ ((ex->iova - parent->iova) >
+ (parent->length - ex->length)))
+ return -EINVAL;
+
+ ex->parent_mr = parent;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_DERIVE_CNT)) {
+ ret = uverbs_copy_from(&ex->derive_cnt, attrs,
+ UVERBS_ATTR_REG_MR_DERIVE_CNT);
+ if (ret)
+ return ret;
+ }
+
+ if (ex->flags & IB_UVERBS_REG_MR_USER_RKEY) {
+ if (!(caps & IB_DEVICE_USER_RKEY))
+ return -EOPNOTSUPP;
+
+ ret = uverbs_copy_from(&ex->req_rkey64, attrs,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64);
+ if (ret)
+ return ret;
+
+ if (ex->req_rkey64 > U32_MAX && !(caps & IB_DEVICE_KEY64))
+ return -EOPNOTSUPP;
+ } else if (uverbs_attr_is_valid(attrs,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64)) {
+ /* a key with no flag to say it is the caller's */
+ return -EINVAL;
+ }
+
+ /* Idempotent-safe is a property of the key so it can only be asked
+ * for on a key the caller chose.
+ */
+ if ((ex->flags & IB_UVERBS_REG_MR_IDEMPOTENT_SAFE) &&
+ !(ex->flags & IB_UVERBS_REG_MR_USER_RKEY))
+ return -EINVAL;
+
+ return 0;
+}
+
static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
struct uverbs_attr_bundle *attrs)
{
@@ -276,10 +395,12 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
u32 valid_access_flags = IB_ACCESS_SUPPORTED;
u64 length, iova, fd_offset = 0, addr = 0;
struct ib_device *ib_dev = pd->device;
+ struct ib_mr_ex_attr ex = { .fd = -1 };
struct ib_dmah *dmah = NULL;
bool has_fd_offset = false;
bool has_addr = false;
bool has_fd = false;
+ bool has_ext;
u32 access_flags;
struct ib_mr *mr;
int fd;
@@ -293,6 +414,10 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
if (ret)
return ret;
+ has_ext = reg_mr_has_ext_attrs(attrs);
+ if (has_ext && !ib_dev->ops.reg_user_mr_ex)
+ return -EOPNOTSUPP;
+
if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_ADDR)) {
ret = uverbs_copy_from(&addr, attrs,
UVERBS_ATTR_REG_MR_ADDR);
@@ -318,7 +443,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
}
if (has_fd) {
- if (!ib_dev->ops.reg_user_mr_dmabuf)
+ /* an extended registration hands the fd to reg_user_mr_ex */
+ if (!has_ext && !ib_dev->ops.reg_user_mr_dmabuf)
return -EOPNOTSUPP;
/* FD requires offset and can't come with addr */
@@ -358,7 +484,24 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
if (ret)
return ret;
- if (has_fd)
+ if (has_ext) {
+ ex.addr = addr;
+ ex.length = length;
+ ex.iova = iova;
+ ex.access_flags = access_flags;
+ ex.dmah = dmah;
+ if (has_fd) {
+ ex.fd = fd;
+ ex.fd_offset = fd_offset;
+ }
+
+ ret = reg_mr_ex_attr(attrs, pd, &ex);
+ if (ret)
+ return ret;
+
+ mr = pd->device->ops.reg_user_mr_ex(pd, &ex,
+ &attrs->driver_udata);
+ } else if (has_fd)
mr = pd->device->ops.reg_user_mr_dmabuf(pd, fd_offset, length,
iova, fd, access_flags,
dmah, attrs);
@@ -373,12 +516,26 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
mr->device = pd->device;
mr->pd = pd;
mr->type = IB_MR_TYPE_USER;
+ mr->iova = iova;
+ mr->length = length;
+ ib_mr_init_core_refs(mr);
mr->uobject = uobj;
atomic_inc(&pd->usecnt);
if (dmah) {
mr->dmah = dmah;
atomic_inc(&dmah->usecnt);
}
+
+ /* hold a reference to the job key, deregistration will drop it */
+ mr->jkey = ex.jkey;
+ if (ex.jkey)
+ atomic_inc(&ex.jkey->usecnt);
+
+ /* and to the region this one was derived from */
+ mr->parent = ex.parent_mr;
+ if (ex.parent_mr)
+ atomic_inc(&ex.parent_mr->usecnt);
+
rdma_restrack_new(&mr->res, RDMA_RESTRACK_MR);
rdma_restrack_set_name(&mr->res, NULL);
rdma_restrack_add(&mr->res);
@@ -529,6 +686,23 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_RKEY,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
+ UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_REG_MR_FLAGS,
+ enum ib_uverbs_reg_mr_flags,
+ UA_OPTIONAL),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_REG_MR_JKEY_HANDLE,
+ UVERBS_OBJECT_JKEY,
+ UVERBS_ACCESS_READ,
+ UA_OPTIONAL),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
+ UVERBS_OBJECT_MR,
+ UVERBS_ACCESS_READ,
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_DERIVE_CNT,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_REQ_RKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
UVERBS_ATTR_UHW());
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 3446aaa16ef4..87dc48ecb77a 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -2358,6 +2358,7 @@ struct ib_mr *ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mr->type = IB_MR_TYPE_USER;
mr->pd = pd;
mr->dm = NULL;
+ ib_mr_init_core_refs(mr);
atomic_inc(&pd->usecnt);
mr->iova = virt_addr;
mr->length = length;
@@ -2389,6 +2390,8 @@ int ib_dereg_mr_user(struct ib_mr *mr, struct ib_udata *udata)
struct ib_pd *pd = mr->pd;
struct ib_dm *dm = mr->dm;
struct ib_dmah *dmah = mr->dmah;
+ struct ib_jkey *jkey = mr->jkey;
+ struct ib_mr *parent = mr->parent;
struct ib_sig_attrs *sig_attrs = mr->sig_attrs;
int ret;
@@ -2401,6 +2404,10 @@ int ib_dereg_mr_user(struct ib_mr *mr, struct ib_udata *udata)
atomic_dec(&dm->usecnt);
if (dmah)
atomic_dec(&dmah->usecnt);
+ if (jkey)
+ atomic_dec(&jkey->usecnt);
+ if (parent)
+ atomic_dec(&parent->usecnt);
kfree(sig_attrs);
}
@@ -2448,6 +2455,7 @@ struct ib_mr *ib_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type,
mr->need_inval = false;
mr->type = mr_type;
mr->sig_attrs = NULL;
+ ib_mr_init_core_refs(mr);
rdma_restrack_new(&mr->res, RDMA_RESTRACK_MR);
rdma_restrack_parent_name(&mr->res, &pd->res);
@@ -2509,6 +2517,7 @@ struct ib_mr *ib_alloc_mr_integrity(struct ib_pd *pd,
mr->need_inval = false;
mr->type = IB_MR_TYPE_INTEGRITY;
mr->sig_attrs = sig_attrs;
+ ib_mr_init_core_refs(mr);
rdma_restrack_new(&mr->res, RDMA_RESTRACK_MR);
rdma_restrack_parent_name(&mr->res, &pd->res);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 3e43d0b8b495..6a0d3a928d97 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2084,6 +2084,13 @@ struct ib_mr {
struct list_head qp_entry; /* FR */
};
+ /* the job this region is restricted to, NULL if it is unrestricted */
+ struct ib_jkey *jkey;
+ /* the region this one was derived from, NULL if registered fresh */
+ struct ib_mr *parent;
+ /* regions derived from this one that are still registered */
+ atomic_t usecnt;
+
struct ib_dm *dm;
struct ib_sig_attrs *sig_attrs; /* only for IB_MR_TYPE_INTEGRITY MRs */
struct ib_dmah *dmah;
@@ -2098,6 +2105,25 @@ struct ib_mr {
struct rdma_restrack_entry res;
};
+struct ib_mr_ex_attr {
+ u64 addr;
+ u64 length;
+ u64 iova;
+ u32 access_flags; /* enum ib_access_flags */
+ u32 flags; /* enum ib_uverbs_reg_mr_flags */
+ int fd; /* -1 = host memory addr */
+ u64 fd_offset;
+ struct ib_dmah *dmah;
+ /* bind the region to a job, NULL leaves it unrestricted */
+ struct ib_jkey *jkey;
+ /* carve/derive the region out of this one, NULL registers fresh */
+ struct ib_mr *parent_mr;
+ /* how many regions the user expects to derive from this one */
+ u32 derive_cnt;
+ /* the key the user chose, valid with IB_UVERBS_REG_MR_USER_RKEY */
+ u64 req_rkey64;
+};
+
struct ib_mw {
struct ib_device *device;
struct ib_pd *pd;
@@ -2813,6 +2839,9 @@ struct ib_device_ops {
u64 length, u64 virt_addr,
int mr_access_flags, struct ib_pd *pd,
struct ib_udata *udata);
+ struct ib_mr *(*reg_user_mr_ex)(struct ib_pd *pd,
+ struct ib_mr_ex_attr *attr,
+ struct ib_udata *udata);
int (*dereg_mr)(struct ib_mr *mr, struct ib_udata *udata);
int (*query_qp_semantics)(struct ib_device *device,
enum ib_qp_type qp_type,
@@ -4823,6 +4852,20 @@ static inline u32 ib_inc_rkey(u32 rkey)
return ((rkey + 1) & mask) | (rkey & ~mask);
}
+/**
+ * ib_mr_init_core_refs - clears what core reads back when an MR is
+ * deregistered.
+ * @mr: struct ib_mr pointer to initialize.
+ *
+ * A driver's MR need not be zeroed, so every path that makes an MR calls this.
+ */
+static inline void ib_mr_init_core_refs(struct ib_mr *mr)
+{
+ mr->jkey = NULL;
+ mr->parent = NULL;
+ atomic_set(&mr->usecnt, 0);
+}
+
/**
* ib_attach_mcast - Attaches the specified QP to a multicast group.
* @qp: QP to attach to the multicast group. The QP must be type
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index e1ffe3a662a0..e4f819ccc288 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -368,6 +368,11 @@ enum uverbs_attrs_reg_mr_cmd_attr_ids {
UVERBS_ATTR_REG_MR_FD_OFFSET,
UVERBS_ATTR_REG_MR_RESP_LKEY,
UVERBS_ATTR_REG_MR_RESP_RKEY,
+ UVERBS_ATTR_REG_MR_FLAGS,
+ UVERBS_ATTR_REG_MR_JKEY_HANDLE,
+ UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
+ UVERBS_ATTR_REG_MR_DERIVE_CNT,
+ UVERBS_ATTR_REG_MR_REQ_RKEY64,
};
enum uverbs_attrs_create_counters_cmd_attr_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 3d05a6c7ca8f..75cdfdbdb2f7 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -303,6 +303,16 @@ struct ib_uverbs_buffer_desc {
__aligned_u64 length;
};
+/* Extended memory registration, for devices whose memory keys are 64 bits
+ * wide and may be bound to a job. See UVERBS_ATTR_REG_MR_FLAGS.
+ */
+enum ib_uverbs_reg_mr_flags {
+ /* REQ_RKEY64 carries the key, rather than the device assigning one */
+ IB_UVERBS_REG_MR_USER_RKEY = 1 << 0,
+ /* the region tolerates a replayed idempotent operation */
+ IB_UVERBS_REG_MR_IDEMPOTENT_SAFE = 1 << 1,
+};
+
/* A UET reliable unconnected queue pair number uses all 32 bits:
*
* [ absolute 1b | reserved 7b | resource index 12b | PIDonFEP 12b ]
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 11/13] RDMA/core: extended memory registration
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:36 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:36 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> Memory registration for an unconnected transport may take more than an
> address and an access mask. A job to bind the region to, a key the caller
> chose, or a region carved out of another. None of it fits the write()
> command, so add a device method that takes the wider set of attributes
> and reach it when the ioctl carries any of these new fields.
>
> A region bound to a job holds a reference to its job key until it is
> deregistered.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=11
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 12/13] RDMA/core: 64-bit memory keys
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
A device whose keys are wider than 32 bits has no way to name a region
through lkey and rkey. Carry the wide pair on the region and return it
from registration when userspace asks for it.
The attributes are optional in both directions, so a kernel that does not
know them ignores the request, and a region without a wide key answers
zero.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_std_types_mr.c | 40 +++++++++++++++++++++++++++
include/rdma/ib_verbs.h | 12 ++++++--
include/uapi/rdma/ib_user_ioctl_cmds.h | 4 +++
3 files changed, 54 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index 035efba54dcb..2957cae25a7b 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -185,6 +185,16 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QUERY_MR)(
ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_MR_RESP_IOVA,
&mr->iova, sizeof(mr->iova));
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_MR_RESP_LKEY64,
+ &mr->lkey64, sizeof(mr->lkey64));
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_MR_RESP_RKEY64,
+ &mr->rkey64, sizeof(mr->rkey64));
return IS_UVERBS_COPY_ERR(ret) ? ret : 0;
}
@@ -367,6 +377,10 @@ static int reg_mr_ex_attr(struct uverbs_attr_bundle *attrs, struct ib_pd *pd,
if (ret)
return ret;
+ /* zero names no wide key, so it can't be one */
+ if (!ex->req_rkey64)
+ return -EINVAL;
+
if (ex->req_rkey64 > U32_MAX && !(caps & IB_DEVICE_KEY64))
return -EOPNOTSUPP;
} else if (uverbs_attr_is_valid(attrs,
@@ -550,6 +564,20 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_RKEY,
&mr->rkey, sizeof(mr->rkey));
+ if (ret)
+ return ret;
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_RESP_LKEY64)) {
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_LKEY64,
+ &mr->lkey64, sizeof(mr->lkey64));
+ if (ret)
+ return ret;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_RESP_RKEY64))
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_RKEY64,
+ &mr->rkey64, sizeof(mr->rkey64));
+
return ret;
}
@@ -586,6 +614,12 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_TYPE(u64),
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_MR_RESP_IOVA,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_MR_RESP_LKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_MR_RESP_RKEY64,
UVERBS_ATTR_TYPE(u64),
UA_OPTIONAL));
@@ -703,6 +737,12 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_REQ_RKEY64,
UVERBS_ATTR_TYPE(u64),
UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_LKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_RKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
UVERBS_ATTR_UHW());
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 6a0d3a928d97..f9171b47ac1b 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2084,6 +2084,9 @@ struct ib_mr {
struct list_head qp_entry; /* FR */
};
+ /* 64b keys, set by a driver reporting IB_DEVICE_KEY64, else zero */
+ u64 lkey64;
+ u64 rkey64;
/* the job this region is restricted to, NULL if it is unrestricted */
struct ib_jkey *jkey;
/* the region this one was derived from, NULL if registered fresh */
@@ -4854,8 +4857,8 @@ static inline u32 ib_inc_rkey(u32 rkey)
/**
* ib_mr_init_core_refs - clears what core reads back when an MR is
- * deregistered.
- * @mr: struct ib_mr pointer to initialize.
+ * deregistered, and the wide keys of an MR on a device without them.
+ * @mr: struct ib_mr pointer to initialize, with its device set.
*
* A driver's MR need not be zeroed, so every path that makes an MR calls this.
*/
@@ -4864,6 +4867,11 @@ static inline void ib_mr_init_core_refs(struct ib_mr *mr)
mr->jkey = NULL;
mr->parent = NULL;
atomic_set(&mr->usecnt, 0);
+
+ if (!(mr->device->attrs.device_cap_flags & IB_DEVICE_KEY64)) {
+ mr->lkey64 = 0;
+ mr->rkey64 = 0;
+ }
}
/**
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index e4f819ccc288..51910240a7e1 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -342,6 +342,8 @@ enum uverbs_attrs_query_mr_cmd_attr_ids {
UVERBS_ATTR_QUERY_MR_RESP_RKEY,
UVERBS_ATTR_QUERY_MR_RESP_LENGTH,
UVERBS_ATTR_QUERY_MR_RESP_IOVA,
+ UVERBS_ATTR_QUERY_MR_RESP_LKEY64,
+ UVERBS_ATTR_QUERY_MR_RESP_RKEY64,
};
enum uverbs_attrs_reg_dmabuf_mr_cmd_attr_ids {
@@ -373,6 +375,8 @@ enum uverbs_attrs_reg_mr_cmd_attr_ids {
UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
UVERBS_ATTR_REG_MR_DERIVE_CNT,
UVERBS_ATTR_REG_MR_REQ_RKEY64,
+ UVERBS_ATTR_REG_MR_RESP_LKEY64,
+ UVERBS_ATTR_REG_MR_RESP_RKEY64,
};
enum uverbs_attrs_create_counters_cmd_attr_ids {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 12/13] RDMA/core: 64-bit memory keys
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
A device whose keys are wider than 32 bits has no way to name a region
through lkey and rkey. Carry the wide pair on the region and return it
from registration when userspace asks for it.
The attributes are optional in both directions, so a kernel that does not
know them ignores the request, and a region without a wide key answers
zero.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/uverbs_std_types_mr.c | 40 +++++++++++++++++++++++++++
include/rdma/ib_verbs.h | 12 ++++++--
include/uapi/rdma/ib_user_ioctl_cmds.h | 4 +++
3 files changed, 54 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index 035efba54dcb..2957cae25a7b 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -185,6 +185,16 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QUERY_MR)(
ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_MR_RESP_IOVA,
&mr->iova, sizeof(mr->iova));
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_MR_RESP_LKEY64,
+ &mr->lkey64, sizeof(mr->lkey64));
+ if (IS_UVERBS_COPY_ERR(ret))
+ return ret;
+
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_MR_RESP_RKEY64,
+ &mr->rkey64, sizeof(mr->rkey64));
return IS_UVERBS_COPY_ERR(ret) ? ret : 0;
}
@@ -367,6 +377,10 @@ static int reg_mr_ex_attr(struct uverbs_attr_bundle *attrs, struct ib_pd *pd,
if (ret)
return ret;
+ /* zero names no wide key, so it can't be one */
+ if (!ex->req_rkey64)
+ return -EINVAL;
+
if (ex->req_rkey64 > U32_MAX && !(caps & IB_DEVICE_KEY64))
return -EOPNOTSUPP;
} else if (uverbs_attr_is_valid(attrs,
@@ -550,6 +564,20 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_RKEY,
&mr->rkey, sizeof(mr->rkey));
+ if (ret)
+ return ret;
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_RESP_LKEY64)) {
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_LKEY64,
+ &mr->lkey64, sizeof(mr->lkey64));
+ if (ret)
+ return ret;
+ }
+
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_RESP_RKEY64))
+ ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_RKEY64,
+ &mr->rkey64, sizeof(mr->rkey64));
+
return ret;
}
@@ -586,6 +614,12 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_TYPE(u64),
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_MR_RESP_IOVA,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_MR_RESP_LKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_QUERY_MR_RESP_RKEY64,
UVERBS_ATTR_TYPE(u64),
UA_OPTIONAL));
@@ -703,6 +737,12 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_REQ_RKEY64,
UVERBS_ATTR_TYPE(u64),
UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_LKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_RKEY64,
+ UVERBS_ATTR_TYPE(u64),
+ UA_OPTIONAL),
UVERBS_ATTR_UHW());
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 6a0d3a928d97..f9171b47ac1b 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2084,6 +2084,9 @@ struct ib_mr {
struct list_head qp_entry; /* FR */
};
+ /* 64b keys, set by a driver reporting IB_DEVICE_KEY64, else zero */
+ u64 lkey64;
+ u64 rkey64;
/* the job this region is restricted to, NULL if it is unrestricted */
struct ib_jkey *jkey;
/* the region this one was derived from, NULL if registered fresh */
@@ -4854,8 +4857,8 @@ static inline u32 ib_inc_rkey(u32 rkey)
/**
* ib_mr_init_core_refs - clears what core reads back when an MR is
- * deregistered.
- * @mr: struct ib_mr pointer to initialize.
+ * deregistered, and the wide keys of an MR on a device without them.
+ * @mr: struct ib_mr pointer to initialize, with its device set.
*
* A driver's MR need not be zeroed, so every path that makes an MR calls this.
*/
@@ -4864,6 +4867,11 @@ static inline void ib_mr_init_core_refs(struct ib_mr *mr)
mr->jkey = NULL;
mr->parent = NULL;
atomic_set(&mr->usecnt, 0);
+
+ if (!(mr->device->attrs.device_cap_flags & IB_DEVICE_KEY64)) {
+ mr->lkey64 = 0;
+ mr->rkey64 = 0;
+ }
}
/**
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index e4f819ccc288..51910240a7e1 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -342,6 +342,8 @@ enum uverbs_attrs_query_mr_cmd_attr_ids {
UVERBS_ATTR_QUERY_MR_RESP_RKEY,
UVERBS_ATTR_QUERY_MR_RESP_LENGTH,
UVERBS_ATTR_QUERY_MR_RESP_IOVA,
+ UVERBS_ATTR_QUERY_MR_RESP_LKEY64,
+ UVERBS_ATTR_QUERY_MR_RESP_RKEY64,
};
enum uverbs_attrs_reg_dmabuf_mr_cmd_attr_ids {
@@ -373,6 +375,8 @@ enum uverbs_attrs_reg_mr_cmd_attr_ids {
UVERBS_ATTR_REG_MR_PARENT_MR_HANDLE,
UVERBS_ATTR_REG_MR_DERIVE_CNT,
UVERBS_ATTR_REG_MR_REQ_RKEY64,
+ UVERBS_ATTR_REG_MR_RESP_LKEY64,
+ UVERBS_ATTR_REG_MR_RESP_RKEY64,
};
enum uverbs_attrs_create_counters_cmd_attr_ids {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 12/13] RDMA/core: 64-bit memory keys
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:32 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:32 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> A device whose keys are wider than 32 bits has no way to name a region
> through lkey and rkey. Carry the wide pair on the region and return it
> from registration when userspace asks for it.
>
> The attributes are optional in both directions, so a kernel that does not
> know them ignores the request, and a region without a wide key answers
> zero.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=12
^ permalink raw reply [flat|nested] 60+ messages in thread
* [PATCH rdma-next 13/13] RDMA/core: attach a memory region to a queue pair
2026-10-14 22:21 ` Eric Davis
@ 2026-10-14 22:21 ` Eric Davis
-1 siblings, 0 replies; 60+ messages in thread
From: Eric Davis via B4 Relay @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
From: Eric Davis <eric.davis@broadcom.com>
A region registered against a protection domain is reachable from every QP
in it. When a device can narrow that, let a region be attached to one QP
and detached again, so the network only reaches the MR over that QP.
Which restrictions a device supports is reported through the memory region
access capability bits.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/device.c | 2 +
drivers/infiniband/core/uverbs_std_types_mr.c | 2 +-
drivers/infiniband/core/uverbs_std_types_qp.c | 110 +++++++++++++++++++++++++-
drivers/infiniband/core/verbs.c | 10 +++
include/rdma/ib_verbs.h | 11 ++-
include/uapi/rdma/ib_user_ioctl_cmds.h | 12 +++
6 files changed, 144 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index fd45decf9c4f..1b838bf3600d 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -3018,6 +3018,8 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, query_pkey);
SET_DEVICE_OP(dev_ops, query_port);
SET_DEVICE_OP(dev_ops, query_port_speed);
+ SET_DEVICE_OP(dev_ops, qp_attach_mr);
+ SET_DEVICE_OP(dev_ops, qp_detach_mr);
SET_DEVICE_OP(dev_ops, query_qp);
SET_DEVICE_OP(dev_ops, query_qp_semantics);
SET_DEVICE_OP(dev_ops, query_srq);
diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index 2957cae25a7b..2b3f9755daba 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -42,7 +42,7 @@ static int uverbs_free_mr(struct ib_uobject *uobject,
{
struct ib_mr *mr = uobject->object;
- /* regions derived from this one still use it */
+ /* regions derived from this one, or the QP it is attached to, use it */
if (atomic_read(&mr->usecnt))
return -EBUSY;
diff --git a/drivers/infiniband/core/uverbs_std_types_qp.c b/drivers/infiniband/core/uverbs_std_types_qp.c
index 47ca994f9ac9..4593965069f1 100644
--- a/drivers/infiniband/core/uverbs_std_types_qp.c
+++ b/drivers/infiniband/core/uverbs_std_types_qp.c
@@ -666,12 +666,120 @@ DECLARE_UVERBS_NAMED_METHOD(
enum ib_uverbs_qp_attach_comp_cntr_op,
UA_MANDATORY));
+/* bind a region to the QP that will use it */
+static int UVERBS_HANDLER(UVERBS_METHOD_QP_ATTACH_MR)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_qp *qp =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_ATTACH_MR_QP_HANDLE);
+ struct ib_mr *mr =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_ATTACH_MR_MR_HANDLE);
+ u64 caps = qp->device->attrs.device_cap_flags;
+ u32 id;
+ int ret;
+
+ if (!qp->device->ops.qp_attach_mr)
+ return -EOPNOTSUPP;
+
+ if (qp->real_qp != qp || qp->qp_type != IB_QPT_RU)
+ return -EINVAL;
+
+ if (mr->pd != qp->pd || mr->type != IB_MR_TYPE_USER)
+ return -EINVAL;
+
+ /* the device has to implement the access class the region asks for */
+ if (!(caps & (mr->jkey ? IB_DEVICE_MR_QP_JOB_RESTRICTED :
+ IB_DEVICE_MR_QP_RESTRICTED)))
+ return -EOPNOTSUPP;
+
+ /* a region restricted to a job serves a QP in that job */
+ if (mr->jkey && (!qp->jkey || qp->jkey->job != mr->jkey->job))
+ return -EINVAL;
+
+ /* one QP at a time */
+ if (cmpxchg(&mr->attached_qp, NULL, qp))
+ return -EBUSY;
+
+ ret = xa_alloc(&qp->attached_mrs, &id, mr, xa_limit_32b, GFP_KERNEL);
+ if (ret)
+ goto err_claim;
+
+ ret = qp->device->ops.qp_attach_mr(qp, mr);
+ if (ret)
+ goto err_xa;
+
+ /* the QP holds the region until detach or its own destroy */
+ mr->attach_id = id;
+ atomic_inc(&mr->usecnt);
+
+ return 0;
+
+err_xa:
+ xa_erase(&qp->attached_mrs, id);
+err_claim:
+ WRITE_ONCE(mr->attached_qp, NULL);
+ return ret;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_QP_DETACH_MR)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_qp *qp =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_DETACH_MR_QP_HANDLE);
+ struct ib_mr *mr =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_DETACH_MR_MR_HANDLE);
+ int ret;
+
+ if (!qp->device->ops.qp_detach_mr)
+ return -EOPNOTSUPP;
+
+ if (qp->real_qp != qp)
+ return -EINVAL;
+
+ if (READ_ONCE(mr->attached_qp) != qp)
+ return -EINVAL;
+
+ ret = qp->device->ops.qp_detach_mr(qp, mr);
+ if (ret)
+ return ret;
+
+ xa_erase(&qp->attached_mrs, mr->attach_id);
+ WRITE_ONCE(mr->attached_qp, NULL);
+ atomic_dec(&mr->usecnt);
+
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_QP_ATTACH_MR,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_ATTACH_MR_QP_HANDLE,
+ UVERBS_OBJECT_QP,
+ UVERBS_ACCESS_WRITE,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_ATTACH_MR_MR_HANDLE,
+ UVERBS_OBJECT_MR,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_QP_DETACH_MR,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_DETACH_MR_QP_HANDLE,
+ UVERBS_OBJECT_QP,
+ UVERBS_ACCESS_WRITE,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_DETACH_MR_MR_HANDLE,
+ UVERBS_OBJECT_MR,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY));
+
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_QP,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uqp_object), uverbs_free_qp),
&UVERBS_METHOD(UVERBS_METHOD_QP_CREATE),
&UVERBS_METHOD(UVERBS_METHOD_QP_DESTROY),
- &UVERBS_METHOD(UVERBS_METHOD_QP_ATTACH_COMP_CNTR));
+ &UVERBS_METHOD(UVERBS_METHOD_QP_ATTACH_COMP_CNTR),
+ &UVERBS_METHOD(UVERBS_METHOD_QP_ATTACH_MR),
+ &UVERBS_METHOD(UVERBS_METHOD_QP_DETACH_MR));
const struct uapi_definition uverbs_def_obj_qp[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_QP,
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 87dc48ecb77a..add6034ff061 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -1351,6 +1351,7 @@ static struct ib_qp *create_qp(struct ib_device *dev, struct ib_pd *pd,
spin_lock_init(&qp->mr_lock);
xa_init(&qp->comp_cntrs);
+ xa_init_flags(&qp->attached_mrs, XA_FLAGS_ALLOC);
INIT_LIST_HEAD(&qp->rdma_mrs);
INIT_LIST_HEAD(&qp->sig_mrs);
init_completion(&qp->srq_completion);
@@ -1385,6 +1386,7 @@ static struct ib_qp *create_qp(struct ib_device *dev, struct ib_pd *pd,
err_create:
rdma_restrack_put(&qp->res);
xa_destroy(&qp->comp_cntrs);
+ xa_destroy(&qp->attached_mrs);
kfree(qp);
return ERR_PTR(ret);
@@ -2212,6 +2214,7 @@ int ib_destroy_qp_user(struct ib_qp *qp, struct ib_udata *udata)
struct ib_qp_security *sec;
struct ib_comp_cntr *cc;
unsigned long index;
+ struct ib_mr *mr;
int ret;
WARN_ON_ONCE(qp->mrs_used > 0);
@@ -2249,6 +2252,13 @@ int ib_destroy_qp_user(struct ib_qp *qp, struct ib_udata *udata)
atomic_dec(&cc->usecnt);
xa_destroy(&qp->comp_cntrs);
+ /* the driver ended the bindings with the QP, release the regions */
+ xa_for_each(&qp->attached_mrs, index, mr) {
+ WRITE_ONCE(mr->attached_qp, NULL);
+ atomic_dec(&mr->usecnt);
+ }
+ xa_destroy(&qp->attached_mrs);
+
ib_qp_usecnt_dec(qp);
if (sec)
ib_destroy_qp_security_end(sec);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index f9171b47ac1b..c9ddc4147ef8 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2003,6 +2003,8 @@ struct ib_qp {
struct list_head xrcd_list;
struct xarray comp_cntrs; /* op_mask -> comp_cntr */
u32 comp_cntr_op_mask;
+ /* regions attached to this QP, each held until detach or destroy */
+ struct xarray attached_mrs;
/* count times opened, mcast attaches, flow attaches */
atomic_t usecnt;
@@ -2091,8 +2093,11 @@ struct ib_mr {
struct ib_jkey *jkey;
/* the region this one was derived from, NULL if registered fresh */
struct ib_mr *parent;
- /* regions derived from this one that are still registered */
+ /* regions derived from this one, and an attachment, that hold it */
atomic_t usecnt;
+ /* the QP this region is attached to, NULL if none */
+ struct ib_qp *attached_qp;
+ u32 attach_id; /* index in attached_qp->attached_mrs */
struct ib_dm *dm;
struct ib_sig_attrs *sig_attrs; /* only for IB_MR_TYPE_INTEGRITY MRs */
@@ -2846,6 +2851,9 @@ struct ib_device_ops {
struct ib_mr_ex_attr *attr,
struct ib_udata *udata);
int (*dereg_mr)(struct ib_mr *mr, struct ib_udata *udata);
+ /* attach/detach a region so it is bound to single QP (restricted) */
+ int (*qp_attach_mr)(struct ib_qp *qp, struct ib_mr *mr);
+ int (*qp_detach_mr)(struct ib_qp *qp, struct ib_mr *mr);
int (*query_qp_semantics)(struct ib_device *device,
enum ib_qp_type qp_type,
struct ib_qp_semantics *sem);
@@ -4866,6 +4874,7 @@ static inline void ib_mr_init_core_refs(struct ib_mr *mr)
{
mr->jkey = NULL;
mr->parent = NULL;
+ mr->attached_qp = NULL;
atomic_set(&mr->usecnt, 0);
if (!(mr->device->attrs.device_cap_flags & IB_DEVICE_KEY64)) {
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 51910240a7e1..8359f7afeded 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -199,6 +199,18 @@ enum uverbs_methods_qp {
UVERBS_METHOD_QP_CREATE,
UVERBS_METHOD_QP_DESTROY,
UVERBS_METHOD_QP_ATTACH_COMP_CNTR,
+ UVERBS_METHOD_QP_ATTACH_MR,
+ UVERBS_METHOD_QP_DETACH_MR,
+};
+
+enum uverbs_attrs_qp_attach_mr_cmd_attr_ids {
+ UVERBS_ATTR_QP_ATTACH_MR_QP_HANDLE,
+ UVERBS_ATTR_QP_ATTACH_MR_MR_HANDLE,
+};
+
+enum uverbs_attrs_qp_detach_mr_cmd_attr_ids {
+ UVERBS_ATTR_QP_DETACH_MR_QP_HANDLE,
+ UVERBS_ATTR_QP_DETACH_MR_MR_HANDLE,
};
enum uverbs_attrs_create_srq_cmd_attr_ids {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* [PATCH rdma-next 13/13] RDMA/core: attach a memory region to a queue pair
@ 2026-10-14 22:21 ` Eric Davis
0 siblings, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-14 22:21 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma, linux-kernel, Eric Davis
A region registered against a protection domain is reachable from every QP
in it. When a device can narrow that, let a region be attached to one QP
and detached again, so the network only reaches the MR over that QP.
Which restrictions a device supports is reported through the memory region
access capability bits.
Signed-off-by: Eric Davis <eric.davis@broadcom.com>
---
drivers/infiniband/core/device.c | 2 +
drivers/infiniband/core/uverbs_std_types_mr.c | 2 +-
drivers/infiniband/core/uverbs_std_types_qp.c | 110 +++++++++++++++++++++++++-
drivers/infiniband/core/verbs.c | 10 +++
include/rdma/ib_verbs.h | 11 ++-
include/uapi/rdma/ib_user_ioctl_cmds.h | 12 +++
6 files changed, 144 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index fd45decf9c4f..1b838bf3600d 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -3018,6 +3018,8 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, query_pkey);
SET_DEVICE_OP(dev_ops, query_port);
SET_DEVICE_OP(dev_ops, query_port_speed);
+ SET_DEVICE_OP(dev_ops, qp_attach_mr);
+ SET_DEVICE_OP(dev_ops, qp_detach_mr);
SET_DEVICE_OP(dev_ops, query_qp);
SET_DEVICE_OP(dev_ops, query_qp_semantics);
SET_DEVICE_OP(dev_ops, query_srq);
diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index 2957cae25a7b..2b3f9755daba 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -42,7 +42,7 @@ static int uverbs_free_mr(struct ib_uobject *uobject,
{
struct ib_mr *mr = uobject->object;
- /* regions derived from this one still use it */
+ /* regions derived from this one, or the QP it is attached to, use it */
if (atomic_read(&mr->usecnt))
return -EBUSY;
diff --git a/drivers/infiniband/core/uverbs_std_types_qp.c b/drivers/infiniband/core/uverbs_std_types_qp.c
index 47ca994f9ac9..4593965069f1 100644
--- a/drivers/infiniband/core/uverbs_std_types_qp.c
+++ b/drivers/infiniband/core/uverbs_std_types_qp.c
@@ -666,12 +666,120 @@ DECLARE_UVERBS_NAMED_METHOD(
enum ib_uverbs_qp_attach_comp_cntr_op,
UA_MANDATORY));
+/* bind a region to the QP that will use it */
+static int UVERBS_HANDLER(UVERBS_METHOD_QP_ATTACH_MR)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_qp *qp =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_ATTACH_MR_QP_HANDLE);
+ struct ib_mr *mr =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_ATTACH_MR_MR_HANDLE);
+ u64 caps = qp->device->attrs.device_cap_flags;
+ u32 id;
+ int ret;
+
+ if (!qp->device->ops.qp_attach_mr)
+ return -EOPNOTSUPP;
+
+ if (qp->real_qp != qp || qp->qp_type != IB_QPT_RU)
+ return -EINVAL;
+
+ if (mr->pd != qp->pd || mr->type != IB_MR_TYPE_USER)
+ return -EINVAL;
+
+ /* the device has to implement the access class the region asks for */
+ if (!(caps & (mr->jkey ? IB_DEVICE_MR_QP_JOB_RESTRICTED :
+ IB_DEVICE_MR_QP_RESTRICTED)))
+ return -EOPNOTSUPP;
+
+ /* a region restricted to a job serves a QP in that job */
+ if (mr->jkey && (!qp->jkey || qp->jkey->job != mr->jkey->job))
+ return -EINVAL;
+
+ /* one QP at a time */
+ if (cmpxchg(&mr->attached_qp, NULL, qp))
+ return -EBUSY;
+
+ ret = xa_alloc(&qp->attached_mrs, &id, mr, xa_limit_32b, GFP_KERNEL);
+ if (ret)
+ goto err_claim;
+
+ ret = qp->device->ops.qp_attach_mr(qp, mr);
+ if (ret)
+ goto err_xa;
+
+ /* the QP holds the region until detach or its own destroy */
+ mr->attach_id = id;
+ atomic_inc(&mr->usecnt);
+
+ return 0;
+
+err_xa:
+ xa_erase(&qp->attached_mrs, id);
+err_claim:
+ WRITE_ONCE(mr->attached_qp, NULL);
+ return ret;
+}
+
+static int UVERBS_HANDLER(UVERBS_METHOD_QP_DETACH_MR)(
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_qp *qp =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_DETACH_MR_QP_HANDLE);
+ struct ib_mr *mr =
+ uverbs_attr_get_obj(attrs, UVERBS_ATTR_QP_DETACH_MR_MR_HANDLE);
+ int ret;
+
+ if (!qp->device->ops.qp_detach_mr)
+ return -EOPNOTSUPP;
+
+ if (qp->real_qp != qp)
+ return -EINVAL;
+
+ if (READ_ONCE(mr->attached_qp) != qp)
+ return -EINVAL;
+
+ ret = qp->device->ops.qp_detach_mr(qp, mr);
+ if (ret)
+ return ret;
+
+ xa_erase(&qp->attached_mrs, mr->attach_id);
+ WRITE_ONCE(mr->attached_qp, NULL);
+ atomic_dec(&mr->usecnt);
+
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_QP_ATTACH_MR,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_ATTACH_MR_QP_HANDLE,
+ UVERBS_OBJECT_QP,
+ UVERBS_ACCESS_WRITE,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_ATTACH_MR_MR_HANDLE,
+ UVERBS_OBJECT_MR,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD(
+ UVERBS_METHOD_QP_DETACH_MR,
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_DETACH_MR_QP_HANDLE,
+ UVERBS_OBJECT_QP,
+ UVERBS_ACCESS_WRITE,
+ UA_MANDATORY),
+ UVERBS_ATTR_IDR(UVERBS_ATTR_QP_DETACH_MR_MR_HANDLE,
+ UVERBS_OBJECT_MR,
+ UVERBS_ACCESS_READ,
+ UA_MANDATORY));
+
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_QP,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uqp_object), uverbs_free_qp),
&UVERBS_METHOD(UVERBS_METHOD_QP_CREATE),
&UVERBS_METHOD(UVERBS_METHOD_QP_DESTROY),
- &UVERBS_METHOD(UVERBS_METHOD_QP_ATTACH_COMP_CNTR));
+ &UVERBS_METHOD(UVERBS_METHOD_QP_ATTACH_COMP_CNTR),
+ &UVERBS_METHOD(UVERBS_METHOD_QP_ATTACH_MR),
+ &UVERBS_METHOD(UVERBS_METHOD_QP_DETACH_MR));
const struct uapi_definition uverbs_def_obj_qp[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_QP,
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 87dc48ecb77a..add6034ff061 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -1351,6 +1351,7 @@ static struct ib_qp *create_qp(struct ib_device *dev, struct ib_pd *pd,
spin_lock_init(&qp->mr_lock);
xa_init(&qp->comp_cntrs);
+ xa_init_flags(&qp->attached_mrs, XA_FLAGS_ALLOC);
INIT_LIST_HEAD(&qp->rdma_mrs);
INIT_LIST_HEAD(&qp->sig_mrs);
init_completion(&qp->srq_completion);
@@ -1385,6 +1386,7 @@ static struct ib_qp *create_qp(struct ib_device *dev, struct ib_pd *pd,
err_create:
rdma_restrack_put(&qp->res);
xa_destroy(&qp->comp_cntrs);
+ xa_destroy(&qp->attached_mrs);
kfree(qp);
return ERR_PTR(ret);
@@ -2212,6 +2214,7 @@ int ib_destroy_qp_user(struct ib_qp *qp, struct ib_udata *udata)
struct ib_qp_security *sec;
struct ib_comp_cntr *cc;
unsigned long index;
+ struct ib_mr *mr;
int ret;
WARN_ON_ONCE(qp->mrs_used > 0);
@@ -2249,6 +2252,13 @@ int ib_destroy_qp_user(struct ib_qp *qp, struct ib_udata *udata)
atomic_dec(&cc->usecnt);
xa_destroy(&qp->comp_cntrs);
+ /* the driver ended the bindings with the QP, release the regions */
+ xa_for_each(&qp->attached_mrs, index, mr) {
+ WRITE_ONCE(mr->attached_qp, NULL);
+ atomic_dec(&mr->usecnt);
+ }
+ xa_destroy(&qp->attached_mrs);
+
ib_qp_usecnt_dec(qp);
if (sec)
ib_destroy_qp_security_end(sec);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index f9171b47ac1b..c9ddc4147ef8 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2003,6 +2003,8 @@ struct ib_qp {
struct list_head xrcd_list;
struct xarray comp_cntrs; /* op_mask -> comp_cntr */
u32 comp_cntr_op_mask;
+ /* regions attached to this QP, each held until detach or destroy */
+ struct xarray attached_mrs;
/* count times opened, mcast attaches, flow attaches */
atomic_t usecnt;
@@ -2091,8 +2093,11 @@ struct ib_mr {
struct ib_jkey *jkey;
/* the region this one was derived from, NULL if registered fresh */
struct ib_mr *parent;
- /* regions derived from this one that are still registered */
+ /* regions derived from this one, and an attachment, that hold it */
atomic_t usecnt;
+ /* the QP this region is attached to, NULL if none */
+ struct ib_qp *attached_qp;
+ u32 attach_id; /* index in attached_qp->attached_mrs */
struct ib_dm *dm;
struct ib_sig_attrs *sig_attrs; /* only for IB_MR_TYPE_INTEGRITY MRs */
@@ -2846,6 +2851,9 @@ struct ib_device_ops {
struct ib_mr_ex_attr *attr,
struct ib_udata *udata);
int (*dereg_mr)(struct ib_mr *mr, struct ib_udata *udata);
+ /* attach/detach a region so it is bound to single QP (restricted) */
+ int (*qp_attach_mr)(struct ib_qp *qp, struct ib_mr *mr);
+ int (*qp_detach_mr)(struct ib_qp *qp, struct ib_mr *mr);
int (*query_qp_semantics)(struct ib_device *device,
enum ib_qp_type qp_type,
struct ib_qp_semantics *sem);
@@ -4866,6 +4874,7 @@ static inline void ib_mr_init_core_refs(struct ib_mr *mr)
{
mr->jkey = NULL;
mr->parent = NULL;
+ mr->attached_qp = NULL;
atomic_set(&mr->usecnt, 0);
if (!(mr->device->attrs.device_cap_flags & IB_DEVICE_KEY64)) {
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 51910240a7e1..8359f7afeded 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -199,6 +199,18 @@ enum uverbs_methods_qp {
UVERBS_METHOD_QP_CREATE,
UVERBS_METHOD_QP_DESTROY,
UVERBS_METHOD_QP_ATTACH_COMP_CNTR,
+ UVERBS_METHOD_QP_ATTACH_MR,
+ UVERBS_METHOD_QP_DETACH_MR,
+};
+
+enum uverbs_attrs_qp_attach_mr_cmd_attr_ids {
+ UVERBS_ATTR_QP_ATTACH_MR_QP_HANDLE,
+ UVERBS_ATTR_QP_ATTACH_MR_MR_HANDLE,
+};
+
+enum uverbs_attrs_qp_detach_mr_cmd_attr_ids {
+ UVERBS_ATTR_QP_DETACH_MR_QP_HANDLE,
+ UVERBS_ATTR_QP_DETACH_MR_MR_HANDLE,
};
enum uverbs_attrs_create_srq_cmd_attr_ids {
--
2.56.0
^ permalink raw reply related [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 13/13] RDMA/core: attach a memory region to a queue pair
2026-10-14 22:21 ` Eric Davis
(?)
@ 2026-10-14 22:37 ` sashiko-bot
-1 siblings, 0 replies; 60+ messages in thread
From: sashiko-bot @ 2026-10-14 22:37 UTC (permalink / raw)
To: Eric Davis; +Cc: linux-rdma
> From: Eric Davis <eric.davis@broadcom.com>
>
> A region registered against a protection domain is reachable from every QP
> in it. When a device can narrow that, let a region be attached to one QP
> and detached again, so the network only reaches the MR over that QP.
>
> Which restrictions a device supports is reported through the memory region
> access capability bits.
>
> Signed-off-by: Eric Davis <eric.davis@broadcom.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261014-uet-ib-core-v1-0-a0e626117113@broadcom.com?part=13
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs
2026-10-14 22:21 ` Eric Davis
` (13 preceding siblings ...)
(?)
@ 2026-10-14 23:57 ` Doug Ledford
2026-10-15 0:08 ` Jason Gunthorpe
-1 siblings, 1 reply; 60+ messages in thread
From: Doug Ledford @ 2026-10-14 23:57 UTC (permalink / raw)
To: eric.davis, Jason Gunthorpe, Leon Romanovsky; +Cc: linux-rdma
[-- Attachment #1.1: Type: text/plain, Size: 1346 bytes --]
On 10/14/26 3:21 PM, Eric Davis via B4 Relay wrote:
> Ultra Ethernet defines a reliable unconnected transport. A queue pair
> sends to any peer in its job without a connection, incoming traffic is
> demultiplexed on a JobID rather than on a connection, and what a device
> orders is far weaker than IB or RoCE and varies between devices. None of
> that has a home in the current uverbs ABI.
>
> This series adds the generic half. The objects, methods, and attributes
> any UET device implements. No driver is part of it, though two already
> exist against this ABI. A reference device model that implements all of
> it, and HPE's Slingshot driver implementing a large subset. That is the
> argument for ib_core carrying this. What is device specific should come
> from private interfaces per vendor.
I'm doing some cleanups and then will be sending our Slingshot driver
for initial review as a dependent item on this patchset. Look for that
by the end of this week.
Note: I dropped linux-kernel. Unless things have changed, they usually
don't care too much about the rdma stuff until it's time for the merge
window and that list is already way overloaded as it stands.
--
Doug Ledford <doug.ledford@hpe.com>
GPG KeyID: B826A3330E572FDD
Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs
2026-10-14 23:57 ` [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs Doug Ledford
@ 2026-10-15 0:08 ` Jason Gunthorpe
2026-10-15 1:02 ` Doug Ledford
2026-10-15 16:27 ` Eric Davis
0 siblings, 2 replies; 60+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 0:08 UTC (permalink / raw)
To: Doug Ledford; +Cc: eric.davis, Leon Romanovsky, linux-rdma
On Wed, Oct 14, 2026 at 04:57:16PM -0700, Doug Ledford wrote:
> On 10/14/26 3:21 PM, Eric Davis via B4 Relay wrote:
> > Ultra Ethernet defines a reliable unconnected transport. A queue pair
> > sends to any peer in its job without a connection, incoming traffic is
> > demultiplexed on a JobID rather than on a connection, and what a device
> > orders is far weaker than IB or RoCE and varies between devices. None of
> > that has a home in the current uverbs ABI.
> >
> > This series adds the generic half. The objects, methods, and attributes
> > any UET device implements. No driver is part of it, though two already
> > exist against this ABI. A reference device model that implements all of
> > it, and HPE's Slingshot driver implementing a large subset. That is the
> > argument for ib_core carrying this. What is device specific should come
> > from private interfaces per vendor.
>
> I'm doing some cleanups and then will be sending our Slingshot driver
> for initial review as a dependent item on this patchset. Look for that
> by the end of this week.
You each will give reviewd-by's for each others?
You are running sashiko locally and fixing the bugs it finds?
There are several other new drivers right now, do we need to do
another full driver review trading scheme?
> Note: I dropped linux-kernel. Unless things have changed, they usually
> don't care too much about the rdma stuff until it's time for the merge
> window and that list is already way overloaded as it stands.
Nobody reads that list I'm sure?
Jason
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs
2026-10-15 0:08 ` Jason Gunthorpe
@ 2026-10-15 1:02 ` Doug Ledford
2026-10-15 17:29 ` Jason Gunthorpe
2026-10-15 16:27 ` Eric Davis
1 sibling, 1 reply; 60+ messages in thread
From: Doug Ledford @ 2026-10-15 1:02 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: eric.davis, Leon Romanovsky, linux-rdma
[-- Attachment #1.1: Type: text/plain, Size: 3296 bytes --]
On 10/14/26 5:08 PM, Jason Gunthorpe wrote:
> On Wed, Oct 14, 2026 at 04:57:16PM -0700, Doug Ledford wrote:
>> On 10/14/26 3:21 PM, Eric Davis via B4 Relay wrote:
>>> Ultra Ethernet defines a reliable unconnected transport. A queue pair
>>> sends to any peer in its job without a connection, incoming traffic is
>>> demultiplexed on a JobID rather than on a connection, and what a device
>>> orders is far weaker than IB or RoCE and varies between devices. None of
>>> that has a home in the current uverbs ABI.
>>>
>>> This series adds the generic half. The objects, methods, and attributes
>>> any UET device implements. No driver is part of it, though two already
>>> exist against this ABI. A reference device model that implements all of
>>> it, and HPE's Slingshot driver implementing a large subset. That is the
>>> argument for ib_core carrying this. What is device specific should come
>>> from private interfaces per vendor.
>>
>> I'm doing some cleanups and then will be sending our Slingshot driver
>> for initial review as a dependent item on this patchset. Look for that
>> by the end of this week.
>
> You each will give reviewd-by's for each others?
Absolutely. We've been working with each other's patch series behind
the scenes already.
> You are running sashiko locally and fixing the bugs it finds?
Yes, but with one caveat here at HPE anyway: We have very limited AI
resources we are allowed to use for data exfiltration reasons (yeah, I
know, this is open source and we're intentionally exfiltrating it, but
company policy guides who we have contracts with). We can currently
only use Github CoPilot. We can select models, but those aren't the
same as going elsewhere. I noticed for example that Eric used Gemini
for his, and I think he honestly got better responses than we did.
However, also keep in mind that this is adding a new QP type with
semantics much closer to Portals 4 API than verbs, and that does create
some friction not just with the rest of the stack, but with the AI
reviews judging by what I've already seen from the Sashiko reviews to
this series.
> There are several other new drivers right now, do we need to do
> another full driver review trading scheme?
Probably wouldn't hurt. But, our driver is many more lines over in the
drivers/net/ethernet space than it is in the RDMA space. Unless netdev
community makes me move a bunch of stuff over, we don't plan to move
what we don't have to. There are reasons for it to be there in the
first place (namely that the device can run as a simple Ethernet
controller if someone so chooses, but it actually needs a lot of that
infrastructure the RDMA portion of the driver calls into just to get
simple Ethernet up and running).
>> Note: I dropped linux-kernel. Unless things have changed, they usually
>> don't care too much about the rdma stuff until it's time for the merge
>> window and that list is already way overloaded as it stands.
>
> Nobody reads that list I'm sure?
I dunno...does LWN still do a summary with each report? Maybe only
automated tools deal with it...
--
Doug Ledford <doug.ledford@hpe.com>
GPG KeyID: B826A3330E572FDD
Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread* Re: [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs
2026-10-15 1:02 ` Doug Ledford
@ 2026-10-15 17:29 ` Jason Gunthorpe
2026-10-15 18:22 ` Doug Ledford
0 siblings, 1 reply; 60+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 17:29 UTC (permalink / raw)
To: Doug Ledford; +Cc: eric.davis, Leon Romanovsky, linux-rdma
On Wed, Oct 14, 2026 at 06:02:38PM -0700, Doug Ledford wrote:
> > You are running sashiko locally and fixing the bugs it finds?
>
> Yes, but with one caveat here at HPE anyway: We have very limited AI
> resources we are allowed to use for data exfiltration reasons
Ah, that will be hard for you. You need several runs through sashiko
before you post anything so big. That needs a real AI license. Gemini
or GPT are good choices.
> However, also keep in mind that this is adding a new QP type with
> semantics much closer to Portals 4 API than verbs, and that does create
> some friction not just with the rest of the stack, but with the AI
> reviews judging by what I've already seen from the Sashiko reviews to
> this series.
Lovely. It is a great sign when AI can't even understand the code :\
AI reads the comments, humans read the comments. Same with commit
messages. Try describing a little more what is going on and why
things are OK. It does actually help get better results.
> Probably wouldn't hurt. But, our driver is many more lines over in the
> drivers/net/ethernet space than it is in the RDMA space.
Well, you need to get the netdev driver merged first, with a core
device and aux bus scheme. That means I don't have to look at that for
about 6 months probably.. Yay.
Jason
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs
2026-10-15 17:29 ` Jason Gunthorpe
@ 2026-10-15 18:22 ` Doug Ledford
0 siblings, 0 replies; 60+ messages in thread
From: Doug Ledford @ 2026-10-15 18:22 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: eric.davis, Leon Romanovsky, linux-rdma
[-- Attachment #1.1: Type: text/plain, Size: 1833 bytes --]
On 10/15/26 10:29 AM, Jason Gunthorpe wrote:
> On Wed, Oct 14, 2026 at 06:02:38PM -0700, Doug Ledford wrote:
>
>>> You are running sashiko locally and fixing the bugs it finds?
>>
>> Yes, but with one caveat here at HPE anyway: We have very limited AI
>> resources we are allowed to use for data exfiltration reasons
>
> Ah, that will be hard for you. You need several runs through sashiko
> before you post anything so big. That needs a real AI license. Gemini
> or GPT are good choices.
One of the people on our open source team is running it right now. We
selected the GPT 6.1 Sol model, so hopefully it will do a good enough job.
>> However, also keep in mind that this is adding a new QP type with
>> semantics much closer to Portals 4 API than verbs, and that does create
>> some friction not just with the rest of the stack, but with the AI
>> reviews judging by what I've already seen from the Sashiko reviews to
>> this series.
>
> Lovely. It is a great sign when AI can't even understand the code :\
>
> AI reads the comments, humans read the comments. Same with commit
> messages. Try describing a little more what is going on and why
> things are OK. It does actually help get better results.
Thanks for the tip, will do.
>> Probably wouldn't hurt. But, our driver is many more lines over in the
>> drivers/net/ethernet space than it is in the RDMA space.
>
> Well, you need to get the netdev driver merged first, with a core
> device and aux bus scheme. That means I don't have to look at that for
> about 6 months probably.. Yay.
We did the netdev driver + aux driver model. Hopefully it won't take
that long :-/.
--
Doug Ledford <doug.ledford@hpe.com>
GPG KeyID: B826A3330E572FDD
Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread
* Re: [PATCH rdma-next 00/13] RDMA/core: Ultra Ethernet transport verbs
2026-10-15 0:08 ` Jason Gunthorpe
2026-10-15 1:02 ` Doug Ledford
@ 2026-10-15 16:27 ` Eric Davis
1 sibling, 0 replies; 60+ messages in thread
From: Eric Davis @ 2026-10-15 16:27 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Doug Ledford, Leon Romanovsky, linux-rdma
[-- Attachment #1: Type: text/plain, Size: 540 bytes --]
Hi Jason,
On Wed, Oct 14, 2026 at 5:08 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> You each will give reviewd-by's for each others?
Yes, definitely.
> You are running sashiko locally and fixing the bugs it finds?
Yes. I ran my patch series through a local Sashiko run before posting and
I fixed those findings. It was amazing what it found. I've answered this round
per patch. Two will be fixed for v2 (patches 5 and 6), the rest don't apply to
UET ports or how core handles RU QPs.
Hope this helps.
Thanks,
- eric
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5461 bytes --]
^ permalink raw reply [flat|nested] 60+ messages in thread