* [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5
@ 2025-03-06 11:51 Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API Leon Romanovsky
` (6 more replies)
0 siblings, 7 replies; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-06 11:51 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Chiara Meiohas, Jonathan Corbet, linux-doc, linux-kernel,
linux-rdma, Mark Bloch, Patrisious Haddad, Yishai Hadas
Changelog:
v1:
* Used kref primitives instead of open-coded variant
* Check return value from dev_set_name()
* Added extra brackets around type in UCAP_ENABLED macro
v0: https://lore.kernel.org/all/cover.1740574943.git.leon@kernel.org
--------------------------------------------------------------------------
This series introduces the User CAPability (UCAP) API that allows
creating user contexts with various firmware privileges.
The UCAP API provides fine-grained control over specific firmware
features by representing each capability as a character device with root
read-write access. Root processes can grant users special privileges by
allowing access to these character devices. User contexts created using
a file descriptor of a UCAP will have specific UCAP privileges.
Two UCAP character devices are created for mlx5, and user contexts
opened with at least one of these UCAPs are considered privileged. To
ensure that privileged commands can always proceed, non-privileged
commands are limited when a privileged user is present on the device.
Thanks
Chiara Meiohas (5):
RDMA/uverbs: Introduce UCAP (User CAPabilities) API
RDMA/mlx5: Create UCAP char devices for supported device capabilities
RDMA/uverbs: Add support for UCAPs in context creation
RDMA/mlx5: Check enabled UCAPs when creating ucontext
docs: infiniband: document the UCAP API
Patrisious Haddad (1):
RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
Documentation/infiniband/index.rst | 1 +
Documentation/infiniband/ucaps.rst | 71 +++++
drivers/infiniband/core/Makefile | 3 +-
drivers/infiniband/core/ucaps.c | 267 ++++++++++++++++++
drivers/infiniband/core/uverbs_cmd.c | 19 ++
drivers/infiniband/core/uverbs_main.c | 2 +
.../infiniband/core/uverbs_std_types_device.c | 4 +
drivers/infiniband/hw/mlx5/devx.c | 31 +-
drivers/infiniband/hw/mlx5/devx.h | 5 +-
drivers/infiniband/hw/mlx5/fs.c | 154 +++++++++-
drivers/infiniband/hw/mlx5/fs.h | 2 +
drivers/infiniband/hw/mlx5/main.c | 77 ++++-
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +
include/rdma/ib_ucaps.h | 25 ++
include/rdma/ib_verbs.h | 1 +
include/uapi/rdma/ib_user_ioctl_cmds.h | 1 +
include/uapi/rdma/mlx5_user_ioctl_cmds.h | 1 +
include/uapi/rdma/mlx5_user_ioctl_verbs.h | 2 +
18 files changed, 647 insertions(+), 22 deletions(-)
create mode 100644 Documentation/infiniband/ucaps.rst
create mode 100644 drivers/infiniband/core/ucaps.c
create mode 100644 include/rdma/ib_ucaps.h
--
2.48.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
@ 2025-03-06 11:51 ` Leon Romanovsky
2025-03-08 7:25 ` Zhu Yanjun
2025-03-06 11:51 ` [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities Leon Romanovsky
` (5 subsequent siblings)
6 siblings, 1 reply; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-06 11:51 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Chiara Meiohas, linux-rdma, Yishai Hadas
From: Chiara Meiohas <cmeiohas@nvidia.com>
Implement a new User CAPabilities (UCAP) API to provide fine-grained
control over specific firmware features.
This approach offers more granular capabilities than the existing Linux
capabilities, which may be too generic for certain FW features.
This mechanism represents each capability as a character device with
root read-write access. Root processes can grant users special
privileges by allowing access to these character devices (e.g., using
chown).
UCAP character devices are located in /dev/infiniband and the class path
is /sys/class/infiniband_ucaps.
Signed-off-by: Chiara Meiohas <cmeiohas@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
---
drivers/infiniband/core/Makefile | 3 +-
drivers/infiniband/core/ucaps.c | 267 ++++++++++++++++++++++++++
drivers/infiniband/core/uverbs_main.c | 2 +
include/rdma/ib_ucaps.h | 25 +++
4 files changed, 296 insertions(+), 1 deletion(-)
create mode 100644 drivers/infiniband/core/ucaps.c
create mode 100644 include/rdma/ib_ucaps.h
diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index 8ab4eea5a0a5..d49ded7e95f0 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -39,6 +39,7 @@ ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
uverbs_std_types_async_fd.o \
uverbs_std_types_srq.o \
uverbs_std_types_wq.o \
- uverbs_std_types_qp.o
+ uverbs_std_types_qp.o \
+ ucaps.o
ib_uverbs-$(CONFIG_INFINIBAND_USER_MEM) += umem.o umem_dmabuf.o
ib_uverbs-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o
diff --git a/drivers/infiniband/core/ucaps.c b/drivers/infiniband/core/ucaps.c
new file mode 100644
index 000000000000..6853c6d078f9
--- /dev/null
+++ b/drivers/infiniband/core/ucaps.c
@@ -0,0 +1,267 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#include <linux/kref.h>
+#include <linux/cdev.h>
+#include <linux/mutex.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <rdma/ib_ucaps.h>
+
+#define RDMA_UCAP_FIRST RDMA_UCAP_MLX5_CTRL_LOCAL
+
+static DEFINE_MUTEX(ucaps_mutex);
+static struct ib_ucap *ucaps_list[RDMA_UCAP_MAX];
+static bool ucaps_class_is_registered;
+static dev_t ucaps_base_dev;
+
+struct ib_ucap {
+ struct cdev cdev;
+ struct device dev;
+ struct kref ref;
+};
+
+static const char *ucap_names[RDMA_UCAP_MAX] = {
+ [RDMA_UCAP_MLX5_CTRL_LOCAL] = "mlx5_perm_ctrl_local",
+ [RDMA_UCAP_MLX5_CTRL_OTHER_VHCA] = "mlx5_perm_ctrl_other_vhca"
+};
+
+static char *ucaps_devnode(const struct device *dev, umode_t *mode)
+{
+ if (mode)
+ *mode = 0600;
+
+ return kasprintf(GFP_KERNEL, "infiniband/%s", dev_name(dev));
+}
+
+static const struct class ucaps_class = {
+ .name = "infiniband_ucaps",
+ .devnode = ucaps_devnode,
+};
+
+static const struct file_operations ucaps_cdev_fops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+};
+
+/**
+ * ib_cleanup_ucaps - cleanup all API resources and class.
+ *
+ * This is called once, when removing the ib_uverbs module.
+ */
+void ib_cleanup_ucaps(void)
+{
+ mutex_lock(&ucaps_mutex);
+ if (!ucaps_class_is_registered) {
+ mutex_unlock(&ucaps_mutex);
+ return;
+ }
+
+ for (int i = RDMA_UCAP_FIRST; i < RDMA_UCAP_MAX; i++)
+ WARN_ON(ucaps_list[i]);
+
+ class_unregister(&ucaps_class);
+ ucaps_class_is_registered = false;
+ unregister_chrdev_region(ucaps_base_dev, RDMA_UCAP_MAX);
+ mutex_unlock(&ucaps_mutex);
+}
+
+static int get_ucap_from_devt(dev_t devt, u64 *idx_mask)
+{
+ for (int type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
+ if (ucaps_list[type] && ucaps_list[type]->dev.devt == devt) {
+ *idx_mask |= 1 << type;
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+
+static int get_devt_from_fd(unsigned int fd, dev_t *ret_dev)
+{
+ struct file *file;
+
+ file = fget(fd);
+ if (!file)
+ return -EBADF;
+
+ *ret_dev = file_inode(file)->i_rdev;
+ fput(file);
+ return 0;
+}
+
+/**
+ * ib_ucaps_init - Initialization required before ucap creation.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+static int ib_ucaps_init(void)
+{
+ int ret = 0;
+
+ if (ucaps_class_is_registered)
+ return ret;
+
+ ret = class_register(&ucaps_class);
+ if (ret)
+ return ret;
+
+ ret = alloc_chrdev_region(&ucaps_base_dev, 0, RDMA_UCAP_MAX,
+ ucaps_class.name);
+ if (ret < 0) {
+ class_unregister(&ucaps_class);
+ return ret;
+ }
+
+ ucaps_class_is_registered = true;
+
+ return 0;
+}
+
+static void ucap_dev_release(struct device *device)
+{
+ struct ib_ucap *ucap = container_of(device, struct ib_ucap, dev);
+
+ kfree(ucap);
+}
+
+/**
+ * ib_create_ucap - Add a ucap character device
+ * @type: UCAP type
+ *
+ * Creates a ucap character device in the /dev/infiniband directory. By default,
+ * the device has root-only read-write access.
+ *
+ * A driver may call this multiple times with the same UCAP type. A reference
+ * count tracks creations and deletions.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+int ib_create_ucap(enum rdma_user_cap type)
+{
+ struct ib_ucap *ucap;
+ int ret;
+
+ if (type >= RDMA_UCAP_MAX)
+ return -EINVAL;
+
+ mutex_lock(&ucaps_mutex);
+ ret = ib_ucaps_init();
+ if (ret)
+ goto unlock;
+
+ ucap = ucaps_list[type];
+ if (ucap) {
+ kref_get(&ucap->ref);
+ mutex_unlock(&ucaps_mutex);
+ return 0;
+ }
+
+ ucap = kzalloc(sizeof(*ucap), GFP_KERNEL);
+ if (!ucap) {
+ ret = -ENOMEM;
+ goto unlock;
+ }
+
+ device_initialize(&ucap->dev);
+ ucap->dev.class = &ucaps_class;
+ ucap->dev.devt = MKDEV(MAJOR(ucaps_base_dev), type);
+ ucap->dev.release = ucap_dev_release;
+ ret = dev_set_name(&ucap->dev, ucap_names[type]);
+ if (ret)
+ goto err_device;
+
+ cdev_init(&ucap->cdev, &ucaps_cdev_fops);
+ ucap->cdev.owner = THIS_MODULE;
+
+ ret = cdev_device_add(&ucap->cdev, &ucap->dev);
+ if (ret)
+ goto err_device;
+
+ kref_init(&ucap->ref);
+ ucaps_list[type] = ucap;
+ mutex_unlock(&ucaps_mutex);
+
+ return 0;
+
+err_device:
+ put_device(&ucap->dev);
+unlock:
+ mutex_unlock(&ucaps_mutex);
+ return ret;
+}
+EXPORT_SYMBOL(ib_create_ucap);
+
+static void ib_release_ucap(struct kref *ref)
+{
+ struct ib_ucap *ucap = container_of(ref, struct ib_ucap, ref);
+ enum rdma_user_cap type;
+
+ for (type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
+ if (ucaps_list[type] == ucap)
+ break;
+ }
+ WARN_ON(type == RDMA_UCAP_MAX);
+
+ ucaps_list[type] = NULL;
+ cdev_device_del(&ucap->cdev, &ucap->dev);
+ put_device(&ucap->dev);
+}
+
+/**
+ * ib_remove_ucap - Remove a ucap character device
+ * @type: User cap type
+ *
+ * Removes the ucap character device according to type. The device is completely
+ * removed from the filesystem when its reference count reaches 0.
+ */
+void ib_remove_ucap(enum rdma_user_cap type)
+{
+ struct ib_ucap *ucap;
+
+ mutex_lock(&ucaps_mutex);
+ ucap = ucaps_list[type];
+ if (WARN_ON(!ucap))
+ goto end;
+
+ kref_put(&ucap->ref, ib_release_ucap);
+end:
+ mutex_unlock(&ucaps_mutex);
+}
+EXPORT_SYMBOL(ib_remove_ucap);
+
+/**
+ * ib_get_ucaps - Get bitmask of ucap types from file descriptors
+ * @fds: Array of file descriptors
+ * @fd_count: Number of file descriptors in the array
+ * @idx_mask: Bitmask to be updated based on the ucaps in the fd list
+ *
+ * Given an array of file descriptors, this function returns a bitmask of
+ * the ucaps where a bit is set if an FD for that ucap type was in the array.
+ *
+ * Return: 0 on success, or a negative errno value on failure
+ */
+int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask)
+{
+ int ret = 0;
+ dev_t dev;
+
+ *idx_mask = 0;
+ mutex_lock(&ucaps_mutex);
+ for (int i = 0; i < fd_count; i++) {
+ ret = get_devt_from_fd(fds[i], &dev);
+ if (ret)
+ goto end;
+
+ ret = get_ucap_from_devt(dev, idx_mask);
+ if (ret)
+ goto end;
+ }
+
+end:
+ mutex_unlock(&ucaps_mutex);
+ return ret;
+}
diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
index 85cfc790a7bb..973fe2c7ef53 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -52,6 +52,7 @@
#include <rdma/ib.h>
#include <rdma/uverbs_std_types.h>
#include <rdma/rdma_netlink.h>
+#include <rdma/ib_ucaps.h>
#include "uverbs.h"
#include "core_priv.h"
@@ -1345,6 +1346,7 @@ static void __exit ib_uverbs_cleanup(void)
IB_UVERBS_NUM_FIXED_MINOR);
unregister_chrdev_region(dynamic_uverbs_dev,
IB_UVERBS_NUM_DYNAMIC_MINOR);
+ ib_cleanup_ucaps();
mmu_notifier_synchronize();
}
diff --git a/include/rdma/ib_ucaps.h b/include/rdma/ib_ucaps.h
new file mode 100644
index 000000000000..8f0552a2b2b0
--- /dev/null
+++ b/include/rdma/ib_ucaps.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#ifndef _IB_UCAPS_H_
+#define _IB_UCAPS_H_
+
+#define UCAP_ENABLED(ucaps, type) (!!((ucaps) & (1U << (type))))
+
+enum rdma_user_cap {
+ RDMA_UCAP_MLX5_CTRL_LOCAL,
+ RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
+ RDMA_UCAP_MAX
+};
+
+void ib_cleanup_ucaps(void);
+
+int ib_create_ucap(enum rdma_user_cap type);
+
+void ib_remove_ucap(enum rdma_user_cap type);
+
+int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask);
+
+#endif /* _IB_UCAPS_H_ */
--
2.48.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API Leon Romanovsky
@ 2025-03-06 11:51 ` Leon Romanovsky
2025-03-08 12:33 ` kernel test robot
2025-03-06 11:51 ` [PATCH rdma-next v1 3/6] RDMA/uverbs: Add support for UCAPs in context creation Leon Romanovsky
` (4 subsequent siblings)
6 siblings, 1 reply; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-06 11:51 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Chiara Meiohas, linux-rdma, Yishai Hadas
From: Chiara Meiohas <cmeiohas@nvidia.com>
Create UCAP character devices when probing an IB device with supported
firmware capabilities.
If the RDMA_CTRL general object type is supported, check for specific
UCTX capabilities:
Create /dev/infiniband/mlx5_perm_ctrl_local for RDMA_UCAP_MLX5_CTRL_LOCAL
Create /dev/infiniband/mlx5_perm_ctrl_other_vhca for RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
Signed-off-by: Chiara Meiohas <cmeiohas@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
---
drivers/infiniband/hw/mlx5/main.c | 47 +++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 81849eb671a1..04b489a6a449 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -47,6 +47,7 @@
#include <rdma/uverbs_ioctl.h>
#include <rdma/mlx5_user_ioctl_verbs.h>
#include <rdma/mlx5_user_ioctl_cmds.h>
+#include <rdma/ib_ucaps.h>
#include "macsec.h"
#include "data_direct.h"
@@ -4201,8 +4202,47 @@ static int mlx5_ib_init_var_table(struct mlx5_ib_dev *dev)
return (var_table->bitmap) ? 0 : -ENOMEM;
}
+static void mlx5_ib_cleanup_ucaps(struct mlx5_ib_dev *dev)
+{
+ if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
+ ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
+
+ if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
+ MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
+ ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+}
+
+static int mlx5_ib_init_ucaps(struct mlx5_ib_dev *dev)
+{
+ int ret;
+
+ if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
+ ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
+ if (ret)
+ return ret;
+ }
+
+ if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
+ MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
+ ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+ if (ret)
+ goto remove_local;
+ }
+
+ return 0;
+
+remove_local:
+ if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
+ ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
+ return ret;
+}
+
static void mlx5_ib_stage_caps_cleanup(struct mlx5_ib_dev *dev)
{
+ if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
+ MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)
+ mlx5_ib_cleanup_ucaps(dev);
+
bitmap_free(dev->var_table.bitmap);
}
@@ -4253,6 +4293,13 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev)
return err;
}
+ if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
+ MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL) {
+ err = mlx5_ib_init_ucaps(dev);
+ if (err)
+ return err;
+ }
+
dev->ib_dev.use_cq_dim = true;
return 0;
--
2.48.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH rdma-next v1 3/6] RDMA/uverbs: Add support for UCAPs in context creation
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities Leon Romanovsky
@ 2025-03-06 11:51 ` Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext Leon Romanovsky
` (3 subsequent siblings)
6 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-06 11:51 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Chiara Meiohas, linux-rdma, Yishai Hadas
From: Chiara Meiohas <cmeiohas@nvidia.com>
Add support for file descriptor array attribute for GET_CONTEXT
commands.
Check that the file descriptor (fd) array represents fds for valid UCAPs.
Store the enabled UCAPs from the fd array as a bitmask in ib_ucontext.
Signed-off-by: Chiara Meiohas <cmeiohas@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
---
drivers/infiniband/core/uverbs_cmd.c | 19 +++++++++++++++++++
.../infiniband/core/uverbs_std_types_device.c | 4 ++++
include/rdma/ib_verbs.h | 1 +
include/uapi/rdma/ib_user_ioctl_cmds.h | 1 +
4 files changed, 25 insertions(+)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 5ad14c39d48c..96d639e1ffa0 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -42,6 +42,7 @@
#include <rdma/uverbs_types.h>
#include <rdma/uverbs_std_types.h>
+#include <rdma/ib_ucaps.h>
#include "rdma_core.h"
#include "uverbs.h"
@@ -232,6 +233,8 @@ int ib_init_ucontext(struct uverbs_attr_bundle *attrs)
{
struct ib_ucontext *ucontext = attrs->context;
struct ib_uverbs_file *file = attrs->ufile;
+ int *fd_array;
+ int fd_count;
int ret;
if (!down_read_trylock(&file->hw_destroy_rwsem))
@@ -247,6 +250,22 @@ int ib_init_ucontext(struct uverbs_attr_bundle *attrs)
if (ret)
goto err;
+ if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_GET_CONTEXT_FD_ARR)) {
+ fd_count = uverbs_attr_ptr_get_array_size(attrs,
+ UVERBS_ATTR_GET_CONTEXT_FD_ARR,
+ sizeof(int));
+ if (fd_count < 0) {
+ ret = fd_count;
+ goto err_uncharge;
+ }
+
+ fd_array = uverbs_attr_get_alloced_ptr(attrs,
+ UVERBS_ATTR_GET_CONTEXT_FD_ARR);
+ ret = ib_get_ucaps(fd_array, fd_count, &ucontext->enabled_caps);
+ if (ret)
+ goto err_uncharge;
+ }
+
ret = ucontext->device->ops.alloc_ucontext(ucontext,
&attrs->driver_udata);
if (ret)
diff --git a/drivers/infiniband/core/uverbs_std_types_device.c b/drivers/infiniband/core/uverbs_std_types_device.c
index fb0555647336..c0fd283d9d6c 100644
--- a/drivers/infiniband/core/uverbs_std_types_device.c
+++ b/drivers/infiniband/core/uverbs_std_types_device.c
@@ -437,6 +437,10 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_TYPE(u32), UA_OPTIONAL),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
UVERBS_ATTR_TYPE(u64), UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(UVERBS_ATTR_GET_CONTEXT_FD_ARR,
+ UVERBS_ATTR_MIN_SIZE(sizeof(int)),
+ UA_OPTIONAL,
+ UA_ALLOC_AND_COPY),
UVERBS_ATTR_UHW());
DECLARE_UVERBS_NAMED_METHOD(
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index a5761038935d..9941f4185c79 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1530,6 +1530,7 @@ struct ib_ucontext {
struct ib_uverbs_file *ufile;
struct ib_rdmacg_object cg_obj;
+ u64 enabled_caps;
/*
* Implementation details of the RDMA core, don't use in drivers:
*/
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index ec719053aab9..ac7b162611ed 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -88,6 +88,7 @@ enum uverbs_attrs_query_port_cmd_attr_ids {
enum uverbs_attrs_get_context_attr_ids {
UVERBS_ATTR_GET_CONTEXT_NUM_COMP_VECTORS,
UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
+ UVERBS_ATTR_GET_CONTEXT_FD_ARR,
};
enum uverbs_attrs_query_context_attr_ids {
--
2.48.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
` (2 preceding siblings ...)
2025-03-06 11:51 ` [PATCH rdma-next v1 3/6] RDMA/uverbs: Add support for UCAPs in context creation Leon Romanovsky
@ 2025-03-06 11:51 ` Leon Romanovsky
2025-03-08 4:22 ` kernel test robot
2025-03-06 11:51 ` [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace Leon Romanovsky
` (2 subsequent siblings)
6 siblings, 1 reply; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-06 11:51 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Chiara Meiohas, linux-rdma, Yishai Hadas
From: Chiara Meiohas <cmeiohas@nvidia.com>
Verify that the enabled UCAPs are supported by the device before
creating the ucontext.
If supported, create the ucontext with the associated capabilities.
Store the privileged ucontext UID on creation and remove it when
destroying the privileged ucontext. This allows the command interface
to recognize privileged commands through its UID.
Signed-off-by: Chiara Meiohas <cmeiohas@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
---
drivers/infiniband/hw/mlx5/devx.c | 31 +++++++++++++++++++++++++++++--
drivers/infiniband/hw/mlx5/devx.h | 5 +++--
drivers/infiniband/hw/mlx5/main.c | 30 ++++++++++++++++++++++++++----
3 files changed, 58 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index 39304cae5b10..2479da8620ca 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -13,6 +13,7 @@
#include <rdma/uverbs_std_types.h>
#include <linux/mlx5/driver.h>
#include <linux/mlx5/fs.h>
+#include <rdma/ib_ucaps.h>
#include "mlx5_ib.h"
#include "devx.h"
#include "qp.h"
@@ -122,7 +123,27 @@ devx_ufile2uctx(const struct uverbs_attr_bundle *attrs)
return to_mucontext(ib_uverbs_get_ucontext(attrs));
}
-int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user)
+static int set_uctx_ucaps(struct mlx5_ib_dev *dev, u64 req_ucaps, u32 *cap)
+{
+ if (UCAP_ENABLED(req_ucaps, RDMA_UCAP_MLX5_CTRL_LOCAL)) {
+ if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
+ *cap |= MLX5_UCTX_CAP_RDMA_CTRL;
+ else
+ return -EOPNOTSUPP;
+ }
+
+ if (UCAP_ENABLED(req_ucaps, RDMA_UCAP_MLX5_CTRL_OTHER_VHCA)) {
+ if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
+ MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
+ *cap |= MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA;
+ else
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user, u64 req_ucaps)
{
u32 in[MLX5_ST_SZ_DW(create_uctx_in)] = {};
u32 out[MLX5_ST_SZ_DW(create_uctx_out)] = {};
@@ -146,6 +167,12 @@ int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user)
capable(CAP_SYS_RAWIO))
cap |= MLX5_UCTX_CAP_INTERNAL_DEV_RES;
+ if (req_ucaps) {
+ err = set_uctx_ucaps(dev, req_ucaps, &cap);
+ if (err)
+ return err;
+ }
+
MLX5_SET(create_uctx_in, in, opcode, MLX5_CMD_OP_CREATE_UCTX);
MLX5_SET(uctx, uctx, cap, cap);
@@ -2575,7 +2602,7 @@ int mlx5_ib_devx_init(struct mlx5_ib_dev *dev)
struct mlx5_devx_event_table *table = &dev->devx_event_table;
int uid;
- uid = mlx5_ib_devx_create(dev, false);
+ uid = mlx5_ib_devx_create(dev, false, 0);
if (uid > 0) {
dev->devx_whitelist_uid = uid;
xa_init(&table->event_xa);
diff --git a/drivers/infiniband/hw/mlx5/devx.h b/drivers/infiniband/hw/mlx5/devx.h
index 1344bf4c9d21..ee9e7d3af93f 100644
--- a/drivers/infiniband/hw/mlx5/devx.h
+++ b/drivers/infiniband/hw/mlx5/devx.h
@@ -24,13 +24,14 @@ struct devx_obj {
struct list_head event_sub; /* holds devx_event_subscription entries */
};
#if IS_ENABLED(CONFIG_INFINIBAND_USER_ACCESS)
-int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user);
+int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user, u64 req_ucaps);
void mlx5_ib_devx_destroy(struct mlx5_ib_dev *dev, u16 uid);
int mlx5_ib_devx_init(struct mlx5_ib_dev *dev);
void mlx5_ib_devx_cleanup(struct mlx5_ib_dev *dev);
void mlx5_ib_ufile_hw_cleanup(struct ib_uverbs_file *ufile);
#else
-static inline int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user)
+static inline int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user,
+ u64 req_ucaps)
{
return -EOPNOTSUPP;
}
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 04b489a6a449..d07cacaa0abd 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -1935,6 +1935,12 @@ static int set_ucontext_resp(struct ib_ucontext *uctx,
return 0;
}
+static bool uctx_rdma_ctrl_is_enabled(u64 enabled_caps)
+{
+ return UCAP_ENABLED(enabled_caps, RDMA_UCAP_MLX5_CTRL_LOCAL) ||
+ UCAP_ENABLED(enabled_caps, RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+}
+
static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
struct ib_udata *udata)
{
@@ -1977,10 +1983,17 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
return -EINVAL;
if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX) {
- err = mlx5_ib_devx_create(dev, true);
+ err = mlx5_ib_devx_create(dev, true, uctx->enabled_caps);
if (err < 0)
goto out_ctx;
context->devx_uid = err;
+
+ if (uctx_rdma_ctrl_is_enabled(uctx->enabled_caps)) {
+ err = mlx5_cmd_add_privileged_uid(dev->mdev,
+ context->devx_uid);
+ if (err)
+ goto out_devx;
+ }
}
lib_uar_4k = req.lib_caps & MLX5_LIB_CAP_4K_UAR;
@@ -1995,7 +2008,7 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
/* updates req->total_num_bfregs */
err = calc_total_bfregs(dev, lib_uar_4k, &req, bfregi);
if (err)
- goto out_devx;
+ goto out_ucap;
mutex_init(&bfregi->lock);
bfregi->lib_uar_4k = lib_uar_4k;
@@ -2003,7 +2016,7 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
GFP_KERNEL);
if (!bfregi->count) {
err = -ENOMEM;
- goto out_devx;
+ goto out_ucap;
}
bfregi->sys_pages = kcalloc(bfregi->num_sys_pages,
@@ -2067,6 +2080,11 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
out_count:
kfree(bfregi->count);
+out_ucap:
+ if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX &&
+ uctx_rdma_ctrl_is_enabled(uctx->enabled_caps))
+ mlx5_cmd_remove_privileged_uid(dev->mdev, context->devx_uid);
+
out_devx:
if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX)
mlx5_ib_devx_destroy(dev, context->devx_uid);
@@ -2111,8 +2129,12 @@ static void mlx5_ib_dealloc_ucontext(struct ib_ucontext *ibcontext)
kfree(bfregi->sys_pages);
kfree(bfregi->count);
- if (context->devx_uid)
+ if (context->devx_uid) {
+ if (uctx_rdma_ctrl_is_enabled(ibcontext->enabled_caps))
+ mlx5_cmd_remove_privileged_uid(dev->mdev,
+ context->devx_uid);
mlx5_ib_devx_destroy(dev, context->devx_uid);
+ }
}
static phys_addr_t uar_index2pfn(struct mlx5_ib_dev *dev,
--
2.48.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
` (3 preceding siblings ...)
2025-03-06 11:51 ` [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext Leon Romanovsky
@ 2025-03-06 11:51 ` Leon Romanovsky
2025-03-07 12:30 ` kernel test robot
2025-03-08 5:37 ` kernel test robot
2025-03-06 11:51 ` [PATCH rdma-next v1 6/6] docs: infiniband: document the UCAP API Leon Romanovsky
2025-03-08 19:23 ` [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
6 siblings, 2 replies; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-06 11:51 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Patrisious Haddad, linux-rdma, Mark Bloch
From: Patrisious Haddad <phaddad@nvidia.com>
This patch adds RDMA_TRANSPORT_RX and RDMA_TRANSPORT_TX as a new flow
table type for matcher creation.
Signed-off-by: Patrisious Haddad <phaddad@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
drivers/infiniband/hw/mlx5/fs.c | 154 ++++++++++++++++++++--
drivers/infiniband/hw/mlx5/fs.h | 2 +
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +
include/uapi/rdma/mlx5_user_ioctl_cmds.h | 1 +
include/uapi/rdma/mlx5_user_ioctl_verbs.h | 2 +
5 files changed, 149 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/fs.c b/drivers/infiniband/hw/mlx5/fs.c
index 162814ae8cb4..6ae2801fa13f 100644
--- a/drivers/infiniband/hw/mlx5/fs.c
+++ b/drivers/infiniband/hw/mlx5/fs.c
@@ -12,6 +12,7 @@
#include <rdma/mlx5_user_ioctl_verbs.h>
#include <rdma/ib_hdrs.h>
#include <rdma/ib_umem.h>
+#include <rdma/ib_ucaps.h>
#include <linux/mlx5/driver.h>
#include <linux/mlx5/fs.h>
#include <linux/mlx5/fs_helpers.h>
@@ -690,7 +691,7 @@ static struct mlx5_ib_flow_prio *_get_prio(struct mlx5_ib_dev *dev,
struct mlx5_ib_flow_prio *prio,
int priority,
int num_entries, int num_groups,
- u32 flags)
+ u32 flags, u16 vport)
{
struct mlx5_flow_table_attr ft_attr = {};
struct mlx5_flow_table *ft;
@@ -698,6 +699,7 @@ static struct mlx5_ib_flow_prio *_get_prio(struct mlx5_ib_dev *dev,
ft_attr.prio = priority;
ft_attr.max_fte = num_entries;
ft_attr.flags = flags;
+ ft_attr.vport = vport;
ft_attr.autogroup.max_num_groups = num_groups;
ft = mlx5_create_auto_grouped_flow_table(ns, &ft_attr);
if (IS_ERR(ft))
@@ -792,7 +794,7 @@ static struct mlx5_ib_flow_prio *get_flow_table(struct mlx5_ib_dev *dev,
ft = prio->flow_table;
if (!ft)
return _get_prio(dev, ns, prio, priority, max_table_size,
- num_groups, flags);
+ num_groups, flags, 0);
return prio;
}
@@ -935,7 +937,7 @@ int mlx5_ib_fs_add_op_fc(struct mlx5_ib_dev *dev, u32 port_num,
prio = &dev->flow_db->opfcs[type];
if (!prio->flow_table) {
prio = _get_prio(dev, ns, prio, priority,
- dev->num_ports * MAX_OPFC_RULES, 1, 0);
+ dev->num_ports * MAX_OPFC_RULES, 1, 0, 0);
if (IS_ERR(prio)) {
err = PTR_ERR(prio);
goto free;
@@ -1413,17 +1415,51 @@ static struct ib_flow *mlx5_ib_create_flow(struct ib_qp *qp,
return ERR_PTR(err);
}
+static int mlx5_ib_fill_transport_ns_info(struct mlx5_ib_dev *dev,
+ enum mlx5_flow_namespace_type type,
+ u32 *flags, u16 *vport_idx,
+ u16 *vport,
+ struct mlx5_core_dev **ft_mdev,
+ u32 ib_port)
+{
+ struct mlx5_core_dev *esw_mdev;
+
+ if (!is_mdev_switchdev_mode(dev->mdev))
+ return 0;
+
+ if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
+ return -EOPNOTSUPP;
+
+ if (!dev->port[ib_port - 1].rep)
+ return -EINVAL;
+
+ esw_mdev = mlx5_eswitch_get_core_dev(dev->port[ib_port - 1].rep->esw);
+ if (esw_mdev != dev->mdev)
+ return -EOPNOTSUPP;
+
+ *flags |= MLX5_FLOW_TABLE_OTHER_VPORT;
+ *ft_mdev = esw_mdev;
+ *vport = dev->port[ib_port - 1].rep->vport;
+ *vport_idx = dev->port[ib_port - 1].rep->vport_index;
+
+ return 0;
+}
+
static struct mlx5_ib_flow_prio *
_get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
enum mlx5_flow_namespace_type ns_type,
- bool mcast)
+ bool mcast, u32 ib_port)
{
+ struct mlx5_core_dev *ft_mdev = dev->mdev;
struct mlx5_flow_namespace *ns = NULL;
struct mlx5_ib_flow_prio *prio = NULL;
int max_table_size = 0;
+ u16 vport_idx = 0;
bool esw_encap;
u32 flags = 0;
+ u16 vport = 0;
int priority;
+ int ret;
if (mcast)
priority = MLX5_IB_FLOW_MCAST_PRIO;
@@ -1471,13 +1507,38 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
MLX5_CAP_FLOWTABLE_RDMA_TX(dev->mdev, log_max_ft_size));
priority = user_priority;
break;
+ case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
+ case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
+ if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
+ return ERR_PTR(-EINVAL);
+ ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
+ &vport_idx, &vport,
+ &ft_mdev, ib_port);
+ if (ret)
+ return ERR_PTR(ret);
+
+ if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
+ max_table_size =
+ BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
+ ft_mdev, log_max_ft_size));
+ else
+ max_table_size =
+ BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
+ ft_mdev, log_max_ft_size));
+ priority = user_priority;
+ break;
default:
break;
}
max_table_size = min_t(int, max_table_size, MLX5_FS_MAX_ENTRIES);
- ns = mlx5_get_flow_namespace(dev->mdev, ns_type);
+ if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
+ ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)
+ ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
+ else
+ ns = mlx5_get_flow_namespace(ft_mdev, ns_type);
+
if (!ns)
return ERR_PTR(-EOPNOTSUPP);
@@ -1497,6 +1558,12 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
case MLX5_FLOW_NAMESPACE_RDMA_TX:
prio = &dev->flow_db->rdma_tx[priority];
break;
+ case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
+ prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];
+ break;
+ case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
+ prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];
+ break;
default: return ERR_PTR(-EINVAL);
}
@@ -1507,7 +1574,7 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
return prio;
return _get_prio(dev, ns, prio, priority, max_table_size,
- MLX5_FS_MAX_TYPES, flags);
+ MLX5_FS_MAX_TYPES, flags, vport);
}
static struct mlx5_ib_flow_handler *
@@ -1626,7 +1693,8 @@ static struct mlx5_ib_flow_handler *raw_fs_rule_add(
mutex_lock(&dev->flow_db->lock);
ft_prio = _get_flow_table(dev, fs_matcher->priority,
- fs_matcher->ns_type, mcast);
+ fs_matcher->ns_type, mcast,
+ fs_matcher->ib_port);
if (IS_ERR(ft_prio)) {
err = PTR_ERR(ft_prio);
goto unlock;
@@ -1742,6 +1810,12 @@ mlx5_ib_ft_type_to_namespace(enum mlx5_ib_uapi_flow_table_type table_type,
case MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TX:
*namespace = MLX5_FLOW_NAMESPACE_RDMA_TX;
break;
+ case MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_RX:
+ *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX;
+ break;
+ case MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_TX:
+ *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX;
+ break;
default:
return -EINVAL;
}
@@ -1831,7 +1905,8 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
return -EINVAL;
/* Allow only DEVX object or QP as dest when inserting to RDMA_RX */
- if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX) &&
+ if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX ||
+ fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX) &&
((!dest_devx && !dest_qp) || (dest_devx && dest_qp)))
return -EINVAL;
@@ -1848,7 +1923,8 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
return -EINVAL;
/* Allow only flow table as dest when inserting to FDB or RDMA_RX */
if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_FDB_BYPASS ||
- fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX) &&
+ fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_RX ||
+ fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX) &&
*dest_type != MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE)
return -EINVAL;
} else if (dest_qp) {
@@ -1869,14 +1945,16 @@ static int get_dests(struct uverbs_attr_bundle *attrs,
*dest_id = mqp->raw_packet_qp.rq.tirn;
*dest_type = MLX5_FLOW_DESTINATION_TYPE_TIR;
} else if ((fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_EGRESS ||
- fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX) &&
+ fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX ||
+ fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) &&
!(*flags & MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP)) {
*dest_type = MLX5_FLOW_DESTINATION_TYPE_PORT;
}
if (*dest_type == MLX5_FLOW_DESTINATION_TYPE_TIR &&
(fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_EGRESS ||
- fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX))
+ fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TX ||
+ fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX))
return -EINVAL;
return 0;
@@ -2353,6 +2431,15 @@ static int mlx5_ib_matcher_ns(struct uverbs_attr_bundle *attrs,
return 0;
}
+static bool verify_context_caps(struct mlx5_ib_dev *dev, u64 enabled_caps)
+{
+ if (is_mdev_switchdev_mode(dev->mdev))
+ return UCAP_ENABLED(enabled_caps,
+ RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
+
+ return UCAP_ENABLED(enabled_caps, RDMA_UCAP_MLX5_CTRL_LOCAL);
+}
+
static int UVERBS_HANDLER(MLX5_IB_METHOD_FLOW_MATCHER_CREATE)(
struct uverbs_attr_bundle *attrs)
{
@@ -2401,6 +2488,26 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_FLOW_MATCHER_CREATE)(
goto end;
}
+ if (uverbs_attr_is_valid(attrs, MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT)) {
+ err = uverbs_copy_from(&obj->ib_port, attrs,
+ MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT);
+ if (err)
+ goto end;
+ if (!rdma_is_port_valid(&dev->ib_dev, obj->ib_port)) {
+ err = -EINVAL;
+ goto end;
+ }
+ if (obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX &&
+ obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) {
+ err = -EINVAL;
+ goto end;
+ }
+ if (!verify_context_caps(dev, uobj->context->enabled_caps)) {
+ err = -EOPNOTSUPP;
+ goto end;
+ }
+ }
+
uobj->object = obj;
obj->mdev = dev->mdev;
atomic_set(&obj->usecnt, 0);
@@ -2448,7 +2555,7 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_STEERING_ANCHOR_CREATE)(
mutex_lock(&dev->flow_db->lock);
- ft_prio = _get_flow_table(dev, priority, ns_type, 0);
+ ft_prio = _get_flow_table(dev, priority, ns_type, 0, 0);
if (IS_ERR(ft_prio)) {
err = PTR_ERR(ft_prio);
goto free_obj;
@@ -2834,7 +2941,10 @@ DECLARE_UVERBS_NAMED_METHOD(
UA_OPTIONAL),
UVERBS_ATTR_CONST_IN(MLX5_IB_ATTR_FLOW_MATCHER_FT_TYPE,
enum mlx5_ib_uapi_flow_table_type,
- UA_OPTIONAL));
+ UA_OPTIONAL),
+ UVERBS_ATTR_PTR_IN(MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT,
+ UVERBS_ATTR_TYPE(u32),
+ UA_OPTIONAL));
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
MLX5_IB_METHOD_FLOW_MATCHER_DESTROY,
@@ -2904,8 +3014,26 @@ int mlx5_ib_fs_init(struct mlx5_ib_dev *dev)
if (!dev->flow_db)
return -ENOMEM;
+ dev->flow_db->rdma_transport_rx = kcalloc(dev->num_ports,
+ sizeof(struct mlx5_ib_flow_prio),
+ GFP_KERNEL);
+ if (!dev->flow_db->rdma_transport_rx)
+ goto free_flow_db;
+
+ dev->flow_db->rdma_transport_tx = kcalloc(dev->num_ports,
+ sizeof(struct mlx5_ib_flow_prio),
+ GFP_KERNEL);
+ if (!dev->flow_db->rdma_transport_tx)
+ goto free_rdma_transport_rx;
+
mutex_init(&dev->flow_db->lock);
ib_set_device_ops(&dev->ib_dev, &flow_ops);
return 0;
+
+free_rdma_transport_rx:
+ kfree(dev->flow_db->rdma_transport_rx);
+free_flow_db:
+ kfree(dev->flow_db);
+ return -ENOMEM;
}
diff --git a/drivers/infiniband/hw/mlx5/fs.h b/drivers/infiniband/hw/mlx5/fs.h
index b9734904f5f0..0516555eb1c1 100644
--- a/drivers/infiniband/hw/mlx5/fs.h
+++ b/drivers/infiniband/hw/mlx5/fs.h
@@ -40,6 +40,8 @@ static inline void mlx5_ib_fs_cleanup(struct mlx5_ib_dev *dev)
* is a safe assumption that all references are gone.
*/
mlx5_ib_fs_cleanup_anchor(dev);
+ kfree(dev->flow_db->rdma_transport_tx);
+ kfree(dev->flow_db->rdma_transport_rx);
kfree(dev->flow_db);
}
#endif /* _MLX5_IB_FS_H */
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 974a45c92fbb..ccaaef20f50d 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -276,6 +276,7 @@ struct mlx5_ib_flow_matcher {
struct mlx5_core_dev *mdev;
atomic_t usecnt;
u8 match_criteria_enable;
+ u32 ib_port;
};
struct mlx5_ib_steering_anchor {
@@ -307,6 +308,8 @@ struct mlx5_ib_flow_db {
struct mlx5_ib_flow_prio rdma_tx[MLX5_IB_NUM_FLOW_FT];
struct mlx5_ib_flow_prio opfcs[MLX5_IB_OPCOUNTER_MAX];
struct mlx5_flow_table *lag_demux_ft;
+ struct mlx5_ib_flow_prio *rdma_transport_rx;
+ struct mlx5_ib_flow_prio *rdma_transport_tx;
/* Protect flow steering bypass flow tables
* when add/del flow rules.
* only single add/removal of flow steering rule could be done
diff --git a/include/uapi/rdma/mlx5_user_ioctl_cmds.h b/include/uapi/rdma/mlx5_user_ioctl_cmds.h
index fd2e4a3a56b3..18f9fe070213 100644
--- a/include/uapi/rdma/mlx5_user_ioctl_cmds.h
+++ b/include/uapi/rdma/mlx5_user_ioctl_cmds.h
@@ -239,6 +239,7 @@ enum mlx5_ib_flow_matcher_create_attrs {
MLX5_IB_ATTR_FLOW_MATCHER_MATCH_CRITERIA,
MLX5_IB_ATTR_FLOW_MATCHER_FLOW_FLAGS,
MLX5_IB_ATTR_FLOW_MATCHER_FT_TYPE,
+ MLX5_IB_ATTR_FLOW_MATCHER_IB_PORT,
};
enum mlx5_ib_flow_matcher_destroy_attrs {
diff --git a/include/uapi/rdma/mlx5_user_ioctl_verbs.h b/include/uapi/rdma/mlx5_user_ioctl_verbs.h
index 7c233df475e7..8f86e79d78a5 100644
--- a/include/uapi/rdma/mlx5_user_ioctl_verbs.h
+++ b/include/uapi/rdma/mlx5_user_ioctl_verbs.h
@@ -45,6 +45,8 @@ enum mlx5_ib_uapi_flow_table_type {
MLX5_IB_UAPI_FLOW_TABLE_TYPE_FDB = 0x2,
MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_RX = 0x3,
MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TX = 0x4,
+ MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_RX = 0x5,
+ MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_TRANSPORT_TX = 0x6,
};
enum mlx5_ib_uapi_flow_action_packet_reformat_type {
--
2.48.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH rdma-next v1 6/6] docs: infiniband: document the UCAP API
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
` (4 preceding siblings ...)
2025-03-06 11:51 ` [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace Leon Romanovsky
@ 2025-03-06 11:51 ` Leon Romanovsky
2025-03-08 19:23 ` [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
6 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-06 11:51 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Chiara Meiohas, Jonathan Corbet, linux-doc, linux-rdma,
Yishai Hadas
From: Chiara Meiohas <cmeiohas@nvidia.com>
Add an explanation on the newly added UCAP API.
Signed-off-by: Chiara Meiohas <cmeiohas@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
---
Documentation/infiniband/index.rst | 1 +
Documentation/infiniband/ucaps.rst | 71 ++++++++++++++++++++++++++++++
2 files changed, 72 insertions(+)
create mode 100644 Documentation/infiniband/ucaps.rst
diff --git a/Documentation/infiniband/index.rst b/Documentation/infiniband/index.rst
index 9cd7615438b9..5b4c24125f66 100644
--- a/Documentation/infiniband/index.rst
+++ b/Documentation/infiniband/index.rst
@@ -12,6 +12,7 @@ InfiniBand
opa_vnic
sysfs
tag_matching
+ ucaps
user_mad
user_verbs
diff --git a/Documentation/infiniband/ucaps.rst b/Documentation/infiniband/ucaps.rst
new file mode 100644
index 000000000000..b8b6927742f4
--- /dev/null
+++ b/Documentation/infiniband/ucaps.rst
@@ -0,0 +1,71 @@
+=================================
+Infiniband Userspace Capabilities
+=================================
+
+ User CAPabilities (UCAPs) provide fine-grained control over specific
+ firmware features in Infiniband (IB) devices. This approach offers
+ more granular capabilities than the existing Linux capabilities,
+ which may be too generic for certain FW features.
+
+ Each user capability is represented as a character device with root
+ read-write access. Root processes can grant users special privileges
+ by allowing access to these character devices (e.g., using chown).
+
+Usage
+=====
+
+ UCAPs allow control over specific features of an IB device using file
+ descriptors of UCAP character devices. Here is how a user enables
+ specific features of an IB device:
+
+ * A root process grants the user access to the UCAP files that
+ represents the capabilities (e.g., using chown).
+ * The user opens the UCAP files, obtaining file descriptors.
+ * When opening an IB device, include an array of the UCAP file
+ descriptors as an attribute.
+ * The ib_uverbs driver recognizes the UCAP file descriptors and enables
+ the corresponding capabilities for the IB device.
+
+Creating UCAPs
+==============
+
+ To create a new UCAP, drivers must first define a type in the
+ rdma_user_cap enum in rdma/ib_ucaps.h. The name of the UCAP character
+ device should be added to the ucap_names array in
+ drivers/infiniband/core/ucaps.c. Then, the driver can create the UCAP
+ character device by calling the ib_create_ucap API with the UCAP
+ type.
+
+ A reference count is stored for each UCAP to track creations and
+ removals of the UCAP device. If multiple creation calls are made with
+ the same type (e.g., for two IB devices), the UCAP character device
+ is created during the first call and subsequent calls increment the
+ reference count.
+
+ The UCAP character device is created under /dev/infiniband, and its
+ permissions are set to allow root read and write access only.
+
+Removing UCAPs
+==============
+
+ Each removal decrements the reference count of the UCAP. The UCAP
+ character device is removed from the filesystem only when the
+ reference count is decreased to 0.
+
+/dev and /sys/class files
+=========================
+
+ The class::
+
+ /sys/class/infiniband_ucaps
+
+ is created when the first UCAP character device is created.
+
+ The UCAP character device is created under /dev/infiniband.
+
+ For example, if mlx5_ib adds the rdma_user_cap
+ RDMA_UCAP_MLX5_CTRL_LOCAL with name "mlx5_perm_ctrl_local", this will
+ create the device node::
+
+ /dev/infiniband/mlx5_perm_ctrl_local
+
--
2.48.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
2025-03-06 11:51 ` [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace Leon Romanovsky
@ 2025-03-07 12:30 ` kernel test robot
2025-03-08 5:37 ` kernel test robot
1 sibling, 0 replies; 14+ messages in thread
From: kernel test robot @ 2025-03-07 12:30 UTC (permalink / raw)
To: Leon Romanovsky; +Cc: oe-kbuild-all
Hi Leon,
kernel test robot noticed the following build warnings:
[auto build test WARNING on rdma/for-next]
[also build test WARNING on linus/master v6.14-rc5 next-20250306]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base: https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link: https://lore.kernel.org/r/2287d8c50483e880450c7e8e08d9de34cdec1b14.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
config: arc-allyesconfig (https://download.01.org/0day-ci/archive/20250307/202503072017.WC5d8E9k-lkp@intel.com/config)
compiler: arceb-elf-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250307/202503072017.WC5d8E9k-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202503072017.WC5d8E9k-lkp@intel.com/
All warnings (new ones prefixed by >>):
drivers/infiniband/hw/mlx5/fs.c: In function '_get_prio':
drivers/infiniband/hw/mlx5/fs.c:702:16: error: 'struct mlx5_flow_table_attr' has no member named 'vport'
702 | ft_attr.vport = vport;
| ^
drivers/infiniband/hw/mlx5/fs.c: In function 'mlx5_ib_fill_transport_ns_info':
drivers/infiniband/hw/mlx5/fs.c:1430:14: error: implicit declaration of function 'MLX5_CAP_ADV_RDMA'; did you mean 'MLX5_CAP_ODP_MAX'? [-Werror=implicit-function-declaration]
1430 | if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
| ^~~~~~~~~~~~~~~~~
| MLX5_CAP_ODP_MAX
drivers/infiniband/hw/mlx5/fs.c:1430:43: error: 'rdma_transport_manager' undeclared (first use in this function); did you mean 'rdma_transport_type'?
1430 | if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
| ^~~~~~~~~~~~~~~~~~~~~~
| rdma_transport_type
drivers/infiniband/hw/mlx5/fs.c:1430:43: note: each undeclared identifier is reported only once for each function it appears in
drivers/infiniband/hw/mlx5/fs.c: In function '_get_flow_table':
drivers/infiniband/hw/mlx5/fs.c:1510:14: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
1510 | case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
drivers/infiniband/hw/mlx5/fs.c:1511:14: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
1511 | case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC
drivers/infiniband/hw/mlx5/fs.c:1512:53: error: 'MLX5_RDMA_TRANSPORT_BYPASS_PRIO' undeclared (first use in this function)
1512 | if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from include/linux/bits.h:6,
from include/linux/bitops.h:6,
from include/linux/bitmap.h:8,
from include/linux/ethtool.h:16,
from include/rdma/ib_verbs.h:15,
from drivers/infiniband/hw/mlx5/fs.c:7:
drivers/infiniband/hw/mlx5/fs.c:1522:37: error: implicit declaration of function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX'; did you mean 'MLX5_CAP_FLOWTABLE_RDMA_TX'? [-Werror=implicit-function-declaration]
1522 | BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/vdso/bits.h:7:44: note: in definition of macro 'BIT'
7 | #define BIT(nr) (UL(1) << (nr))
| ^~
drivers/infiniband/hw/mlx5/fs.c:1523:50: error: 'log_max_ft_size' undeclared (first use in this function)
1523 | ft_mdev, log_max_ft_size));
| ^~~~~~~~~~~~~~~
include/vdso/bits.h:7:44: note: in definition of macro 'BIT'
7 | #define BIT(nr) (UL(1) << (nr))
| ^~
drivers/infiniband/hw/mlx5/fs.c:1526:37: error: implicit declaration of function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX'; did you mean 'MLX5_CAP_FLOWTABLE_RDMA_TX'? [-Werror=implicit-function-declaration]
1526 | BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/vdso/bits.h:7:44: note: in definition of macro 'BIT'
7 | #define BIT(nr) (UL(1) << (nr))
| ^~
drivers/infiniband/hw/mlx5/fs.c:1538:22: error: implicit declaration of function 'mlx5_get_flow_vport_namespace'; did you mean 'mlx5_get_flow_vport_acl_namespace'? [-Werror=implicit-function-declaration]
1538 | ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| mlx5_get_flow_vport_acl_namespace
>> drivers/infiniband/hw/mlx5/fs.c:1538:20: warning: assignment to 'struct mlx5_flow_namespace *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
1538 | ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
| ^
drivers/infiniband/hw/mlx5/fs.c: In function 'mlx5_ib_ft_type_to_namespace':
drivers/infiniband/hw/mlx5/fs.c:1814:30: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
1814 | *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
drivers/infiniband/hw/mlx5/fs.c:1817:30: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
1817 | *namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC
drivers/infiniband/hw/mlx5/fs.c: In function 'get_dests':
drivers/infiniband/hw/mlx5/fs.c:1909:37: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
1909 | fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX) &&
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
drivers/infiniband/hw/mlx5/fs.c:1949:44: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
1949 | fs_matcher->ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) &&
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC
drivers/infiniband/hw/mlx5/fs.c: In function 'mlx5_ib_handler_MLX5_IB_METHOD_FLOW_MATCHER_CREATE':
drivers/infiniband/hw/mlx5/fs.c:2500:37: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS'?
2500 | if (obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX &&
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS
drivers/infiniband/hw/mlx5/fs.c:2501:37: error: 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX' undeclared (first use in this function); did you mean 'MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC'?
2501 | obj->ns_type != MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_FLOW_NAMESPACE_RDMA_TX_IPSEC
cc1: some warnings being treated as errors
vim +1538 drivers/infiniband/hw/mlx5/fs.c
1447
1448 static struct mlx5_ib_flow_prio *
1449 _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
1450 enum mlx5_flow_namespace_type ns_type,
1451 bool mcast, u32 ib_port)
1452 {
1453 struct mlx5_core_dev *ft_mdev = dev->mdev;
1454 struct mlx5_flow_namespace *ns = NULL;
1455 struct mlx5_ib_flow_prio *prio = NULL;
1456 int max_table_size = 0;
1457 u16 vport_idx = 0;
1458 bool esw_encap;
1459 u32 flags = 0;
1460 u16 vport = 0;
1461 int priority;
1462 int ret;
1463
1464 if (mcast)
1465 priority = MLX5_IB_FLOW_MCAST_PRIO;
1466 else
1467 priority = ib_prio_to_core_prio(user_priority, false);
1468
1469 esw_encap = mlx5_eswitch_get_encap_mode(dev->mdev) !=
1470 DEVLINK_ESWITCH_ENCAP_MODE_NONE;
1471 switch (ns_type) {
1472 case MLX5_FLOW_NAMESPACE_BYPASS:
1473 max_table_size = BIT(
1474 MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, log_max_ft_size));
1475 if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, decap) && !esw_encap)
1476 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
1477 if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev,
1478 reformat_l3_tunnel_to_l2) &&
1479 !esw_encap)
1480 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1481 break;
1482 case MLX5_FLOW_NAMESPACE_EGRESS:
1483 max_table_size = BIT(
1484 MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, log_max_ft_size));
1485 if (MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, reformat) &&
1486 !esw_encap)
1487 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1488 break;
1489 case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
1490 max_table_size = BIT(
1491 MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, log_max_ft_size));
1492 if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, decap) && esw_encap)
1493 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
1494 if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev,
1495 reformat_l3_tunnel_to_l2) &&
1496 esw_encap)
1497 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1498 priority = user_priority;
1499 break;
1500 case MLX5_FLOW_NAMESPACE_RDMA_RX:
1501 max_table_size = BIT(
1502 MLX5_CAP_FLOWTABLE_RDMA_RX(dev->mdev, log_max_ft_size));
1503 priority = user_priority;
1504 break;
1505 case MLX5_FLOW_NAMESPACE_RDMA_TX:
1506 max_table_size = BIT(
1507 MLX5_CAP_FLOWTABLE_RDMA_TX(dev->mdev, log_max_ft_size));
1508 priority = user_priority;
1509 break;
1510 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1511 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1512 if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
1513 return ERR_PTR(-EINVAL);
1514 ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
1515 &vport_idx, &vport,
1516 &ft_mdev, ib_port);
1517 if (ret)
1518 return ERR_PTR(ret);
1519
1520 if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
1521 max_table_size =
1522 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
1523 ft_mdev, log_max_ft_size));
1524 else
1525 max_table_size =
1526 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
1527 ft_mdev, log_max_ft_size));
1528 priority = user_priority;
1529 break;
1530 default:
1531 break;
1532 }
1533
1534 max_table_size = min_t(int, max_table_size, MLX5_FS_MAX_ENTRIES);
1535
1536 if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
1537 ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)
> 1538 ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
1539 else
1540 ns = mlx5_get_flow_namespace(ft_mdev, ns_type);
1541
1542 if (!ns)
1543 return ERR_PTR(-EOPNOTSUPP);
1544
1545 switch (ns_type) {
1546 case MLX5_FLOW_NAMESPACE_BYPASS:
1547 prio = &dev->flow_db->prios[priority];
1548 break;
1549 case MLX5_FLOW_NAMESPACE_EGRESS:
1550 prio = &dev->flow_db->egress_prios[priority];
1551 break;
1552 case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
1553 prio = &dev->flow_db->fdb[priority];
1554 break;
1555 case MLX5_FLOW_NAMESPACE_RDMA_RX:
1556 prio = &dev->flow_db->rdma_rx[priority];
1557 break;
1558 case MLX5_FLOW_NAMESPACE_RDMA_TX:
1559 prio = &dev->flow_db->rdma_tx[priority];
1560 break;
1561 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1562 prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];
1563 break;
1564 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1565 prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];
1566 break;
1567 default: return ERR_PTR(-EINVAL);
1568 }
1569
1570 if (!prio)
1571 return ERR_PTR(-EINVAL);
1572
1573 if (prio->flow_table)
1574 return prio;
1575
1576 return _get_prio(dev, ns, prio, priority, max_table_size,
1577 MLX5_FS_MAX_TYPES, flags, vport);
1578 }
1579
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext
2025-03-06 11:51 ` [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext Leon Romanovsky
@ 2025-03-08 4:22 ` kernel test robot
0 siblings, 0 replies; 14+ messages in thread
From: kernel test robot @ 2025-03-08 4:22 UTC (permalink / raw)
To: Leon Romanovsky; +Cc: llvm, oe-kbuild-all
Hi Leon,
kernel test robot noticed the following build errors:
[auto build test ERROR on rdma/for-next]
[also build test ERROR on linus/master v6.14-rc5 next-20250307]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base: https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link: https://lore.kernel.org/r/8b180583a207cb30deb7a2967934079749cdcc44.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext
config: s390-defconfig (https://download.01.org/0day-ci/archive/20250308/202503081223.ISLOmyVZ-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250308/202503081223.ISLOmyVZ-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202503081223.ISLOmyVZ-lkp@intel.com/
All errors (new ones prefixed by >>):
>> drivers/infiniband/hw/mlx5/main.c:1992:10: error: call to undeclared function 'mlx5_cmd_add_privileged_uid'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
err = mlx5_cmd_add_privileged_uid(dev->mdev,
^
>> drivers/infiniband/hw/mlx5/main.c:2086:3: error: call to undeclared function 'mlx5_cmd_remove_privileged_uid'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
mlx5_cmd_remove_privileged_uid(dev->mdev, context->devx_uid);
^
drivers/infiniband/hw/mlx5/main.c:2134:4: error: call to undeclared function 'mlx5_cmd_remove_privileged_uid'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
mlx5_cmd_remove_privileged_uid(dev->mdev,
^
drivers/infiniband/hw/mlx5/main.c:4229:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
^~~~~~~~~~~~~~~~~~~~~~~
MLX5_UCTX_CAP_RAW_TX
include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
^
drivers/infiniband/hw/mlx5/main.c:4233:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
^
drivers/infiniband/hw/mlx5/main.c:4241:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
^~~~~~~~~~~~~~~~~~~~~~~
MLX5_UCTX_CAP_RAW_TX
include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
^
drivers/infiniband/hw/mlx5/main.c:4248:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
^
drivers/infiniband/hw/mlx5/main.c:4257:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
^~~~~~~~~~~~~~~~~~~~~~~
MLX5_UCTX_CAP_RAW_TX
include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
^
drivers/infiniband/hw/mlx5/main.c:4264:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
#define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
#define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
^ ~~~
include/linux/mlx5/device.h:52:35: note: expanded from macro '__mlx5_bit_off'
#define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
^ ~~~
include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
#define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER)
^ ~~~~~~
include/uapi/linux/byteorder/big_endian.h:39:51: note: expanded from macro '__be64_to_cpu'
#define __be64_to_cpu(x) ((__force __u64)(__be64)(x))
^
drivers/infiniband/hw/mlx5/main.c:4265:6: error: use of undeclared identifier 'MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL'; did you mean 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER'?
MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER
include/linux/mlx5/mlx5_ifc.h:12493:2: note: 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER' declared here
MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
^
drivers/infiniband/hw/mlx5/main.c:4318:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
#define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
#define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
^ ~~~
include/linux/mlx5/device.h:52:35: note: expanded from macro '__mlx5_bit_off'
#define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
^ ~~~
include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
#define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER)
^ ~~~~~~
include/uapi/linux/byteorder/big_endian.h:39:51: note: expanded from macro '__be64_to_cpu'
#define __be64_to_cpu(x) ((__force __u64)(__be64)(x))
^
drivers/infiniband/hw/mlx5/main.c:4319:6: error: use of undeclared identifier 'MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL'; did you mean 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER'?
MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER
include/linux/mlx5/mlx5_ifc.h:12493:2: note: 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER' declared here
MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
^
12 errors generated.
vim +/mlx5_cmd_add_privileged_uid +1992 drivers/infiniband/hw/mlx5/main.c
1943
1944 static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx,
1945 struct ib_udata *udata)
1946 {
1947 struct ib_device *ibdev = uctx->device;
1948 struct mlx5_ib_dev *dev = to_mdev(ibdev);
1949 struct mlx5_ib_alloc_ucontext_req_v2 req = {};
1950 struct mlx5_ib_alloc_ucontext_resp resp = {};
1951 struct mlx5_ib_ucontext *context = to_mucontext(uctx);
1952 struct mlx5_bfreg_info *bfregi;
1953 int ver;
1954 int err;
1955 size_t min_req_v2 = offsetof(struct mlx5_ib_alloc_ucontext_req_v2,
1956 max_cqe_version);
1957 bool lib_uar_4k;
1958 bool lib_uar_dyn;
1959
1960 if (!dev->ib_active)
1961 return -EAGAIN;
1962
1963 if (udata->inlen == sizeof(struct mlx5_ib_alloc_ucontext_req))
1964 ver = 0;
1965 else if (udata->inlen >= min_req_v2)
1966 ver = 2;
1967 else
1968 return -EINVAL;
1969
1970 err = ib_copy_from_udata(&req, udata, min(udata->inlen, sizeof(req)));
1971 if (err)
1972 return err;
1973
1974 if (req.flags & ~MLX5_IB_ALLOC_UCTX_DEVX)
1975 return -EOPNOTSUPP;
1976
1977 if (req.comp_mask || req.reserved0 || req.reserved1 || req.reserved2)
1978 return -EOPNOTSUPP;
1979
1980 req.total_num_bfregs = ALIGN(req.total_num_bfregs,
1981 MLX5_NON_FP_BFREGS_PER_UAR);
1982 if (req.num_low_latency_bfregs > req.total_num_bfregs - 1)
1983 return -EINVAL;
1984
1985 if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX) {
1986 err = mlx5_ib_devx_create(dev, true, uctx->enabled_caps);
1987 if (err < 0)
1988 goto out_ctx;
1989 context->devx_uid = err;
1990
1991 if (uctx_rdma_ctrl_is_enabled(uctx->enabled_caps)) {
> 1992 err = mlx5_cmd_add_privileged_uid(dev->mdev,
1993 context->devx_uid);
1994 if (err)
1995 goto out_devx;
1996 }
1997 }
1998
1999 lib_uar_4k = req.lib_caps & MLX5_LIB_CAP_4K_UAR;
2000 lib_uar_dyn = req.lib_caps & MLX5_LIB_CAP_DYN_UAR;
2001 bfregi = &context->bfregi;
2002
2003 if (lib_uar_dyn) {
2004 bfregi->lib_uar_dyn = lib_uar_dyn;
2005 goto uar_done;
2006 }
2007
2008 /* updates req->total_num_bfregs */
2009 err = calc_total_bfregs(dev, lib_uar_4k, &req, bfregi);
2010 if (err)
2011 goto out_ucap;
2012
2013 mutex_init(&bfregi->lock);
2014 bfregi->lib_uar_4k = lib_uar_4k;
2015 bfregi->count = kcalloc(bfregi->total_num_bfregs, sizeof(*bfregi->count),
2016 GFP_KERNEL);
2017 if (!bfregi->count) {
2018 err = -ENOMEM;
2019 goto out_ucap;
2020 }
2021
2022 bfregi->sys_pages = kcalloc(bfregi->num_sys_pages,
2023 sizeof(*bfregi->sys_pages),
2024 GFP_KERNEL);
2025 if (!bfregi->sys_pages) {
2026 err = -ENOMEM;
2027 goto out_count;
2028 }
2029
2030 err = allocate_uars(dev, context);
2031 if (err)
2032 goto out_sys_pages;
2033
2034 uar_done:
2035 err = mlx5_ib_alloc_transport_domain(dev, &context->tdn,
2036 context->devx_uid);
2037 if (err)
2038 goto out_uars;
2039
2040 INIT_LIST_HEAD(&context->db_page_list);
2041 mutex_init(&context->db_page_mutex);
2042
2043 context->cqe_version = min_t(__u8,
2044 (__u8)MLX5_CAP_GEN(dev->mdev, cqe_version),
2045 req.max_cqe_version);
2046
2047 err = set_ucontext_resp(uctx, &resp);
2048 if (err)
2049 goto out_mdev;
2050
2051 resp.response_length = min(udata->outlen, sizeof(resp));
2052 err = ib_copy_to_udata(udata, &resp, resp.response_length);
2053 if (err)
2054 goto out_mdev;
2055
2056 bfregi->ver = ver;
2057 bfregi->num_low_latency_bfregs = req.num_low_latency_bfregs;
2058 context->lib_caps = req.lib_caps;
2059 print_lib_caps(dev, context->lib_caps);
2060
2061 if (mlx5_ib_lag_should_assign_affinity(dev)) {
2062 u32 port = mlx5_core_native_port_num(dev->mdev) - 1;
2063
2064 atomic_set(&context->tx_port_affinity,
2065 atomic_add_return(
2066 1, &dev->port[port].roce.tx_port_affinity));
2067 }
2068
2069 return 0;
2070
2071 out_mdev:
2072 mlx5_ib_dealloc_transport_domain(dev, context->tdn, context->devx_uid);
2073
2074 out_uars:
2075 deallocate_uars(dev, context);
2076
2077 out_sys_pages:
2078 kfree(bfregi->sys_pages);
2079
2080 out_count:
2081 kfree(bfregi->count);
2082
2083 out_ucap:
2084 if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX &&
2085 uctx_rdma_ctrl_is_enabled(uctx->enabled_caps))
> 2086 mlx5_cmd_remove_privileged_uid(dev->mdev, context->devx_uid);
2087
2088 out_devx:
2089 if (req.flags & MLX5_IB_ALLOC_UCTX_DEVX)
2090 mlx5_ib_devx_destroy(dev, context->devx_uid);
2091
2092 out_ctx:
2093 return err;
2094 }
2095
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
2025-03-06 11:51 ` [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace Leon Romanovsky
2025-03-07 12:30 ` kernel test robot
@ 2025-03-08 5:37 ` kernel test robot
1 sibling, 0 replies; 14+ messages in thread
From: kernel test robot @ 2025-03-08 5:37 UTC (permalink / raw)
To: Leon Romanovsky; +Cc: llvm, oe-kbuild-all
Hi Leon,
kernel test robot noticed the following build errors:
[auto build test ERROR on rdma/for-next]
[also build test ERROR on linus/master v6.14-rc5 next-20250307]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base: https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link: https://lore.kernel.org/r/2287d8c50483e880450c7e8e08d9de34cdec1b14.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
config: s390-defconfig (https://download.01.org/0day-ci/archive/20250308/202503081358.5IRVqXGS-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250308/202503081358.5IRVqXGS-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202503081358.5IRVqXGS-lkp@intel.com/
All errors (new ones prefixed by >>):
drivers/infiniband/hw/mlx5/fs.c:702:10: error: no member named 'vport' in 'struct mlx5_flow_table_attr'
ft_attr.vport = vport;
~~~~~~~ ^
>> drivers/infiniband/hw/mlx5/fs.c:1430:7: error: call to undeclared function 'MLX5_CAP_ADV_RDMA'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
^
drivers/infiniband/hw/mlx5/fs.c:1430:36: error: use of undeclared identifier 'rdma_transport_manager'; did you mean 'skb_transport_header'?
if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
^~~~~~~~~~~~~~~~~~~~~~
skb_transport_header
include/linux/skbuff.h:3018:30: note: 'skb_transport_header' declared here
static inline unsigned char *skb_transport_header(const struct sk_buff *skb)
^
drivers/infiniband/hw/mlx5/fs.c:1510:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
^
drivers/infiniband/hw/mlx5/fs.c:1511:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
^
drivers/infiniband/hw/mlx5/fs.c:1512:39: error: use of undeclared identifier 'MLX5_RDMA_TRANSPORT_BYPASS_PRIO'
if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
^
drivers/infiniband/hw/mlx5/fs.c:1520:18: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
^
>> drivers/infiniband/hw/mlx5/fs.c:1522:9: error: call to undeclared function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
^
drivers/infiniband/hw/mlx5/fs.c:1523:15: error: use of undeclared identifier 'log_max_ft_size'
ft_mdev, log_max_ft_size));
^
>> drivers/infiniband/hw/mlx5/fs.c:1526:9: error: call to undeclared function 'MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
^
drivers/infiniband/hw/mlx5/fs.c:1527:15: error: use of undeclared identifier 'log_max_ft_size'
ft_mdev, log_max_ft_size));
^
drivers/infiniband/hw/mlx5/fs.c:1536:17: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
^
drivers/infiniband/hw/mlx5/fs.c:1537:17: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)
^
>> drivers/infiniband/hw/mlx5/fs.c:1538:8: error: call to undeclared function 'mlx5_get_flow_vport_namespace'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
^
drivers/infiniband/hw/mlx5/fs.c:1538:8: note: did you mean 'mlx5_get_flow_vport_acl_namespace'?
include/linux/mlx5/fs.h:197:1: note: 'mlx5_get_flow_vport_acl_namespace' declared here
mlx5_get_flow_vport_acl_namespace(struct mlx5_core_dev *dev,
^
drivers/infiniband/hw/mlx5/fs.c:1538:6: error: incompatible integer to pointer conversion assigning to 'struct mlx5_flow_namespace *' from 'int' [-Wint-conversion]
ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/infiniband/hw/mlx5/fs.c:1561:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
^
drivers/infiniband/hw/mlx5/fs.c:1564:7: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
^
drivers/infiniband/hw/mlx5/fs.c:1814:16: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX'
*namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX;
^
drivers/infiniband/hw/mlx5/fs.c:1817:16: error: use of undeclared identifier 'MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX'
*namespace = MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX;
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
vim +/MLX5_CAP_ADV_RDMA +1430 drivers/infiniband/hw/mlx5/fs.c
1417
1418 static int mlx5_ib_fill_transport_ns_info(struct mlx5_ib_dev *dev,
1419 enum mlx5_flow_namespace_type type,
1420 u32 *flags, u16 *vport_idx,
1421 u16 *vport,
1422 struct mlx5_core_dev **ft_mdev,
1423 u32 ib_port)
1424 {
1425 struct mlx5_core_dev *esw_mdev;
1426
1427 if (!is_mdev_switchdev_mode(dev->mdev))
1428 return 0;
1429
> 1430 if (!MLX5_CAP_ADV_RDMA(dev->mdev, rdma_transport_manager))
1431 return -EOPNOTSUPP;
1432
1433 if (!dev->port[ib_port - 1].rep)
1434 return -EINVAL;
1435
1436 esw_mdev = mlx5_eswitch_get_core_dev(dev->port[ib_port - 1].rep->esw);
1437 if (esw_mdev != dev->mdev)
1438 return -EOPNOTSUPP;
1439
1440 *flags |= MLX5_FLOW_TABLE_OTHER_VPORT;
1441 *ft_mdev = esw_mdev;
1442 *vport = dev->port[ib_port - 1].rep->vport;
1443 *vport_idx = dev->port[ib_port - 1].rep->vport_index;
1444
1445 return 0;
1446 }
1447
1448 static struct mlx5_ib_flow_prio *
1449 _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
1450 enum mlx5_flow_namespace_type ns_type,
1451 bool mcast, u32 ib_port)
1452 {
1453 struct mlx5_core_dev *ft_mdev = dev->mdev;
1454 struct mlx5_flow_namespace *ns = NULL;
1455 struct mlx5_ib_flow_prio *prio = NULL;
1456 int max_table_size = 0;
1457 u16 vport_idx = 0;
1458 bool esw_encap;
1459 u32 flags = 0;
1460 u16 vport = 0;
1461 int priority;
1462 int ret;
1463
1464 if (mcast)
1465 priority = MLX5_IB_FLOW_MCAST_PRIO;
1466 else
1467 priority = ib_prio_to_core_prio(user_priority, false);
1468
1469 esw_encap = mlx5_eswitch_get_encap_mode(dev->mdev) !=
1470 DEVLINK_ESWITCH_ENCAP_MODE_NONE;
1471 switch (ns_type) {
1472 case MLX5_FLOW_NAMESPACE_BYPASS:
1473 max_table_size = BIT(
1474 MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, log_max_ft_size));
1475 if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, decap) && !esw_encap)
1476 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
1477 if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev,
1478 reformat_l3_tunnel_to_l2) &&
1479 !esw_encap)
1480 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1481 break;
1482 case MLX5_FLOW_NAMESPACE_EGRESS:
1483 max_table_size = BIT(
1484 MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, log_max_ft_size));
1485 if (MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, reformat) &&
1486 !esw_encap)
1487 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1488 break;
1489 case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
1490 max_table_size = BIT(
1491 MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, log_max_ft_size));
1492 if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev, decap) && esw_encap)
1493 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
1494 if (MLX5_CAP_ESW_FLOWTABLE_FDB(dev->mdev,
1495 reformat_l3_tunnel_to_l2) &&
1496 esw_encap)
1497 flags |= MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
1498 priority = user_priority;
1499 break;
1500 case MLX5_FLOW_NAMESPACE_RDMA_RX:
1501 max_table_size = BIT(
1502 MLX5_CAP_FLOWTABLE_RDMA_RX(dev->mdev, log_max_ft_size));
1503 priority = user_priority;
1504 break;
1505 case MLX5_FLOW_NAMESPACE_RDMA_TX:
1506 max_table_size = BIT(
1507 MLX5_CAP_FLOWTABLE_RDMA_TX(dev->mdev, log_max_ft_size));
1508 priority = user_priority;
1509 break;
1510 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1511 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1512 if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
1513 return ERR_PTR(-EINVAL);
1514 ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
1515 &vport_idx, &vport,
1516 &ft_mdev, ib_port);
1517 if (ret)
1518 return ERR_PTR(ret);
1519
1520 if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX)
1521 max_table_size =
> 1522 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_RX(
1523 ft_mdev, log_max_ft_size));
1524 else
1525 max_table_size =
> 1526 BIT(MLX5_CAP_FLOWTABLE_RDMA_TRANSPORT_TX(
1527 ft_mdev, log_max_ft_size));
1528 priority = user_priority;
1529 break;
1530 default:
1531 break;
1532 }
1533
1534 max_table_size = min_t(int, max_table_size, MLX5_FS_MAX_ENTRIES);
1535
1536 if (ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX ||
1537 ns_type == MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX)
> 1538 ns = mlx5_get_flow_vport_namespace(ft_mdev, ns_type, vport_idx);
1539 else
1540 ns = mlx5_get_flow_namespace(ft_mdev, ns_type);
1541
1542 if (!ns)
1543 return ERR_PTR(-EOPNOTSUPP);
1544
1545 switch (ns_type) {
1546 case MLX5_FLOW_NAMESPACE_BYPASS:
1547 prio = &dev->flow_db->prios[priority];
1548 break;
1549 case MLX5_FLOW_NAMESPACE_EGRESS:
1550 prio = &dev->flow_db->egress_prios[priority];
1551 break;
1552 case MLX5_FLOW_NAMESPACE_FDB_BYPASS:
1553 prio = &dev->flow_db->fdb[priority];
1554 break;
1555 case MLX5_FLOW_NAMESPACE_RDMA_RX:
1556 prio = &dev->flow_db->rdma_rx[priority];
1557 break;
1558 case MLX5_FLOW_NAMESPACE_RDMA_TX:
1559 prio = &dev->flow_db->rdma_tx[priority];
1560 break;
1561 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1562 prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];
1563 break;
1564 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1565 prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];
1566 break;
1567 default: return ERR_PTR(-EINVAL);
1568 }
1569
1570 if (!prio)
1571 return ERR_PTR(-EINVAL);
1572
1573 if (prio->flow_table)
1574 return prio;
1575
1576 return _get_prio(dev, ns, prio, priority, max_table_size,
1577 MLX5_FS_MAX_TYPES, flags, vport);
1578 }
1579
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API
2025-03-06 11:51 ` [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API Leon Romanovsky
@ 2025-03-08 7:25 ` Zhu Yanjun
2025-03-08 19:21 ` Leon Romanovsky
0 siblings, 1 reply; 14+ messages in thread
From: Zhu Yanjun @ 2025-03-08 7:25 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe; +Cc: Chiara Meiohas, linux-rdma, Yishai Hadas
在 2025/3/6 12:51, Leon Romanovsky 写道:
> From: Chiara Meiohas <cmeiohas@nvidia.com>
>
> Implement a new User CAPabilities (UCAP) API to provide fine-grained
> control over specific firmware features.
>
> This approach offers more granular capabilities than the existing Linux
> capabilities, which may be too generic for certain FW features.
>
> This mechanism represents each capability as a character device with
> root read-write access. Root processes can grant users special
> privileges by allowing access to these character devices (e.g., using
> chown).
Hi, Chiara
I read this patch-set carefully. If I get this patch-set correctly, this
patch-set introduces a new User CAPabilities API to control specific
firmware feature.
Do we have a user guide to use this UCAP? For example, we suspect that a
Firmware problem will occur in production environment, how can we use
this UCAP to debug this Firmware problem?
Thanks.
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
Zhu Yanjun
>
> UCAP character devices are located in /dev/infiniband and the class path
> is /sys/class/infiniband_ucaps.
>
> Signed-off-by: Chiara Meiohas <cmeiohas@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Leon Romanovsky <leon@kernel.org>
> ---
> drivers/infiniband/core/Makefile | 3 +-
> drivers/infiniband/core/ucaps.c | 267 ++++++++++++++++++++++++++
> drivers/infiniband/core/uverbs_main.c | 2 +
> include/rdma/ib_ucaps.h | 25 +++
> 4 files changed, 296 insertions(+), 1 deletion(-)
> create mode 100644 drivers/infiniband/core/ucaps.c
> create mode 100644 include/rdma/ib_ucaps.h
>
> diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
> index 8ab4eea5a0a5..d49ded7e95f0 100644
> --- a/drivers/infiniband/core/Makefile
> +++ b/drivers/infiniband/core/Makefile
> @@ -39,6 +39,7 @@ ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
> uverbs_std_types_async_fd.o \
> uverbs_std_types_srq.o \
> uverbs_std_types_wq.o \
> - uverbs_std_types_qp.o
> + uverbs_std_types_qp.o \
> + ucaps.o
> ib_uverbs-$(CONFIG_INFINIBAND_USER_MEM) += umem.o umem_dmabuf.o
> ib_uverbs-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o
> diff --git a/drivers/infiniband/core/ucaps.c b/drivers/infiniband/core/ucaps.c
> new file mode 100644
> index 000000000000..6853c6d078f9
> --- /dev/null
> +++ b/drivers/infiniband/core/ucaps.c
> @@ -0,0 +1,267 @@
> +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
> +/*
> + * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + */
> +
> +#include <linux/kref.h>
> +#include <linux/cdev.h>
> +#include <linux/mutex.h>
> +#include <linux/file.h>
> +#include <linux/fs.h>
> +#include <rdma/ib_ucaps.h>
> +
> +#define RDMA_UCAP_FIRST RDMA_UCAP_MLX5_CTRL_LOCAL
> +
> +static DEFINE_MUTEX(ucaps_mutex);
> +static struct ib_ucap *ucaps_list[RDMA_UCAP_MAX];
> +static bool ucaps_class_is_registered;
> +static dev_t ucaps_base_dev;
> +
> +struct ib_ucap {
> + struct cdev cdev;
> + struct device dev;
> + struct kref ref;
> +};
> +
> +static const char *ucap_names[RDMA_UCAP_MAX] = {
> + [RDMA_UCAP_MLX5_CTRL_LOCAL] = "mlx5_perm_ctrl_local",
> + [RDMA_UCAP_MLX5_CTRL_OTHER_VHCA] = "mlx5_perm_ctrl_other_vhca"
> +};
> +
> +static char *ucaps_devnode(const struct device *dev, umode_t *mode)
> +{
> + if (mode)
> + *mode = 0600;
> +
> + return kasprintf(GFP_KERNEL, "infiniband/%s", dev_name(dev));
> +}
> +
> +static const struct class ucaps_class = {
> + .name = "infiniband_ucaps",
> + .devnode = ucaps_devnode,
> +};
> +
> +static const struct file_operations ucaps_cdev_fops = {
> + .owner = THIS_MODULE,
> + .open = simple_open,
> +};
> +
> +/**
> + * ib_cleanup_ucaps - cleanup all API resources and class.
> + *
> + * This is called once, when removing the ib_uverbs module.
> + */
> +void ib_cleanup_ucaps(void)
> +{
> + mutex_lock(&ucaps_mutex);
> + if (!ucaps_class_is_registered) {
> + mutex_unlock(&ucaps_mutex);
> + return;
> + }
> +
> + for (int i = RDMA_UCAP_FIRST; i < RDMA_UCAP_MAX; i++)
> + WARN_ON(ucaps_list[i]);
> +
> + class_unregister(&ucaps_class);
> + ucaps_class_is_registered = false;
> + unregister_chrdev_region(ucaps_base_dev, RDMA_UCAP_MAX);
> + mutex_unlock(&ucaps_mutex);
> +}
> +
> +static int get_ucap_from_devt(dev_t devt, u64 *idx_mask)
> +{
> + for (int type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
> + if (ucaps_list[type] && ucaps_list[type]->dev.devt == devt) {
> + *idx_mask |= 1 << type;
> + return 0;
> + }
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int get_devt_from_fd(unsigned int fd, dev_t *ret_dev)
> +{
> + struct file *file;
> +
> + file = fget(fd);
> + if (!file)
> + return -EBADF;
> +
> + *ret_dev = file_inode(file)->i_rdev;
> + fput(file);
> + return 0;
> +}
> +
> +/**
> + * ib_ucaps_init - Initialization required before ucap creation.
> + *
> + * Return: 0 on success, or a negative errno value on failure
> + */
> +static int ib_ucaps_init(void)
> +{
> + int ret = 0;
> +
> + if (ucaps_class_is_registered)
> + return ret;
> +
> + ret = class_register(&ucaps_class);
> + if (ret)
> + return ret;
> +
> + ret = alloc_chrdev_region(&ucaps_base_dev, 0, RDMA_UCAP_MAX,
> + ucaps_class.name);
> + if (ret < 0) {
> + class_unregister(&ucaps_class);
> + return ret;
> + }
> +
> + ucaps_class_is_registered = true;
> +
> + return 0;
> +}
> +
> +static void ucap_dev_release(struct device *device)
> +{
> + struct ib_ucap *ucap = container_of(device, struct ib_ucap, dev);
> +
> + kfree(ucap);
> +}
> +
> +/**
> + * ib_create_ucap - Add a ucap character device
> + * @type: UCAP type
> + *
> + * Creates a ucap character device in the /dev/infiniband directory. By default,
> + * the device has root-only read-write access.
> + *
> + * A driver may call this multiple times with the same UCAP type. A reference
> + * count tracks creations and deletions.
> + *
> + * Return: 0 on success, or a negative errno value on failure
> + */
> +int ib_create_ucap(enum rdma_user_cap type)
> +{
> + struct ib_ucap *ucap;
> + int ret;
> +
> + if (type >= RDMA_UCAP_MAX)
> + return -EINVAL;
> +
> + mutex_lock(&ucaps_mutex);
> + ret = ib_ucaps_init();
> + if (ret)
> + goto unlock;
> +
> + ucap = ucaps_list[type];
> + if (ucap) {
> + kref_get(&ucap->ref);
> + mutex_unlock(&ucaps_mutex);
> + return 0;
> + }
> +
> + ucap = kzalloc(sizeof(*ucap), GFP_KERNEL);
> + if (!ucap) {
> + ret = -ENOMEM;
> + goto unlock;
> + }
> +
> + device_initialize(&ucap->dev);
> + ucap->dev.class = &ucaps_class;
> + ucap->dev.devt = MKDEV(MAJOR(ucaps_base_dev), type);
> + ucap->dev.release = ucap_dev_release;
> + ret = dev_set_name(&ucap->dev, ucap_names[type]);
> + if (ret)
> + goto err_device;
> +
> + cdev_init(&ucap->cdev, &ucaps_cdev_fops);
> + ucap->cdev.owner = THIS_MODULE;
> +
> + ret = cdev_device_add(&ucap->cdev, &ucap->dev);
> + if (ret)
> + goto err_device;
> +
> + kref_init(&ucap->ref);
> + ucaps_list[type] = ucap;
> + mutex_unlock(&ucaps_mutex);
> +
> + return 0;
> +
> +err_device:
> + put_device(&ucap->dev);
> +unlock:
> + mutex_unlock(&ucaps_mutex);
> + return ret;
> +}
> +EXPORT_SYMBOL(ib_create_ucap);
> +
> +static void ib_release_ucap(struct kref *ref)
> +{
> + struct ib_ucap *ucap = container_of(ref, struct ib_ucap, ref);
> + enum rdma_user_cap type;
> +
> + for (type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
> + if (ucaps_list[type] == ucap)
> + break;
> + }
> + WARN_ON(type == RDMA_UCAP_MAX);
> +
> + ucaps_list[type] = NULL;
> + cdev_device_del(&ucap->cdev, &ucap->dev);
> + put_device(&ucap->dev);
> +}
> +
> +/**
> + * ib_remove_ucap - Remove a ucap character device
> + * @type: User cap type
> + *
> + * Removes the ucap character device according to type. The device is completely
> + * removed from the filesystem when its reference count reaches 0.
> + */
> +void ib_remove_ucap(enum rdma_user_cap type)
> +{
> + struct ib_ucap *ucap;
> +
> + mutex_lock(&ucaps_mutex);
> + ucap = ucaps_list[type];
> + if (WARN_ON(!ucap))
> + goto end;
> +
> + kref_put(&ucap->ref, ib_release_ucap);
> +end:
> + mutex_unlock(&ucaps_mutex);
> +}
> +EXPORT_SYMBOL(ib_remove_ucap);
> +
> +/**
> + * ib_get_ucaps - Get bitmask of ucap types from file descriptors
> + * @fds: Array of file descriptors
> + * @fd_count: Number of file descriptors in the array
> + * @idx_mask: Bitmask to be updated based on the ucaps in the fd list
> + *
> + * Given an array of file descriptors, this function returns a bitmask of
> + * the ucaps where a bit is set if an FD for that ucap type was in the array.
> + *
> + * Return: 0 on success, or a negative errno value on failure
> + */
> +int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask)
> +{
> + int ret = 0;
> + dev_t dev;
> +
> + *idx_mask = 0;
> + mutex_lock(&ucaps_mutex);
> + for (int i = 0; i < fd_count; i++) {
> + ret = get_devt_from_fd(fds[i], &dev);
> + if (ret)
> + goto end;
> +
> + ret = get_ucap_from_devt(dev, idx_mask);
> + if (ret)
> + goto end;
> + }
> +
> +end:
> + mutex_unlock(&ucaps_mutex);
> + return ret;
> +}
> diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
> index 85cfc790a7bb..973fe2c7ef53 100644
> --- a/drivers/infiniband/core/uverbs_main.c
> +++ b/drivers/infiniband/core/uverbs_main.c
> @@ -52,6 +52,7 @@
> #include <rdma/ib.h>
> #include <rdma/uverbs_std_types.h>
> #include <rdma/rdma_netlink.h>
> +#include <rdma/ib_ucaps.h>
>
> #include "uverbs.h"
> #include "core_priv.h"
> @@ -1345,6 +1346,7 @@ static void __exit ib_uverbs_cleanup(void)
> IB_UVERBS_NUM_FIXED_MINOR);
> unregister_chrdev_region(dynamic_uverbs_dev,
> IB_UVERBS_NUM_DYNAMIC_MINOR);
> + ib_cleanup_ucaps();
> mmu_notifier_synchronize();
> }
>
> diff --git a/include/rdma/ib_ucaps.h b/include/rdma/ib_ucaps.h
> new file mode 100644
> index 000000000000..8f0552a2b2b0
> --- /dev/null
> +++ b/include/rdma/ib_ucaps.h
> @@ -0,0 +1,25 @@
> +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
> +/*
> + * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + */
> +
> +#ifndef _IB_UCAPS_H_
> +#define _IB_UCAPS_H_
> +
> +#define UCAP_ENABLED(ucaps, type) (!!((ucaps) & (1U << (type))))
> +
> +enum rdma_user_cap {
> + RDMA_UCAP_MLX5_CTRL_LOCAL,
> + RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
> + RDMA_UCAP_MAX
> +};
> +
> +void ib_cleanup_ucaps(void);
> +
> +int ib_create_ucap(enum rdma_user_cap type);
> +
> +void ib_remove_ucap(enum rdma_user_cap type);
> +
> +int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask);
> +
> +#endif /* _IB_UCAPS_H_ */
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities
2025-03-06 11:51 ` [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities Leon Romanovsky
@ 2025-03-08 12:33 ` kernel test robot
0 siblings, 0 replies; 14+ messages in thread
From: kernel test robot @ 2025-03-08 12:33 UTC (permalink / raw)
To: Leon Romanovsky; +Cc: llvm, oe-kbuild-all
Hi Leon,
kernel test robot noticed the following build errors:
[auto build test ERROR on rdma/for-next]
[also build test ERROR on linus/master v6.14-rc5 next-20250307]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Leon-Romanovsky/RDMA-uverbs-Introduce-UCAP-User-CAPabilities-API/20250306-195708
base: https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
patch link: https://lore.kernel.org/r/30ed40e7a12a694cf4ee257459ed61b145b7837d.1741261611.git.leon%40kernel.org
patch subject: [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities
config: arm64-allmodconfig (https://download.01.org/0day-ci/archive/20250308/202503082057.0xxskyFs-lkp@intel.com/config)
compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250308/202503082057.0xxskyFs-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202503082057.0xxskyFs-lkp@intel.com/
All errors (new ones prefixed by >>):
>> drivers/infiniband/hw/mlx5/main.c:4207:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
4207 | if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
| ^~~~~~~~~~~~~~~~~~~~~~~
| MLX5_UCTX_CAP_RAW_TX
include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
1571 | MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
| ^
>> drivers/infiniband/hw/mlx5/main.c:4211:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
4211 | MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
13 | RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
| ^
drivers/infiniband/hw/mlx5/main.c:4219:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
4219 | if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
| ^~~~~~~~~~~~~~~~~~~~~~~
| MLX5_UCTX_CAP_RAW_TX
include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
1571 | MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
| ^
drivers/infiniband/hw/mlx5/main.c:4226:6: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA'; did you mean 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA'?
4226 | MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| RDMA_UCAP_MLX5_CTRL_OTHER_VHCA
include/rdma/ib_ucaps.h:13:2: note: 'RDMA_UCAP_MLX5_CTRL_OTHER_VHCA' declared here
13 | RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
| ^
drivers/infiniband/hw/mlx5/main.c:4235:42: error: use of undeclared identifier 'MLX5_UCTX_CAP_RDMA_CTRL'; did you mean 'MLX5_UCTX_CAP_RAW_TX'?
4235 | if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
| ^~~~~~~~~~~~~~~~~~~~~~~
| MLX5_UCTX_CAP_RAW_TX
include/linux/mlx5/mlx5_ifc.h:1571:2: note: 'MLX5_UCTX_CAP_RAW_TX' declared here
1571 | MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
16 | #define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER)
| ^ ~~~~~~
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:131:31: note: expanded from macro '__swab64'
131 | (__u64)(__builtin_constant_p(x) ? \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:25:12: note: expanded from macro '___constant_swab64'
25 | (((__u64)(x) & (__u64)0x00000000000000ffULL) << 56) | \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:26:12: note: expanded from macro '___constant_swab64'
26 | (((__u64)(x) & (__u64)0x000000000000ff00ULL) << 40) | \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:27:12: note: expanded from macro '___constant_swab64'
27 | (((__u64)(x) & (__u64)0x0000000000ff0000ULL) << 24) | \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:28:12: note: expanded from macro '___constant_swab64'
28 | (((__u64)(x) & (__u64)0x00000000ff000000ULL) << 8) | \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:29:12: note: expanded from macro '___constant_swab64'
29 | (((__u64)(x) & (__u64)0x000000ff00000000ULL) >> 8) | \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:30:12: note: expanded from macro '___constant_swab64'
30 | (((__u64)(x) & (__u64)0x0000ff0000000000ULL) >> 24) | \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:31:12: note: expanded from macro '___constant_swab64'
31 | (((__u64)(x) & (__u64)0x00ff000000000000ULL) >> 40) | \
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:32:12: note: expanded from macro '___constant_swab64'
32 | (((__u64)(x) & (__u64)0xff00000000000000ULL) >> 56)))
| ^
>> drivers/infiniband/hw/mlx5/main.c:4242:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4242 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
16 | #define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER)
| ^ ~~~~~~
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:133:12: note: expanded from macro '__swab64'
133 | __fswab64(x))
| ^
>> drivers/infiniband/hw/mlx5/main.c:4243:6: error: use of undeclared identifier 'MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL'; did you mean 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER'?
4243 | MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER
include/linux/mlx5/mlx5_ifc.h:12493:2: note: 'MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER' declared here
12493 | MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
| ^
drivers/infiniband/hw/mlx5/main.c:4296:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4296 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/linux/stddef.h:16:32: note: expanded from macro 'offsetof'
16 | #define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER)
| ^ ~~~~~~
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:131:31: note: expanded from macro '__swab64'
131 | (__u64)(__builtin_constant_p(x) ? \
| ^
drivers/infiniband/hw/mlx5/main.c:4296:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4296 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:25:12: note: expanded from macro '___constant_swab64'
25 | (((__u64)(x) & (__u64)0x00000000000000ffULL) << 56) | \
| ^
drivers/infiniband/hw/mlx5/main.c:4296:6: error: no member named 'general_obj_types_127_64' in 'mlx5_ifc_cmd_hca_cap_2_bits'
4296 | if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mlx5/device.h:1299:2: note: expanded from macro 'MLX5_CAP_GEN_2_64'
1299 | MLX5_GET64(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
| ^ ~~~
include/linux/mlx5/device.h:121:63: note: expanded from macro 'MLX5_GET64'
121 | #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
| ^ ~~~
include/linux/mlx5/device.h:55:34: note: expanded from macro '__mlx5_64_off'
55 | #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
| ^ ~~~
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
include/uapi/linux/byteorder/little_endian.h:39:59: note: expanded from macro '__be64_to_cpu'
39 | #define __be64_to_cpu(x) __swab64((__force __u64)(__be64)(x))
| ^
include/uapi/linux/swab.h:132:21: note: expanded from macro '__swab64'
132 | ___constant_swab64(x) : \
| ^
include/uapi/linux/swab.h:26:12: note: expanded from macro '___constant_swab64'
26 | (((__u64)(x) & (__u64)0x000000000000ff00ULL) << 40) | \
| ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
vim +4207 drivers/infiniband/hw/mlx5/main.c
4204
4205 static void mlx5_ib_cleanup_ucaps(struct mlx5_ib_dev *dev)
4206 {
> 4207 if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
4208 ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
4209
4210 if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
> 4211 MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA)
4212 ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
4213 }
4214
4215 static int mlx5_ib_init_ucaps(struct mlx5_ib_dev *dev)
4216 {
4217 int ret;
4218
4219 if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL) {
4220 ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
4221 if (ret)
4222 return ret;
4223 }
4224
4225 if (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
4226 MLX5_UCTX_CAP_RDMA_CTRL_OTHER_VHCA) {
4227 ret = ib_create_ucap(RDMA_UCAP_MLX5_CTRL_OTHER_VHCA);
4228 if (ret)
4229 goto remove_local;
4230 }
4231
4232 return 0;
4233
4234 remove_local:
4235 if (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RDMA_CTRL)
4236 ib_remove_ucap(RDMA_UCAP_MLX5_CTRL_LOCAL);
4237 return ret;
4238 }
4239
4240 static void mlx5_ib_stage_caps_cleanup(struct mlx5_ib_dev *dev)
4241 {
> 4242 if (MLX5_CAP_GEN_2_64(dev->mdev, general_obj_types_127_64) &
> 4243 MLX5_HCA_CAP_2_GENERAL_OBJECT_TYPES_RDMA_CTRL)
4244 mlx5_ib_cleanup_ucaps(dev);
4245
4246 bitmap_free(dev->var_table.bitmap);
4247 }
4248
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API
2025-03-08 7:25 ` Zhu Yanjun
@ 2025-03-08 19:21 ` Leon Romanovsky
0 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-08 19:21 UTC (permalink / raw)
To: Zhu Yanjun; +Cc: Jason Gunthorpe, Chiara Meiohas, linux-rdma, Yishai Hadas
On Sat, Mar 08, 2025 at 08:25:55AM +0100, Zhu Yanjun wrote:
> 在 2025/3/6 12:51, Leon Romanovsky 写道:
> > From: Chiara Meiohas <cmeiohas@nvidia.com>
> >
> > Implement a new User CAPabilities (UCAP) API to provide fine-grained
> > control over specific firmware features.
> >
> > This approach offers more granular capabilities than the existing Linux
> > capabilities, which may be too generic for certain FW features.
> >
> > This mechanism represents each capability as a character device with
> > root read-write access. Root processes can grant users special
> > privileges by allowing access to these character devices (e.g., using
> > chown).
>
> Hi, Chiara
>
> I read this patch-set carefully. If I get this patch-set correctly, this
> patch-set introduces a new User CAPabilities API to control specific
> firmware feature.
> Do we have a user guide to use this UCAP? For example, we suspect that a
> Firmware problem will occur in production environment, how can we use this
> UCAP to debug this Firmware problem?
It is not helpful for debugging FW as these capabilities are needed when
you are opening ucontext. Your FW shouldn't get an access from one ucontext
access to others.
These UCAPs are used when administrator wants to control specific FW feature
by separating between userA(containerA) and userB(containerB), which are running
on same machine with same FW.
The better debug interface is provided by FWCTL, use it:
https://lore.kernel.org/linux-rdma/0-v5-642aa0c94070+4447f-fwctl_jgg@nvidia.com/
Thanks
>
> Thanks.
> Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
>
> Zhu Yanjun
>
> >
> > UCAP character devices are located in /dev/infiniband and the class path
> > is /sys/class/infiniband_ucaps.
> >
> > Signed-off-by: Chiara Meiohas <cmeiohas@nvidia.com>
> > Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> > Signed-off-by: Leon Romanovsky <leon@kernel.org>
> > ---
> > drivers/infiniband/core/Makefile | 3 +-
> > drivers/infiniband/core/ucaps.c | 267 ++++++++++++++++++++++++++
> > drivers/infiniband/core/uverbs_main.c | 2 +
> > include/rdma/ib_ucaps.h | 25 +++
> > 4 files changed, 296 insertions(+), 1 deletion(-)
> > create mode 100644 drivers/infiniband/core/ucaps.c
> > create mode 100644 include/rdma/ib_ucaps.h
> >
> > diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
> > index 8ab4eea5a0a5..d49ded7e95f0 100644
> > --- a/drivers/infiniband/core/Makefile
> > +++ b/drivers/infiniband/core/Makefile
> > @@ -39,6 +39,7 @@ ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
> > uverbs_std_types_async_fd.o \
> > uverbs_std_types_srq.o \
> > uverbs_std_types_wq.o \
> > - uverbs_std_types_qp.o
> > + uverbs_std_types_qp.o \
> > + ucaps.o
> > ib_uverbs-$(CONFIG_INFINIBAND_USER_MEM) += umem.o umem_dmabuf.o
> > ib_uverbs-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o
> > diff --git a/drivers/infiniband/core/ucaps.c b/drivers/infiniband/core/ucaps.c
> > new file mode 100644
> > index 000000000000..6853c6d078f9
> > --- /dev/null
> > +++ b/drivers/infiniband/core/ucaps.c
> > @@ -0,0 +1,267 @@
> > +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
> > +/*
> > + * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
> > + */
> > +
> > +#include <linux/kref.h>
> > +#include <linux/cdev.h>
> > +#include <linux/mutex.h>
> > +#include <linux/file.h>
> > +#include <linux/fs.h>
> > +#include <rdma/ib_ucaps.h>
> > +
> > +#define RDMA_UCAP_FIRST RDMA_UCAP_MLX5_CTRL_LOCAL
> > +
> > +static DEFINE_MUTEX(ucaps_mutex);
> > +static struct ib_ucap *ucaps_list[RDMA_UCAP_MAX];
> > +static bool ucaps_class_is_registered;
> > +static dev_t ucaps_base_dev;
> > +
> > +struct ib_ucap {
> > + struct cdev cdev;
> > + struct device dev;
> > + struct kref ref;
> > +};
> > +
> > +static const char *ucap_names[RDMA_UCAP_MAX] = {
> > + [RDMA_UCAP_MLX5_CTRL_LOCAL] = "mlx5_perm_ctrl_local",
> > + [RDMA_UCAP_MLX5_CTRL_OTHER_VHCA] = "mlx5_perm_ctrl_other_vhca"
> > +};
> > +
> > +static char *ucaps_devnode(const struct device *dev, umode_t *mode)
> > +{
> > + if (mode)
> > + *mode = 0600;
> > +
> > + return kasprintf(GFP_KERNEL, "infiniband/%s", dev_name(dev));
> > +}
> > +
> > +static const struct class ucaps_class = {
> > + .name = "infiniband_ucaps",
> > + .devnode = ucaps_devnode,
> > +};
> > +
> > +static const struct file_operations ucaps_cdev_fops = {
> > + .owner = THIS_MODULE,
> > + .open = simple_open,
> > +};
> > +
> > +/**
> > + * ib_cleanup_ucaps - cleanup all API resources and class.
> > + *
> > + * This is called once, when removing the ib_uverbs module.
> > + */
> > +void ib_cleanup_ucaps(void)
> > +{
> > + mutex_lock(&ucaps_mutex);
> > + if (!ucaps_class_is_registered) {
> > + mutex_unlock(&ucaps_mutex);
> > + return;
> > + }
> > +
> > + for (int i = RDMA_UCAP_FIRST; i < RDMA_UCAP_MAX; i++)
> > + WARN_ON(ucaps_list[i]);
> > +
> > + class_unregister(&ucaps_class);
> > + ucaps_class_is_registered = false;
> > + unregister_chrdev_region(ucaps_base_dev, RDMA_UCAP_MAX);
> > + mutex_unlock(&ucaps_mutex);
> > +}
> > +
> > +static int get_ucap_from_devt(dev_t devt, u64 *idx_mask)
> > +{
> > + for (int type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
> > + if (ucaps_list[type] && ucaps_list[type]->dev.devt == devt) {
> > + *idx_mask |= 1 << type;
> > + return 0;
> > + }
> > + }
> > +
> > + return -EINVAL;
> > +}
> > +
> > +static int get_devt_from_fd(unsigned int fd, dev_t *ret_dev)
> > +{
> > + struct file *file;
> > +
> > + file = fget(fd);
> > + if (!file)
> > + return -EBADF;
> > +
> > + *ret_dev = file_inode(file)->i_rdev;
> > + fput(file);
> > + return 0;
> > +}
> > +
> > +/**
> > + * ib_ucaps_init - Initialization required before ucap creation.
> > + *
> > + * Return: 0 on success, or a negative errno value on failure
> > + */
> > +static int ib_ucaps_init(void)
> > +{
> > + int ret = 0;
> > +
> > + if (ucaps_class_is_registered)
> > + return ret;
> > +
> > + ret = class_register(&ucaps_class);
> > + if (ret)
> > + return ret;
> > +
> > + ret = alloc_chrdev_region(&ucaps_base_dev, 0, RDMA_UCAP_MAX,
> > + ucaps_class.name);
> > + if (ret < 0) {
> > + class_unregister(&ucaps_class);
> > + return ret;
> > + }
> > +
> > + ucaps_class_is_registered = true;
> > +
> > + return 0;
> > +}
> > +
> > +static void ucap_dev_release(struct device *device)
> > +{
> > + struct ib_ucap *ucap = container_of(device, struct ib_ucap, dev);
> > +
> > + kfree(ucap);
> > +}
> > +
> > +/**
> > + * ib_create_ucap - Add a ucap character device
> > + * @type: UCAP type
> > + *
> > + * Creates a ucap character device in the /dev/infiniband directory. By default,
> > + * the device has root-only read-write access.
> > + *
> > + * A driver may call this multiple times with the same UCAP type. A reference
> > + * count tracks creations and deletions.
> > + *
> > + * Return: 0 on success, or a negative errno value on failure
> > + */
> > +int ib_create_ucap(enum rdma_user_cap type)
> > +{
> > + struct ib_ucap *ucap;
> > + int ret;
> > +
> > + if (type >= RDMA_UCAP_MAX)
> > + return -EINVAL;
> > +
> > + mutex_lock(&ucaps_mutex);
> > + ret = ib_ucaps_init();
> > + if (ret)
> > + goto unlock;
> > +
> > + ucap = ucaps_list[type];
> > + if (ucap) {
> > + kref_get(&ucap->ref);
> > + mutex_unlock(&ucaps_mutex);
> > + return 0;
> > + }
> > +
> > + ucap = kzalloc(sizeof(*ucap), GFP_KERNEL);
> > + if (!ucap) {
> > + ret = -ENOMEM;
> > + goto unlock;
> > + }
> > +
> > + device_initialize(&ucap->dev);
> > + ucap->dev.class = &ucaps_class;
> > + ucap->dev.devt = MKDEV(MAJOR(ucaps_base_dev), type);
> > + ucap->dev.release = ucap_dev_release;
> > + ret = dev_set_name(&ucap->dev, ucap_names[type]);
> > + if (ret)
> > + goto err_device;
> > +
> > + cdev_init(&ucap->cdev, &ucaps_cdev_fops);
> > + ucap->cdev.owner = THIS_MODULE;
> > +
> > + ret = cdev_device_add(&ucap->cdev, &ucap->dev);
> > + if (ret)
> > + goto err_device;
> > +
> > + kref_init(&ucap->ref);
> > + ucaps_list[type] = ucap;
> > + mutex_unlock(&ucaps_mutex);
> > +
> > + return 0;
> > +
> > +err_device:
> > + put_device(&ucap->dev);
> > +unlock:
> > + mutex_unlock(&ucaps_mutex);
> > + return ret;
> > +}
> > +EXPORT_SYMBOL(ib_create_ucap);
> > +
> > +static void ib_release_ucap(struct kref *ref)
> > +{
> > + struct ib_ucap *ucap = container_of(ref, struct ib_ucap, ref);
> > + enum rdma_user_cap type;
> > +
> > + for (type = RDMA_UCAP_FIRST; type < RDMA_UCAP_MAX; type++) {
> > + if (ucaps_list[type] == ucap)
> > + break;
> > + }
> > + WARN_ON(type == RDMA_UCAP_MAX);
> > +
> > + ucaps_list[type] = NULL;
> > + cdev_device_del(&ucap->cdev, &ucap->dev);
> > + put_device(&ucap->dev);
> > +}
> > +
> > +/**
> > + * ib_remove_ucap - Remove a ucap character device
> > + * @type: User cap type
> > + *
> > + * Removes the ucap character device according to type. The device is completely
> > + * removed from the filesystem when its reference count reaches 0.
> > + */
> > +void ib_remove_ucap(enum rdma_user_cap type)
> > +{
> > + struct ib_ucap *ucap;
> > +
> > + mutex_lock(&ucaps_mutex);
> > + ucap = ucaps_list[type];
> > + if (WARN_ON(!ucap))
> > + goto end;
> > +
> > + kref_put(&ucap->ref, ib_release_ucap);
> > +end:
> > + mutex_unlock(&ucaps_mutex);
> > +}
> > +EXPORT_SYMBOL(ib_remove_ucap);
> > +
> > +/**
> > + * ib_get_ucaps - Get bitmask of ucap types from file descriptors
> > + * @fds: Array of file descriptors
> > + * @fd_count: Number of file descriptors in the array
> > + * @idx_mask: Bitmask to be updated based on the ucaps in the fd list
> > + *
> > + * Given an array of file descriptors, this function returns a bitmask of
> > + * the ucaps where a bit is set if an FD for that ucap type was in the array.
> > + *
> > + * Return: 0 on success, or a negative errno value on failure
> > + */
> > +int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask)
> > +{
> > + int ret = 0;
> > + dev_t dev;
> > +
> > + *idx_mask = 0;
> > + mutex_lock(&ucaps_mutex);
> > + for (int i = 0; i < fd_count; i++) {
> > + ret = get_devt_from_fd(fds[i], &dev);
> > + if (ret)
> > + goto end;
> > +
> > + ret = get_ucap_from_devt(dev, idx_mask);
> > + if (ret)
> > + goto end;
> > + }
> > +
> > +end:
> > + mutex_unlock(&ucaps_mutex);
> > + return ret;
> > +}
> > diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
> > index 85cfc790a7bb..973fe2c7ef53 100644
> > --- a/drivers/infiniband/core/uverbs_main.c
> > +++ b/drivers/infiniband/core/uverbs_main.c
> > @@ -52,6 +52,7 @@
> > #include <rdma/ib.h>
> > #include <rdma/uverbs_std_types.h>
> > #include <rdma/rdma_netlink.h>
> > +#include <rdma/ib_ucaps.h>
> > #include "uverbs.h"
> > #include "core_priv.h"
> > @@ -1345,6 +1346,7 @@ static void __exit ib_uverbs_cleanup(void)
> > IB_UVERBS_NUM_FIXED_MINOR);
> > unregister_chrdev_region(dynamic_uverbs_dev,
> > IB_UVERBS_NUM_DYNAMIC_MINOR);
> > + ib_cleanup_ucaps();
> > mmu_notifier_synchronize();
> > }
> > diff --git a/include/rdma/ib_ucaps.h b/include/rdma/ib_ucaps.h
> > new file mode 100644
> > index 000000000000..8f0552a2b2b0
> > --- /dev/null
> > +++ b/include/rdma/ib_ucaps.h
> > @@ -0,0 +1,25 @@
> > +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
> > +/*
> > + * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
> > + */
> > +
> > +#ifndef _IB_UCAPS_H_
> > +#define _IB_UCAPS_H_
> > +
> > +#define UCAP_ENABLED(ucaps, type) (!!((ucaps) & (1U << (type))))
> > +
> > +enum rdma_user_cap {
> > + RDMA_UCAP_MLX5_CTRL_LOCAL,
> > + RDMA_UCAP_MLX5_CTRL_OTHER_VHCA,
> > + RDMA_UCAP_MAX
> > +};
> > +
> > +void ib_cleanup_ucaps(void);
> > +
> > +int ib_create_ucap(enum rdma_user_cap type);
> > +
> > +void ib_remove_ucap(enum rdma_user_cap type);
> > +
> > +int ib_get_ucaps(int *fds, int fd_count, uint64_t *idx_mask);
> > +
> > +#endif /* _IB_UCAPS_H_ */
>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
` (5 preceding siblings ...)
2025-03-06 11:51 ` [PATCH rdma-next v1 6/6] docs: infiniband: document the UCAP API Leon Romanovsky
@ 2025-03-08 19:23 ` Leon Romanovsky
6 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2025-03-08 19:23 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky
Cc: Chiara Meiohas, Jonathan Corbet, linux-doc, linux-kernel,
linux-rdma, Mark Bloch, Patrisious Haddad, Yishai Hadas
On Thu, 06 Mar 2025 13:51:25 +0200, Leon Romanovsky wrote:
> Changelog:
> v1:
> * Used kref primitives instead of open-coded variant
> * Check return value from dev_set_name()
> * Added extra brackets around type in UCAP_ENABLED macro
> v0: https://lore.kernel.org/all/cover.1740574943.git.leon@kernel.org
>
> [...]
Applied, thanks!
[1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API
https://git.kernel.org/rdma/rdma/c/054220ae51ceca
[2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities
https://git.kernel.org/rdma/rdma/c/dc0633dbb2abcc
[3/6] RDMA/uverbs: Add support for UCAPs in context creation
https://git.kernel.org/rdma/rdma/c/8108812389cf29
[4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext
https://git.kernel.org/rdma/rdma/c/984ec408e39bf8
[5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace
https://git.kernel.org/rdma/rdma/c/bec5f67b886710
[6/6] docs: infiniband: document the UCAP API
https://git.kernel.org/rdma/rdma/c/850a5ea27782a5
Best regards,
--
Leon Romanovsky <leon@kernel.org>
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2025-03-08 19:23 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-03-06 11:51 [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 1/6] RDMA/uverbs: Introduce UCAP (User CAPabilities) API Leon Romanovsky
2025-03-08 7:25 ` Zhu Yanjun
2025-03-08 19:21 ` Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 2/6] RDMA/mlx5: Create UCAP char devices for supported device capabilities Leon Romanovsky
2025-03-08 12:33 ` kernel test robot
2025-03-06 11:51 ` [PATCH rdma-next v1 3/6] RDMA/uverbs: Add support for UCAPs in context creation Leon Romanovsky
2025-03-06 11:51 ` [PATCH rdma-next v1 4/6] RDMA/mlx5: Check enabled UCAPs when creating ucontext Leon Romanovsky
2025-03-08 4:22 ` kernel test robot
2025-03-06 11:51 ` [PATCH rdma-next v1 5/6] RDMA/mlx5: Expose RDMA TRANSPORT flow table types to userspace Leon Romanovsky
2025-03-07 12:30 ` kernel test robot
2025-03-08 5:37 ` kernel test robot
2025-03-06 11:51 ` [PATCH rdma-next v1 6/6] docs: infiniband: document the UCAP API Leon Romanovsky
2025-03-08 19:23 ` [PATCH rdma-next v1 0/6] Introduce UCAP API and usage in mlx5 Leon Romanovsky
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.