[PATCH V1 rdma-core 0/6] Expanding raw packet capabilities

[PATCH V1 rdma-core 0/6] Expanding raw packet capabilities

[Yishai Hadas]

Sending V1 to better describe the API as part of commit logs and man pages,
details below.

This series from Noa is the complementary user space part
of the kernel code that was merged into 4.11.

This series unifies all existing raw packet capabilities under a new
field in ibv_device_attr_ex named raw_packet_caps. This field is defined
by the ibv_raw_packet_caps enum.

The existing IP csum and scatter FCS capabilities were added to the
enum and a new capability was introduced - cvlan stripping offload.

Cvlan stripping is the device's capability to remove the cvlan from
an incoming raw packet and provide the data in the matching work
completion.

This series also:
- Allows creation of WQ and raw Ethernet QP with cvlan stripping capability.
- Allows setting and unsetting of cvlan stripping capability for an existing WQ.
- Allows creation of WQ with scatter FCS offload.
- Allows reading the cvlan data from ibv_cq_ex.

Details for above was added as part of the commit logs and the relevant
man pages.

Pull request was sent:
https://github.com/linux-rdma/rdma-core/pull/96

Changes from V0:
- Extend commit logs to better describe the usage from user point of view.
- Per API change put the corresponding man page change in the same patch to
  clarify the usage. 
- Drop patch #6 which held the changes for all the man pages together as of
  the above change.
- Align involved man pages with current code where stuff was missing and extend
  to better describe the new functionality.


Noa Osherovich (6):
  ibverbs: Report raw packet caps as part of query device
  ibverbs: Allow creation and modification of WQ with cvlan offload
  ibverbs: Allow creation of QP with cvlan stripping offload
  ibverbs: Add an option to poll cvlan value from a CQ
  mlx5: Add read_cvlan support
  ibverbs: Add support for scatter FCS ability in WQ

 libibverbs/cmd.c                     | 29 ++++++++++++++++++++++++++++-
 libibverbs/examples/devinfo.c        | 14 ++++++++++++++
 libibverbs/kern-abi.h                |  6 +++++-
 libibverbs/man/ibv_create_cq_ex.3    |  9 +++++++--
 libibverbs/man/ibv_create_qp_ex.3    |  8 ++++++++
 libibverbs/man/ibv_create_wq.3       | 12 ++++++++++++
 libibverbs/man/ibv_modify_wq.3       |  2 ++
 libibverbs/man/ibv_query_device_ex.3 |  9 +++++++++
 libibverbs/verbs.h                   | 35 +++++++++++++++++++++++++++++++----
 providers/mlx5/cq.c                  |  9 +++++++++
 providers/mlx5/verbs.c               |  3 ++-
 11 files changed, 127 insertions(+), 9 deletions(-)

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V1 rdma-core 1/6] ibverbs: Report raw packet caps as part of query device

[Yishai Hadas]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Currently, existing raw packet capabilities (IP CSUM and scatter FCS)
are reported separately to the user via ibv_query_device_ex.

Unify those capabilities into a single enum and report them together
for a better user experience.

Also introduce CVLAN stripping offload capability. CVLAN is the
customer VLAN tag (inner tag).
CVLAN stripping offload is the device's ability to strip this tag
from incoming raw Ethernet packets and report the data in the
matching work completion.

This patch includes:
- Reading from the uverbs layer and report back to an application.
- Extending ibv_devinfo to print that information.

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 libibverbs/cmd.c                     |  8 ++++++++
 libibverbs/examples/devinfo.c        | 14 ++++++++++++++
 libibverbs/kern-abi.h                |  2 +-
 libibverbs/man/ibv_query_device_ex.3 |  9 +++++++++
 libibverbs/verbs.h                   |  7 +++++++
 5 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c
index 9b49da0..4aebbb5 100644
--- a/libibverbs/cmd.c
+++ b/libibverbs/cmd.c
@@ -230,6 +230,14 @@ int ibv_cmd_query_device_ex(struct ibv_context *context,
 			attr->max_wq_type_rq = resp->max_wq_type_rq;
 	}
 
+	if (attr_size >= offsetof(struct ibv_device_attr_ex, raw_packet_caps) +
+			 sizeof(attr->raw_packet_caps)) {
+		if (resp->response_length >=
+		    offsetof(struct ibv_query_device_resp_ex, raw_packet_caps) +
+		    sizeof(resp->raw_packet_caps))
+			attr->raw_packet_caps = resp->raw_packet_caps;
+	}
+
 	return 0;
 }
 
diff --git a/libibverbs/examples/devinfo.c b/libibverbs/examples/devinfo.c
index d88562f..42222c4 100644
--- a/libibverbs/examples/devinfo.c
+++ b/libibverbs/examples/devinfo.c
@@ -401,6 +401,17 @@ static void print_packet_pacing_caps(const struct ibv_packet_pacing_caps *caps)
 	}
 }
 
+static void print_raw_packet_caps(uint32_t raw_packet_caps)
+{
+	printf("\traw packet caps:\n");
+	if (raw_packet_caps & IBV_RAW_PACKET_CAP_CVLAN_STRIPPING)
+		printf("\t\t\t\t\tC-VLAN stripping offload\n");
+	if (raw_packet_caps & IBV_RAW_PACKET_CAP_SCATTER_FCS)
+		printf("\t\t\t\t\tScatter FCS offload\n");
+	if (raw_packet_caps & IBV_RAW_PACKET_CAP_IP_CSUM)
+		printf("\t\t\t\t\tIP csum offload\n");
+}
+
 static int print_hca_cap(struct ibv_device *ib_dev, uint8_t ib_port)
 {
 	struct ibv_context *ctx;
@@ -499,6 +510,9 @@ static int print_hca_cap(struct ibv_device *ib_dev, uint8_t ib_port)
 		else
 			printf("\tcore clock not supported\n");
 
+		if (device_attr.raw_packet_caps)
+			print_raw_packet_caps(device_attr.raw_packet_caps);
+
 		printf("\tdevice_cap_flags_ex:\t\t0x%" PRIX64 "\n", device_attr.device_cap_flags_ex);
 		print_device_cap_flags_ex(device_attr.device_cap_flags_ex);
 		print_tso_caps(&device_attr.tso_caps);
diff --git a/libibverbs/kern-abi.h b/libibverbs/kern-abi.h
index 210dd3e..3958f0c 100644
--- a/libibverbs/kern-abi.h
+++ b/libibverbs/kern-abi.h
@@ -290,7 +290,7 @@ struct ibv_query_device_resp_ex {
 	__u64 device_cap_flags_ex;
 	struct ibv_rss_caps_resp rss_caps;
 	__u32  max_wq_type_rq;
-	__u32 reserved;
+	__u32 raw_packet_caps;
 };
 
 struct ibv_query_port {
diff --git a/libibverbs/man/ibv_query_device_ex.3 b/libibverbs/man/ibv_query_device_ex.3
index c291017..fdfb708 100644
--- a/libibverbs/man/ibv_query_device_ex.3
+++ b/libibverbs/man/ibv_query_device_ex.3
@@ -32,6 +32,7 @@ struct ibv_tso_caps    tso_caps;                   /* TCP segmentation offload c
 struct ibv_rss_caps    rss_caps;                   /* RSS capabilities */
 uint32_t               max_wq_type_rq;             /* Max Work Queue from type RQ */
 struct ibv_packet_pacing_caps packet_pacing_caps; /* Packet pacing capabilities */
+uint32_t               raw_packet_caps;            /* Raw packet capabilities, use enum ibv_raw_packet_caps */
 .in -8
 };
 
@@ -75,6 +76,14 @@ struct ibv_packet_pacing_caps {
        uint32_t supported_qpts;    /* Bitmap showing which QP types are supported. */
 };
 
+enum ibv_raw_packet_caps {
+.in +8
+IBV_RAW_PACKET_CAP_CVLAN_STRIPPING	= 1 << 0, /* CVLAN stripping is supported */
+IBV_RAW_PACKET_CAP_SCATTER_FCS		= 1 << 1, /* FCS scattering is supported */
+IBV_RAW_PACKET_CAP_IP_CSUM		= 1 << 2, /* IP CSUM offload is supported */
+.in -8
+};
+
 .fi
 .SH "RETURN VALUE"
 .B ibv_query_device_ex()
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index 25f4ede..604b09e 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h
@@ -252,6 +252,12 @@ struct ibv_packet_pacing_caps {
 	uint32_t supported_qpts;
 };
 
+enum ibv_raw_packet_caps {
+	IBV_RAW_PACKET_CAP_CVLAN_STRIPPING	= 1 << 0,
+	IBV_RAW_PACKET_CAP_SCATTER_FCS		= 1 << 1,
+	IBV_RAW_PACKET_CAP_IP_CSUM		= 1 << 2,
+};
+
 struct ibv_device_attr_ex {
 	struct ibv_device_attr	orig_attr;
 	uint32_t		comp_mask;
@@ -263,6 +269,7 @@ struct ibv_device_attr_ex {
 	struct ibv_rss_caps     rss_caps;
 	uint32_t		max_wq_type_rq;
 	struct ibv_packet_pacing_caps packet_pacing_caps;
+	uint32_t		raw_packet_caps; /* Use ibv_raw_packet_caps */
 };
 
 enum ibv_mtu {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V1 rdma-core 2/6] ibverbs: Allow creation and modification of WQ with cvlan offload

[Yishai Hadas]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Enable WQ creation and modification with cvlan stripping offload.
This includes:
- Adding flags and flags mask fields to ibv_wq_init_attr.
- Similarly extend ibv_wq_attr to allow setting and unsetting this
  offload during ibv_modify_wq.

Creation of a WQ with cvlan offload is done by setting the following
fields of the ibv_wq_init_attr struct:
- Setting the IBV_WQ_FLAGS_CVLAN_STRIPPING bit of the create_flags
  field.
- Setting the IBV_WQ_INIT_ATTR_FLAGS bit of the comp_mask field.

Modification of the cvlan stripping property is done by setting the
following fields of the ibv_wq_attr struct:
- Setting IBV_WQ_ATTR_FLAGS bit of the attr_mask field.
- Setting the IBV_RAW_PACKET_CAP_CVLAN_STRIPPING bit of the
  flags_mask field.
- Setting or unsetting the IBV_RAW_PACKET_CAP_CVLAN_STRIPPING bit of
  the flags field.

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 libibverbs/cmd.c               | 18 ++++++++++++++++++
 libibverbs/kern-abi.h          |  4 ++++
 libibverbs/man/ibv_create_wq.3 | 11 +++++++++++
 libibverbs/man/ibv_modify_wq.3 |  2 ++
 libibverbs/verbs.h             | 14 ++++++++++++--
 5 files changed, 47 insertions(+), 2 deletions(-)

diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c
index 4aebbb5..b8fe76d 100644
--- a/libibverbs/cmd.c
+++ b/libibverbs/cmd.c
@@ -1894,6 +1894,15 @@ int ibv_cmd_create_wq(struct ibv_context *context,
 	cmd->max_wr = wq_init_attr->max_wr;
 	cmd->comp_mask = 0;
 
+	if (cmd_core_size >= offsetof(struct ibv_create_wq, create_flags) +
+	    sizeof(cmd->create_flags)) {
+		if (wq_init_attr->comp_mask & IBV_WQ_INIT_ATTR_FLAGS) {
+			if (wq_init_attr->create_flags & ~(IBV_WQ_FLAGS_RESERVED - 1))
+				return EOPNOTSUPP;
+			cmd->create_flags = wq_init_attr->create_flags;
+		}
+	}
+
 	err = write(context->cmd_fd, cmd, cmd_size);
 	if (err != cmd_size)
 		return errno;
@@ -1927,6 +1936,15 @@ int ibv_cmd_modify_wq(struct ibv_wq *wq, struct ibv_wq_attr *attr,
 
 	cmd->curr_wq_state = attr->curr_wq_state;
 	cmd->wq_state = attr->wq_state;
+	if (cmd_core_size >= offsetof(struct ibv_modify_wq, flags_mask) +
+	    sizeof(cmd->flags_mask)) {
+		if (attr->attr_mask & IBV_WQ_ATTR_FLAGS) {
+			if (attr->flags_mask & ~(IBV_WQ_FLAGS_RESERVED - 1))
+				return EOPNOTSUPP;
+			cmd->flags = attr->flags;
+			cmd->flags_mask = attr->flags_mask;
+		}
+	}
 	cmd->wq_handle = wq->handle;
 	cmd->attr_mask = attr->attr_mask;
 
diff --git a/libibverbs/kern-abi.h b/libibverbs/kern-abi.h
index 3958f0c..72a16b6 100644
--- a/libibverbs/kern-abi.h
+++ b/libibverbs/kern-abi.h
@@ -1249,6 +1249,8 @@ struct ibv_create_wq {
 	__u32 cq_handle;
 	__u32 max_wr;
 	__u32 max_sge;
+	__u32 create_flags;
+	__u32 reserved;
 };
 
 struct ibv_create_wq_resp {
@@ -1279,6 +1281,8 @@ struct ibv_modify_wq  {
 	__u32 wq_handle;
 	__u32 wq_state;
 	__u32 curr_wq_state;
+	__u32 flags;
+	__u32 flags_mask;
 };
 
 struct ibv_create_rwq_ind_table {
diff --git a/libibverbs/man/ibv_create_wq.3 b/libibverbs/man/ibv_create_wq.3
index aad6741..9a541fe 100644
--- a/libibverbs/man/ibv_create_wq.3
+++ b/libibverbs/man/ibv_create_wq.3
@@ -31,8 +31,19 @@ uint32_t                   max_sge;       /* Requested max number of scatter/gat
 struct  ibv_pd            *pd;            /* PD to be associated with the WQ */
 struct  ibv_cq            *cq;            /* CQ to be associated with the WQ */
 uint32_t                   comp_mask;     /* Identifies valid fields. Use ibv_wq_init_attr_mask */
+uint32_t                   create_flags    /* Creation flags for this WQ, use enum ibv_wq_flags */
 .in -8
 };
+
+.sp
+.nf
+enum ibv_wq_flags {
+.in +8
+IBV_WQ_FLAGS_CVLAN_STRIPPING		= 1 << 0, /* CVLAN field will be stripped from incoming packets */
+IBV_WQ_FLAGS_RESERVED			= 1 << 1,
+.in -8
+};
+.nf
 .fi
 .PP
 The function
diff --git a/libibverbs/man/ibv_modify_wq.3 b/libibverbs/man/ibv_modify_wq.3
index f17faed..1972ec2 100644
--- a/libibverbs/man/ibv_modify_wq.3
+++ b/libibverbs/man/ibv_modify_wq.3
@@ -26,6 +26,8 @@ struct ibv_wq_attr {
 uint32_t                attr_mask;     /* Use enum ibv_wq_attr_mask */
 enum ibv_wq_state       wq_state;      /* Move to this state */
 enum ibv_wq_state       curr_wq_state; /* Assume this is the current state */
+uint32_t                flags;         /* Flags values to modify, use enum ibv_wq_flags */
+uint32_t                flags_mask;    /* Which flags to modify, use enum ibv_wq_flags */
 .in -8
 };
 .fi
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index 604b09e..3398566 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h
@@ -667,7 +667,13 @@ enum ibv_wq_type {
 };
 
 enum ibv_wq_init_attr_mask {
-	IBV_WQ_INIT_ATTR_RESERVED	= 1 << 0,
+	IBV_WQ_INIT_ATTR_FLAGS		= 1 << 0,
+	IBV_WQ_INIT_ATTR_RESERVED	= 1 << 1,
+};
+
+enum ibv_wq_flags {
+	IBV_WQ_FLAGS_CVLAN_STRIPPING		= 1 << 0,
+	IBV_WQ_FLAGS_RESERVED			= 1 << 1,
 };
 
 struct ibv_wq_init_attr {
@@ -678,6 +684,7 @@ struct ibv_wq_init_attr {
 	struct	ibv_pd	       *pd;
 	struct	ibv_cq	       *cq;
 	uint32_t		comp_mask;
+	uint32_t		create_flags; /* use ibv_wq_flags */
 };
 
 enum ibv_wq_state {
@@ -690,7 +697,8 @@ enum ibv_wq_state {
 enum ibv_wq_attr_mask {
 	IBV_WQ_ATTR_STATE	= 1 << 0,
 	IBV_WQ_ATTR_CURR_STATE	= 1 << 1,
-	IBV_WQ_ATTR_RESERVED	= 1 << 2
+	IBV_WQ_ATTR_FLAGS	= 1 << 2,
+	IBV_WQ_ATTR_RESERVED	= 1 << 3,
 };
 
 struct ibv_wq_attr {
@@ -700,6 +708,8 @@ struct ibv_wq_attr {
 	enum	ibv_wq_state	wq_state;
 	/* Assume this is the current WQ state */
 	enum	ibv_wq_state	curr_wq_state;
+	uint32_t		flags; /* Use ibv_wq_flags */
+	uint32_t		flags_mask; /* Use ibv_wq_flags */
 };
 
 /*
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V1 rdma-core 3/6] ibverbs: Allow creation of QP with cvlan stripping offload

[Yishai Hadas]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Allow users to create a QP that uses cvlan stripping capabilities if
supported by the hardware.

Setting cvlan stripping offload will cause the device to strip the
cvlan from incoming raw Ethernet packets and provide its data in the
matching work completion.

In addition, aligned ibv_create_qp_ex's man page with current code
(added ibv_qp_create_flags enum).

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 libibverbs/cmd.c                  | 3 ++-
 libibverbs/man/ibv_create_qp_ex.3 | 8 ++++++++
 libibverbs/verbs.h                | 1 +
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c
index b8fe76d..06ec671 100644
--- a/libibverbs/cmd.c
+++ b/libibverbs/cmd.c
@@ -926,7 +926,8 @@ static void create_qp_handle_resp_common(struct ibv_context *context,
 
 enum {
 	CREATE_QP_EX2_SUP_CREATE_FLAGS = IBV_QP_CREATE_BLOCK_SELF_MCAST_LB |
-		IBV_QP_CREATE_SCATTER_FCS,
+					 IBV_QP_CREATE_SCATTER_FCS |
+					 IBV_QP_CREATE_CVLAN_STRIPPING,
 };
 
 int ibv_cmd_create_qp_ex2(struct ibv_context *context,
diff --git a/libibverbs/man/ibv_create_qp_ex.3 b/libibverbs/man/ibv_create_qp_ex.3
index c778d15..99ae497 100644
--- a/libibverbs/man/ibv_create_qp_ex.3
+++ b/libibverbs/man/ibv_create_qp_ex.3
@@ -52,6 +52,14 @@ uint32_t                max_inline_data;/* Requested max number of data (bytes)
 .in -8
 };
 .nf
+enum ibv_qp_create_flags {
+.in +8
+IBV_QP_CREATE_BLOCK_SELF_MCAST_LB       = 1 << 1, /* Prevent self multicast loopback */
+IBV_QP_CREATE_SCATTER_FCS               = 1 << 8, /* FCS field will be scattered to host memory */
+IBV_QP_CREATE_CVLAN_STRIPPING           = 1 << 9, /* CVLAN field will be stripped from incoming packets */
+.in -8
+};
+.nf
 struct ibv_rx_hash_conf {
 .in +8
 uint8_t                rx_hash_function;       /* RX hash function, use enum ibv_rx_hash_function_flags */
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index 3398566..c9084ea 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h
@@ -780,6 +780,7 @@ enum ibv_qp_init_attr_mask {
 enum ibv_qp_create_flags {
 	IBV_QP_CREATE_BLOCK_SELF_MCAST_LB	= 1 << 1,
 	IBV_QP_CREATE_SCATTER_FCS		= 1 << 8,
+	IBV_QP_CREATE_CVLAN_STRIPPING		= 1 << 9,
 };
 
 struct ibv_rx_hash_conf {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V1 rdma-core 4/6] ibverbs: Add an option to poll cvlan value from a CQ

[Yishai Hadas]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

When a WQ or a QP is created with cvlan stripping option, it is
stripped from the incoming packet and included in the work
completion.
Extend the poll_cq_ex mechanism with a function that reads the
stripped cvlan value from the work completion.

In addition, as a part of introcuding the new functionality in the
man page, align it with current code (typo fix, add a missing field).

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih@mellanox>
---
 libibverbs/man/ibv_create_cq_ex.3 |  9 +++++++--
 libibverbs/verbs.h                | 10 +++++++++-
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/libibverbs/man/ibv_create_cq_ex.3 b/libibverbs/man/ibv_create_cq_ex.3
index a6ae769..020bbb4 100644
--- a/libibverbs/man/ibv_create_cq_ex.3
+++ b/libibverbs/man/ibv_create_cq_ex.3
@@ -17,10 +17,10 @@ creates a completion queue (CQ) for RDMA device context
 .I context\fR.
 The argument
 .I cq_attr
-is a pointer to struct ibv_create_cq_attr_ex as defined in <infiniband/verbs.h>.
+is a pointer to struct ibv_cq_init_attr_ex as defined in <infiniband/verbs.h>.
 .PP
 .nf
-struct ibv_create_cq_attr_ex {
+struct ibv_cq_init_attr_ex {
 .in +8
 int                     cqe;               /* Minimum number of entries required for CQ */
 void                    *cq_context;       /* Consumer-supplied context returned for completion events */
@@ -28,6 +28,7 @@ struct ibv_comp_channel *channel;          /* Completion channel where completio
 int                     comp_vector;       /* Completion vector used to signal completion events. Must be >= 0 and < context->num_comp_vectors. */
 uint64_t                wc_flags;          /* The wc_flags that should be returned in ibv_poll_cq_ex. Or'ed bit of enum ibv_wc_flags_ex. */
 uint32_t                comp_mask;         /* compatibility mask (extended verb). */
+uint32_t                flags              /* One or more flags from enum ibv_create_cq_attr_flags */
 .in -8
 };
 
@@ -40,6 +41,7 @@ enum ibv_wc_flags_ex {
         IBV_WC_EX_WITH_SL                    = 1 << 5,  /* Require sl in WC */
         IBV_WC_EX_WITH_DLID_PATH_BITS        = 1 << 6,  /* Require dlid path bits in WC */
         IBV_WC_EX_WITH_COMPLETION_TIMESTAMP  = 1 << 7,  /* Require completion timestamp in WC /*
+        IBV_WC_EX_WITH_CVLAN                 = 1 << 8,  /* Require VLAN info in WC */
 };
 
 enum ibv_cq_init_attr_mask {
@@ -128,6 +130,9 @@ Below members and functions are used in order to poll the current completion. Th
 .BI "uint64_t ibv_wc_read_completion_ts(struct ibv_cq_ex " "*cq"); \c
  Get the completion timestamp from the current completion.
 
+.BI "uint16_t ibv_wc_read_cvlan(struct ibv_cq_ex " "*cq"); \c
+ Get the CVLAN field from the current completion.
+
 .SH "RETURN VALUE"
 .B ibv_create_cq_ex()
 returns a pointer to the CQ, or NULL if the request fails.
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index c9084ea..15e93b3 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h
@@ -435,6 +435,7 @@ enum ibv_create_cq_wc_flags {
 	IBV_WC_EX_WITH_SL		= 1 << 5,
 	IBV_WC_EX_WITH_DLID_PATH_BITS	= 1 << 6,
 	IBV_WC_EX_WITH_COMPLETION_TIMESTAMP	= 1 << 7,
+	IBV_WC_EX_WITH_CVLAN		= 1 << 8,
 };
 
 enum {
@@ -449,7 +450,8 @@ enum {
 
 enum {
 	IBV_CREATE_CQ_SUP_WC_FLAGS = IBV_WC_STANDARD_FLAGS |
-				IBV_WC_EX_WITH_COMPLETION_TIMESTAMP
+				IBV_WC_EX_WITH_COMPLETION_TIMESTAMP |
+				IBV_WC_EX_WITH_CVLAN
 };
 
 enum ibv_wc_flags {
@@ -1092,6 +1094,7 @@ struct ibv_cq_ex {
 	uint8_t (*read_sl)(struct ibv_cq_ex *current);
 	uint8_t (*read_dlid_path_bits)(struct ibv_cq_ex *current);
 	uint64_t (*read_completion_ts)(struct ibv_cq_ex *current);
+	uint16_t (*read_cvlan)(struct ibv_cq_ex *current);
 };
 
 static inline struct ibv_cq *ibv_cq_ex_to_cq(struct ibv_cq_ex *cq)
@@ -1170,6 +1173,11 @@ static inline uint64_t ibv_wc_read_completion_ts(struct ibv_cq_ex *cq)
 	return cq->read_completion_ts(cq);
 }
 
+static inline uint16_t ibv_wc_read_cvlan(struct ibv_cq_ex *cq)
+{
+	return cq->read_cvlan(cq);
+}
+
 static inline int ibv_post_wq_recv(struct ibv_wq *wq,
 				   struct ibv_recv_wr *recv_wr,
 				   struct ibv_recv_wr **bad_recv_wr)
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V1 rdma-core 5/6] mlx5: Add read_cvlan support

[Yishai Hadas]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

When a WQ or a QP is created with cvlan stripping option, the cvlan
is stripped from the packet by the hardware and included in the work
completion.

Implement ibv_cq_ex's member function read_cvlan to allow reading the
cvlan from the work completion.

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Maor Gottlieb <maorg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 providers/mlx5/cq.c    | 9 +++++++++
 providers/mlx5/verbs.c | 3 ++-
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/providers/mlx5/cq.c b/providers/mlx5/cq.c
index 4ecd482..85d0c33 100644
--- a/providers/mlx5/cq.c
+++ b/providers/mlx5/cq.c
@@ -1189,6 +1189,13 @@ static inline uint64_t mlx5_cq_read_wc_completion_ts(struct ibv_cq_ex *ibcq)
 	return be64toh(cq->cqe64->timestamp);
 }
 
+static inline uint16_t mlx5_cq_read_wc_cvlan(struct ibv_cq_ex *ibcq)
+{
+	struct mlx5_cq *cq = to_mcq(ibv_cq_ex_to_cq(ibcq));
+
+	return be16toh(cq->cqe64->vlan_info);
+}
+
 #define BIT(i) (1UL << (i))
 
 #define SINGLE_THREADED BIT(0)
@@ -1261,6 +1268,8 @@ void mlx5_cq_fill_pfns(struct mlx5_cq *cq, const struct ibv_cq_init_attr_ex *cq_
 		cq->ibv_cq.read_dlid_path_bits = mlx5_cq_read_wc_dlid_path_bits;
 	if (cq_attr->wc_flags & IBV_WC_EX_WITH_COMPLETION_TIMESTAMP)
 		cq->ibv_cq.read_completion_ts = mlx5_cq_read_wc_completion_ts;
+	if (cq_attr->wc_flags & IBV_WC_EX_WITH_CVLAN)
+		cq->ibv_cq.read_cvlan = mlx5_cq_read_wc_cvlan;
 }
 
 int mlx5_arm_cq(struct ibv_cq *ibvcq, int solicited)
diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c
index 4d8f26c..67f9748 100644
--- a/providers/mlx5/verbs.c
+++ b/providers/mlx5/verbs.c
@@ -327,7 +327,8 @@ static int qp_sig_enabled(void)
 
 enum {
 	CREATE_CQ_SUPPORTED_WC_FLAGS = IBV_WC_STANDARD_FLAGS	|
-				       IBV_WC_EX_WITH_COMPLETION_TIMESTAMP
+				       IBV_WC_EX_WITH_COMPLETION_TIMESTAMP |
+				       IBV_WC_EX_WITH_CVLAN
 };
 
 enum {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V1 rdma-core 6/6] ibverbs: Add support for scatter FCS ability in WQ

[Yishai Hadas]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Enable the user to create a workqueue object with the scatter FCS
offload.
If this option is set, the FCS (Frame Check Sequence) field of an
incoming raw Ethernet packet will be scattered into host memory along
the packet payload as it appeared on the wire.

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 libibverbs/man/ibv_create_wq.3 | 3 ++-
 libibverbs/verbs.h             | 5 +++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/libibverbs/man/ibv_create_wq.3 b/libibverbs/man/ibv_create_wq.3
index 9a541fe..4a8b83c 100644
--- a/libibverbs/man/ibv_create_wq.3
+++ b/libibverbs/man/ibv_create_wq.3
@@ -40,7 +40,8 @@ uint32_t                   create_flags    /* Creation flags for this WQ, use en
 enum ibv_wq_flags {
 .in +8
 IBV_WQ_FLAGS_CVLAN_STRIPPING		= 1 << 0, /* CVLAN field will be stripped from incoming packets */
-IBV_WQ_FLAGS_RESERVED			= 1 << 1,
+IBV_WQ_FLAGS_SCATTER_FCS		= 1 << 1, /* FCS field will be scattered to host memory */
+IBV_WQ_FLAGS_RESERVED			= 1 << 2,
 .in -8
 };
 .nf
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index 15e93b3..3f5e9fe 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h
@@ -675,7 +675,8 @@ enum ibv_wq_init_attr_mask {
 
 enum ibv_wq_flags {
 	IBV_WQ_FLAGS_CVLAN_STRIPPING		= 1 << 0,
-	IBV_WQ_FLAGS_RESERVED			= 1 << 1,
+	IBV_WQ_FLAGS_SCATTER_FCS		= 1 << 1,
+	IBV_WQ_FLAGS_RESERVED			= 1 << 2,
 };
 
 struct ibv_wq_init_attr {
@@ -685,7 +686,7 @@ struct ibv_wq_init_attr {
 	uint32_t		max_sge;
 	struct	ibv_pd	       *pd;
 	struct	ibv_cq	       *cq;
-	uint32_t		comp_mask;
+	uint32_t		comp_mask; /* Use ibv_wq_init_attr_mask */
 	uint32_t		create_flags; /* use ibv_wq_flags */
 };
 
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V1 rdma-core 0/6] Expanding raw packet capabilities

[Doug Ledford]

On Wed, 2017-03-15 at 17:27 +0200, Yishai Hadas wrote:
> Sending V1 to better describe the API as part of commit logs and man
> pages,
> details below.
> 
> This series from Noa is the complementary user space part
> of the kernel code that was merged into 4.11.
> 
> This series unifies all existing raw packet capabilities under a new
> field in ibv_device_attr_ex named raw_packet_caps. This field is
> defined
> by the ibv_raw_packet_caps enum.
> 
> The existing IP csum and scatter FCS capabilities were added to the
> enum and a new capability was introduced - cvlan stripping offload.
> 
> Cvlan stripping is the device's capability to remove the cvlan from
> an incoming raw packet and provide the data in the matching work
> completion.
> 
> This series also:
> - Allows creation of WQ and raw Ethernet QP with cvlan stripping
> capability.
> - Allows setting and unsetting of cvlan stripping capability for an
> existing WQ.
> - Allows creation of WQ with scatter FCS offload.
> - Allows reading the cvlan data from ibv_cq_ex.
> 
> Details for above was added as part of the commit logs and the
> relevant
> man pages.
> 
> Pull request was sent:
> https://github.com/linux-rdma/rdma-core/pull/96
> 
> Changes from V0:
> - Extend commit logs to better describe the usage from user point of
> view.
> - Per API change put the corresponding man page change in the same
> patch to
>   clarify the usage. 
> - Drop patch #6 which held the changes for all the man pages together
> as of
>   the above change.
> - Align involved man pages with current code where stuff was missing
> and extend
>   to better describe the new functionality.

Thanks, version 1 of the series merged.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html