iser patches for kernel 4.1 [v3]

iser patches for kernel 4.1 [v3]

[Sagi Grimberg]

Hi Roland,

This set contains bug fixes as well as code refactoring patches

- Patches 1-3:	   Bug fixes (stable material)
- Patches 4-6:     Bounce buffer related code cleanups
- Patches 7-12,16: Refactoring
- Patches 13-15:   Minor optimizations
- Patch 17:        Version bump
- Patch 18:        Bounce buffer rewrite to handle situations where
                   large transfers are unaligned causing iser to
		   use high order allocations

Changes from v2:
- Fix regression in DIX mode (introduced in patch 15) by incorrectly
  setting the protection information scatter-entry.

Changes from v1:
- Addressed review comments (patch 18) from Alex Lyakas on bounce buffer
  re-write code (kmap/kunmap mismatch, possible use-after-free,
  and double free).

Sagi Grimberg (18):
  IB/iser: Fix unload during ep_poll wrong dereference
  IB/iser: Handle fastreg/local_inv completion errors
  IB/iser: Fix wrong calculation of protection buffer length
  IB/iser: Remove redundant cmd_data_len calculation
  IB/iser: Remove a redundant struct iser_data_buf
  IB/iser: Don't pass ib_device to fall_to_bounce_buff routine
  IB/iser: Move memory reg/dereg routines to iser_memory.c
  IB/iser: Remove redundant assignments in iser_reg_page_vec
  IB/iser: Get rid of struct iser_rdma_regd
  IB/iser: Merge build page-vec into register page-vec
  IB/iser: Move fastreg descriptor pool get/put to helper functions
  IB/iser: Move PI context alloc/free to routines
  IB/iser: Make fastreg pool cache friendly
  IB/iser: Modify struct iser_mem_reg members
  IB/iser: Pass struct iser_mem_reg to iser_fast_reg_mr and
    iser_reg_sig_mr
  IB/iser: Remove code duplication for a single DMA entry
  IB/iser: Bump version to 1.6
  IB/iser: Rewrite bounce buffer code path

 drivers/infiniband/ulp/iser/iscsi_iser.h     |   66 +---
 drivers/infiniband/ulp/iser/iser_initiator.c |   66 ++--
 drivers/infiniband/ulp/iser/iser_memory.c    |  523 +++++++++++++++-----------
 drivers/infiniband/ulp/iser/iser_verbs.c     |  220 ++++--------
 4 files changed, 431 insertions(+), 444 deletions(-)

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 01/18] IB/iser: Fix unload during ep_poll wrong dereference

[Sagi Grimberg]

In case the user unloaded ib_iser while ep_connect is in
progress, we need to destroy the endpoint although ep_disconnect
wasn't invoked (we detect this by the iser conn state != DOWN).
However, if we got an REJECTED/UNREACHABLE CM event we move the
connection state to DOWN which will prevent us from destroying
the endpoint in the module unload stage. Fix this by setting the
connection state to TERMINATING in iser_conn_error so we can still
destroy the endpoint at unload stage.

Reported-by: Ariel Nahum <arieln-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_verbs.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 4065abe..070c5af 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -721,7 +721,7 @@ static void iser_connect_error(struct rdma_cm_id *cma_id)
 	struct iser_conn *iser_conn;
 
 	iser_conn = (struct iser_conn *)cma_id->context;
-	iser_conn->state = ISER_CONN_DOWN;
+	iser_conn->state = ISER_CONN_TERMINATING;
 }
 
 /**
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 02/18] IB/iser: Handle fastreg/local_inv completion errors

[Sagi Grimberg]

Fast registration and local invalidate work requests can
also fail. We should call error completion handler for them.

Reported-by: Roi Dayan <roid-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_verbs.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 070c5af..7ee4926 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -1210,6 +1210,9 @@ iser_handle_comp_error(struct ib_conn *ib_conn,
 			iscsi_conn_failure(iser_conn->iscsi_conn,
 					   ISCSI_ERR_CONN_FAILED);
 
+	if (wc->wr_id == ISER_FASTREG_LI_WRID)
+		return;
+
 	if (is_iser_tx_desc(iser_conn, wr_id)) {
 		struct iser_tx_desc *desc = wr_id;
 
@@ -1254,13 +1257,11 @@ static void iser_handle_wc(struct ib_wc *wc)
 		else
 			iser_dbg("flush error: wr id %llx\n", wc->wr_id);
 
-		if (wc->wr_id != ISER_FASTREG_LI_WRID &&
-		    wc->wr_id != ISER_BEACON_WRID)
-			iser_handle_comp_error(ib_conn, wc);
-
-		/* complete in case all flush errors were consumed */
 		if (wc->wr_id == ISER_BEACON_WRID)
+			/* all flush errors were consumed */
 			complete(&ib_conn->flush_comp);
+		else
+			iser_handle_comp_error(ib_conn, wc);
 	}
 }
 
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 03/18] IB/iser: Fix wrong calculation of protection buffer length

[Sagi Grimberg]

This length miss-calculation may cause a silent data corruption
in the DIX case and cause the device to reference unmapped area.

Fixes: d77e65350f2d ('libiscsi, iser: Adjust data_length to include protection information')
Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_initiator.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c b/drivers/infiniband/ulp/iser/iser_initiator.c
index 20e859a..76eb57b 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -409,8 +409,8 @@ int iser_send_command(struct iscsi_conn *conn,
 	if (scsi_prot_sg_count(sc)) {
 		prot_buf->buf  = scsi_prot_sglist(sc);
 		prot_buf->size = scsi_prot_sg_count(sc);
-		prot_buf->data_len = data_buf->data_len >>
-				     ilog2(sc->device->sector_size) * 8;
+		prot_buf->data_len = (data_buf->data_len >>
+				     ilog2(sc->device->sector_size)) * 8;
 	}
 
 	if (hdr->flags & ISCSI_FLAG_CMD_READ) {
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 04/18] IB/iser: Remove redundant cmd_data_len calculation

[Sagi Grimberg]

This code was added before we had protection data length
calculation (in iser_send_command), so we needed to calc
the sg data length from the sg itself. This is not needed
anymore.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_memory.c |    5 +----
 1 files changed, 1 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 341040b..32ccd5c 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -53,12 +53,9 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 	struct scatterlist *sgl = (struct scatterlist *)data->buf;
 	struct scatterlist *sg;
 	char *mem = NULL;
-	unsigned long  cmd_data_len = 0;
+	unsigned long  cmd_data_len = data->data_len;
 	int dma_nents, i;
 
-	for_each_sg(sgl, sg, data->size, i)
-		cmd_data_len += ib_sg_dma_len(dev, sg);
-
 	if (cmd_data_len > ISER_KMALLOC_THRESHOLD)
 		mem = (void *)__get_free_pages(GFP_ATOMIC,
 		      ilog2(roundup_pow_of_two(cmd_data_len)) - PAGE_SHIFT);
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 05/18] IB/iser: Remove a redundant struct iser_data_buf

[Sagi Grimberg]

No need to keep two iser_data_buf structures just in case we use
mem copy. We can avoid that just by adding a pointer to the original
sg. So keep only two iser_data_buf per command (data and protection)
and pass the relevant data_buf to bounce buffer routine.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h     |   12 ++---
 drivers/infiniband/ulp/iser/iser_initiator.c |   16 +++----
 drivers/infiniband/ulp/iser/iser_memory.c    |   58 ++++++++++---------------
 3 files changed, 34 insertions(+), 52 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index b47aea1..5c7036c 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -218,20 +218,23 @@ enum iser_data_dir {
 /**
  * struct iser_data_buf - iSER data buffer
  *
- * @buf:          pointer to the sg list
+ * @sg:           pointer to the sg list
  * @size:         num entries of this sg
  * @data_len:     total beffer byte len
  * @dma_nents:    returned by dma_map_sg
  * @copy_buf:     allocated copy buf for SGs unaligned
  *                for rdma which are copied
+ * @orig_sg:      pointer to the original sg list (in case
+ *                we used a copy)
  * @sg_single:    SG-ified clone of a non SG SC or
  *                unaligned SG
  */
 struct iser_data_buf {
-	void               *buf;
+	struct scatterlist *sg;
 	unsigned int       size;
 	unsigned long      data_len;
 	unsigned int       dma_nents;
+	struct scatterlist *orig_sg;
 	char               *copy_buf;
 	struct scatterlist sg_single;
   };
@@ -536,9 +539,7 @@ struct iser_conn {
  * @dir:              iser data direction
  * @rdma_regd:        task rdma registration desc
  * @data:             iser data buffer desc
- * @data_copy:        iser data copy buffer desc (bounce buffer)
  * @prot:             iser protection buffer desc
- * @prot_copy:        iser protection copy buffer desc (bounce buffer)
  */
 struct iscsi_iser_task {
 	struct iser_tx_desc          desc;
@@ -549,9 +550,7 @@ struct iscsi_iser_task {
 	int                          dir[ISER_DIRS_NUM];
 	struct iser_regd_buf         rdma_regd[ISER_DIRS_NUM];
 	struct iser_data_buf         data[ISER_DIRS_NUM];
-	struct iser_data_buf         data_copy[ISER_DIRS_NUM];
 	struct iser_data_buf         prot[ISER_DIRS_NUM];
-	struct iser_data_buf         prot_copy[ISER_DIRS_NUM];
 };
 
 struct iser_page_vec {
@@ -621,7 +620,6 @@ void iser_free_rx_descriptors(struct iser_conn *iser_conn);
 
 void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 				     struct iser_data_buf *mem,
-				     struct iser_data_buf *mem_copy,
 				     enum iser_data_dir cmd_dir);
 
 int  iser_reg_rdma_mem_fmr(struct iscsi_iser_task *task,
diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c b/drivers/infiniband/ulp/iser/iser_initiator.c
index 76eb57b..0e414db 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -401,13 +401,13 @@ int iser_send_command(struct iscsi_conn *conn,
 	}
 
 	if (scsi_sg_count(sc)) { /* using a scatter list */
-		data_buf->buf  = scsi_sglist(sc);
+		data_buf->sg = scsi_sglist(sc);
 		data_buf->size = scsi_sg_count(sc);
 	}
 	data_buf->data_len = scsi_bufflen(sc);
 
 	if (scsi_prot_sg_count(sc)) {
-		prot_buf->buf  = scsi_prot_sglist(sc);
+		prot_buf->sg  = scsi_prot_sglist(sc);
 		prot_buf->size = scsi_prot_sg_count(sc);
 		prot_buf->data_len = (data_buf->data_len >>
 				     ilog2(sc->device->sector_size)) * 8;
@@ -674,35 +674,31 @@ void iser_task_rdma_finalize(struct iscsi_iser_task *iser_task)
 	/* if we were reading, copy back to unaligned sglist,
 	 * anyway dma_unmap and free the copy
 	 */
-	if (iser_task->data_copy[ISER_DIR_IN].copy_buf != NULL) {
+	if (iser_task->data[ISER_DIR_IN].copy_buf) {
 		is_rdma_data_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->data[ISER_DIR_IN],
-						&iser_task->data_copy[ISER_DIR_IN],
 						ISER_DIR_IN);
 	}
 
-	if (iser_task->data_copy[ISER_DIR_OUT].copy_buf != NULL) {
+	if (iser_task->data[ISER_DIR_OUT].copy_buf) {
 		is_rdma_data_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->data[ISER_DIR_OUT],
-						&iser_task->data_copy[ISER_DIR_OUT],
 						ISER_DIR_OUT);
 	}
 
-	if (iser_task->prot_copy[ISER_DIR_IN].copy_buf != NULL) {
+	if (iser_task->prot[ISER_DIR_IN].copy_buf) {
 		is_rdma_prot_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->prot[ISER_DIR_IN],
-						&iser_task->prot_copy[ISER_DIR_IN],
 						ISER_DIR_IN);
 	}
 
-	if (iser_task->prot_copy[ISER_DIR_OUT].copy_buf != NULL) {
+	if (iser_task->prot[ISER_DIR_OUT].copy_buf) {
 		is_rdma_prot_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->prot[ISER_DIR_OUT],
-						&iser_task->prot_copy[ISER_DIR_OUT],
 						ISER_DIR_OUT);
 	}
 
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 32ccd5c..beeabd0 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -46,11 +46,10 @@
  */
 static int iser_start_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 					struct iser_data_buf *data,
-					struct iser_data_buf *data_copy,
 					enum iser_data_dir cmd_dir)
 {
 	struct ib_device *dev = iser_task->iser_conn->ib_conn.device->ib_device;
-	struct scatterlist *sgl = (struct scatterlist *)data->buf;
+	struct scatterlist *sgl = data->sg;
 	struct scatterlist *sg;
 	char *mem = NULL;
 	unsigned long  cmd_data_len = data->data_len;
@@ -72,7 +71,7 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 		/* copy the unaligned sg the buffer which is used for RDMA */
 		char *p, *from;
 
-		sgl = (struct scatterlist *)data->buf;
+		sgl = data->sg;
 		p = mem;
 		for_each_sg(sgl, sg, data->size, i) {
 			from = kmap_atomic(sg_page(sg));
@@ -84,18 +83,16 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 		}
 	}
 
-	sg_init_one(&data_copy->sg_single, mem, cmd_data_len);
-	data_copy->buf = &data_copy->sg_single;
-	data_copy->size = 1;
-	data_copy->copy_buf = mem;
-
-	dma_nents = ib_dma_map_sg(dev, &data_copy->sg_single, 1,
+	sg_init_one(&data->sg_single, mem, cmd_data_len);
+	data->orig_sg = data->sg;
+	data->sg = &data->sg_single;
+	data->copy_buf = mem;
+	dma_nents = ib_dma_map_sg(dev, data->sg, 1,
 				  (cmd_dir == ISER_DIR_OUT) ?
 				  DMA_TO_DEVICE : DMA_FROM_DEVICE);
 	BUG_ON(dma_nents == 0);
 
-	data_copy->dma_nents = dma_nents;
-	data_copy->data_len = cmd_data_len;
+	data->dma_nents = dma_nents;
 
 	return 0;
 }
@@ -106,7 +103,6 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 
 void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 				     struct iser_data_buf *data,
-				     struct iser_data_buf *data_copy,
 				     enum iser_data_dir cmd_dir)
 {
 	struct ib_device *dev;
@@ -114,7 +110,7 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 
 	dev = iser_task->iser_conn->ib_conn.device->ib_device;
 
-	ib_dma_unmap_sg(dev, &data_copy->sg_single, 1,
+	ib_dma_unmap_sg(dev, data->sg, 1,
 			(cmd_dir == ISER_DIR_OUT) ?
 			DMA_TO_DEVICE : DMA_FROM_DEVICE);
 
@@ -126,9 +122,9 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 		int i;
 
 		/* copy back read RDMA to unaligned sg */
-		mem = data_copy->copy_buf;
+		mem = data->copy_buf;
 
-		sgl = (struct scatterlist *)data->buf;
+		sgl = data->sg;
 		sg_size = data->size;
 
 		p = mem;
@@ -145,12 +141,12 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 	cmd_data_len = data->data_len;
 
 	if (cmd_data_len > ISER_KMALLOC_THRESHOLD)
-		free_pages((unsigned long)data_copy->copy_buf,
+		free_pages((unsigned long)data->copy_buf,
 			   ilog2(roundup_pow_of_two(cmd_data_len)) - PAGE_SHIFT);
 	else
-		kfree(data_copy->copy_buf);
+		kfree(data->copy_buf);
 
-	data_copy->copy_buf = NULL;
+	data->copy_buf = NULL;
 }
 
 #define IS_4K_ALIGNED(addr)	((((unsigned long)addr) & ~MASK_4K) == 0)
@@ -172,7 +168,7 @@ static int iser_sg_to_page_vec(struct iser_data_buf *data,
 			       struct ib_device *ibdev, u64 *pages,
 			       int *offset, int *data_size)
 {
-	struct scatterlist *sg, *sgl = (struct scatterlist *)data->buf;
+	struct scatterlist *sg, *sgl = data->sg;
 	u64 start_addr, end_addr, page, chunk_start = 0;
 	unsigned long total_sz = 0;
 	unsigned int dma_len;
@@ -224,14 +220,14 @@ static int iser_sg_to_page_vec(struct iser_data_buf *data,
 static int iser_data_buf_aligned_len(struct iser_data_buf *data,
 				      struct ib_device *ibdev)
 {
-	struct scatterlist *sgl, *sg, *next_sg = NULL;
+	struct scatterlist *sg, *sgl, *next_sg = NULL;
 	u64 start_addr, end_addr;
 	int i, ret_len, start_check = 0;
 
 	if (data->dma_nents == 1)
 		return 1;
 
-	sgl = (struct scatterlist *)data->buf;
+	sgl = data->sg;
 	start_addr  = ib_sg_dma_address(ibdev, sgl);
 
 	for_each_sg(sgl, sg, data->dma_nents, i) {
@@ -263,11 +259,10 @@ static int iser_data_buf_aligned_len(struct iser_data_buf *data,
 static void iser_data_buf_dump(struct iser_data_buf *data,
 			       struct ib_device *ibdev)
 {
-	struct scatterlist *sgl = (struct scatterlist *)data->buf;
 	struct scatterlist *sg;
 	int i;
 
-	for_each_sg(sgl, sg, data->dma_nents, i)
+	for_each_sg(data->sg, sg, data->dma_nents, i)
 		iser_dbg("sg[%d] dma_addr:0x%lX page:0x%p "
 			 "off:0x%x sz:0x%x dma_len:0x%x\n",
 			 i, (unsigned long)ib_sg_dma_address(ibdev, sg),
@@ -320,7 +315,7 @@ int iser_dma_map_task_data(struct iscsi_iser_task *iser_task,
 	iser_task->dir[iser_dir] = 1;
 	dev = iser_task->iser_conn->ib_conn.device->ib_device;
 
-	data->dma_nents = ib_dma_map_sg(dev, data->buf, data->size, dma_dir);
+	data->dma_nents = ib_dma_map_sg(dev, data->sg, data->size, dma_dir);
 	if (data->dma_nents == 0) {
 		iser_err("dma_map_sg failed!!!\n");
 		return -EINVAL;
@@ -335,13 +330,12 @@ void iser_dma_unmap_task_data(struct iscsi_iser_task *iser_task,
 	struct ib_device *dev;
 
 	dev = iser_task->iser_conn->ib_conn.device->ib_device;
-	ib_dma_unmap_sg(dev, data->buf, data->size, dir);
+	ib_dma_unmap_sg(dev, data->sg, data->size, dir);
 }
 
 static int fall_to_bounce_buf(struct iscsi_iser_task *iser_task,
 			      struct ib_device *ibdev,
 			      struct iser_data_buf *mem,
-			      struct iser_data_buf *mem_copy,
 			      enum iser_data_dir cmd_dir,
 			      int aligned_len)
 {
@@ -361,7 +355,7 @@ static int fall_to_bounce_buf(struct iscsi_iser_task *iser_task,
 
 	/* allocate copy buf, if we are writing, copy the */
 	/* unaligned scatterlist, dma map the copy        */
-	if (iser_start_rdma_unaligned_sg(iser_task, mem, mem_copy, cmd_dir) != 0)
+	if (iser_start_rdma_unaligned_sg(iser_task, mem, cmd_dir) != 0)
 		return -ENOMEM;
 
 	return 0;
@@ -391,18 +385,16 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 	aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 	if (aligned_len != mem->dma_nents) {
 		err = fall_to_bounce_buf(iser_task, ibdev, mem,
-					 &iser_task->data_copy[cmd_dir],
 					 cmd_dir, aligned_len);
 		if (err) {
 			iser_err("failed to allocate bounce buffer\n");
 			return err;
 		}
-		mem = &iser_task->data_copy[cmd_dir];
 	}
 
 	/* if there a single dma entry, FMR is not needed */
 	if (mem->dma_nents == 1) {
-		sg = (struct scatterlist *)mem->buf;
+		sg = mem->sg;
 
 		regd_buf->reg.lkey = device->mr->lkey;
 		regd_buf->reg.rkey = device->mr->rkey;
@@ -592,7 +584,7 @@ static int iser_fast_reg_mr(struct iscsi_iser_task *iser_task,
 
 	/* if there a single dma entry, dma mr suffices */
 	if (mem->dma_nents == 1) {
-		struct scatterlist *sg = (struct scatterlist *)mem->buf;
+		struct scatterlist *sg = mem->sg;
 
 		sge->lkey = device->mr->lkey;
 		sge->addr   = ib_sg_dma_address(ibdev, &sg[0]);
@@ -678,13 +670,11 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 	aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 	if (aligned_len != mem->dma_nents) {
 		err = fall_to_bounce_buf(iser_task, ibdev, mem,
-					 &iser_task->data_copy[cmd_dir],
 					 cmd_dir, aligned_len);
 		if (err) {
 			iser_err("failed to allocate bounce buffer\n");
 			return err;
 		}
-		mem = &iser_task->data_copy[cmd_dir];
 	}
 
 	if (mem->dma_nents != 1 ||
@@ -711,13 +701,11 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 			aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 			if (aligned_len != mem->dma_nents) {
 				err = fall_to_bounce_buf(iser_task, ibdev, mem,
-							 &iser_task->prot_copy[cmd_dir],
 							 cmd_dir, aligned_len);
 				if (err) {
 					iser_err("failed to allocate bounce buffer\n");
 					return err;
 				}
-				mem = &iser_task->prot_copy[cmd_dir];
 			}
 
 			err = iser_fast_reg_mr(iser_task, regd_buf, mem,
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 06/18] IB/iser: Don't pass ib_device to fall_to_bounce_buff routine

[Sagi Grimberg]

No need to pass that, we can take it from the task.
In a later stage, this function will be invoked
according to a device capability.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_memory.c |   12 ++++++------
 1 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index beeabd0..9c60ff1 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -334,19 +334,19 @@ void iser_dma_unmap_task_data(struct iscsi_iser_task *iser_task,
 }
 
 static int fall_to_bounce_buf(struct iscsi_iser_task *iser_task,
-			      struct ib_device *ibdev,
 			      struct iser_data_buf *mem,
 			      enum iser_data_dir cmd_dir,
 			      int aligned_len)
 {
-	struct iscsi_conn    *iscsi_conn = iser_task->iser_conn->iscsi_conn;
+	struct iscsi_conn *iscsi_conn = iser_task->iser_conn->iscsi_conn;
+	struct iser_device *device = iser_task->iser_conn->ib_conn.device;
 
 	iscsi_conn->fmr_unalign_cnt++;
 	iser_warn("rdma alignment violation (%d/%d aligned) or FMR not supported\n",
 		  aligned_len, mem->size);
 
 	if (iser_debug_level > 0)
-		iser_data_buf_dump(mem, ibdev);
+		iser_data_buf_dump(mem, device->ib_device);
 
 	/* unmap the command data before accessing it */
 	iser_dma_unmap_task_data(iser_task, mem,
@@ -384,7 +384,7 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 
 	aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 	if (aligned_len != mem->dma_nents) {
-		err = fall_to_bounce_buf(iser_task, ibdev, mem,
+		err = fall_to_bounce_buf(iser_task, mem,
 					 cmd_dir, aligned_len);
 		if (err) {
 			iser_err("failed to allocate bounce buffer\n");
@@ -669,7 +669,7 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 
 	aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 	if (aligned_len != mem->dma_nents) {
-		err = fall_to_bounce_buf(iser_task, ibdev, mem,
+		err = fall_to_bounce_buf(iser_task, mem,
 					 cmd_dir, aligned_len);
 		if (err) {
 			iser_err("failed to allocate bounce buffer\n");
@@ -700,7 +700,7 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 			mem = &iser_task->prot[cmd_dir];
 			aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 			if (aligned_len != mem->dma_nents) {
-				err = fall_to_bounce_buf(iser_task, ibdev, mem,
+				err = fall_to_bounce_buf(iser_task, mem,
 							 cmd_dir, aligned_len);
 				if (err) {
 					iser_err("failed to allocate bounce buffer\n");
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 07/18] IB/iser: Move memory reg/dereg routines to iser_memory.c

[Sagi Grimberg]

As memory registration/de-registration methods, lets
move them to their natural location. While we're at it,
make iser_reg_page_vec routine static.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h  |    4 -
 drivers/infiniband/ulp/iser/iser_memory.c |   88 +++++++++++++++++++++++++++++
 drivers/infiniband/ulp/iser/iser_verbs.c  |   87 ----------------------------
 3 files changed, 88 insertions(+), 91 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index 5c7036c..d5e5288 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -632,10 +632,6 @@ int  iser_connect(struct iser_conn *iser_conn,
 		  struct sockaddr *dst_addr,
 		  int non_blocking);
 
-int  iser_reg_page_vec(struct ib_conn *ib_conn,
-		       struct iser_page_vec *page_vec,
-		       struct iser_mem_reg *mem_reg);
-
 void iser_unreg_mem_fmr(struct iscsi_iser_task *iser_task,
 			enum iser_data_dir cmd_dir);
 void iser_unreg_mem_fastreg(struct iscsi_iser_task *iser_task,
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 9c60ff1..4e0cbbb 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -362,6 +362,94 @@ static int fall_to_bounce_buf(struct iscsi_iser_task *iser_task,
 }
 
 /**
+ * iser_reg_page_vec - Register physical memory
+ *
+ * returns: 0 on success, errno code on failure
+ */
+static
+int iser_reg_page_vec(struct ib_conn *ib_conn,
+		      struct iser_page_vec *page_vec,
+		      struct iser_mem_reg  *mem_reg)
+{
+	struct ib_pool_fmr *mem;
+	u64		   io_addr;
+	u64		   *page_list;
+	int		   status;
+
+	page_list = page_vec->pages;
+	io_addr	  = page_list[0];
+
+	mem  = ib_fmr_pool_map_phys(ib_conn->fmr.pool,
+				    page_list,
+				    page_vec->length,
+				    io_addr);
+
+	if (IS_ERR(mem)) {
+		status = (int)PTR_ERR(mem);
+		iser_err("ib_fmr_pool_map_phys failed: %d\n", status);
+		return status;
+	}
+
+	mem_reg->lkey  = mem->fmr->lkey;
+	mem_reg->rkey  = mem->fmr->rkey;
+	mem_reg->len   = page_vec->length * SIZE_4K;
+	mem_reg->va    = io_addr;
+	mem_reg->mem_h = (void *)mem;
+
+	mem_reg->va   += page_vec->offset;
+	mem_reg->len   = page_vec->data_size;
+
+	iser_dbg("PHYSICAL Mem.register, [PHYS p_array: 0x%p, sz: %d, "
+		 "entry[0]: (0x%08lx,%ld)] -> "
+		 "[lkey: 0x%08X mem_h: 0x%p va: 0x%08lX sz: %ld]\n",
+		 page_vec, page_vec->length,
+		 (unsigned long)page_vec->pages[0],
+		 (unsigned long)page_vec->data_size,
+		 (unsigned int)mem_reg->lkey, mem_reg->mem_h,
+		 (unsigned long)mem_reg->va, (unsigned long)mem_reg->len);
+	return 0;
+}
+
+/**
+ * Unregister (previosuly registered using FMR) memory.
+ * If memory is non-FMR does nothing.
+ */
+void iser_unreg_mem_fmr(struct iscsi_iser_task *iser_task,
+			enum iser_data_dir cmd_dir)
+{
+	struct iser_mem_reg *reg = &iser_task->rdma_regd[cmd_dir].reg;
+	int ret;
+
+	if (!reg->mem_h)
+		return;
+
+	iser_dbg("PHYSICAL Mem.Unregister mem_h %p\n", reg->mem_h);
+
+	ret = ib_fmr_pool_unmap((struct ib_pool_fmr *)reg->mem_h);
+	if (ret)
+		iser_err("ib_fmr_pool_unmap failed %d\n", ret);
+
+	reg->mem_h = NULL;
+}
+
+void iser_unreg_mem_fastreg(struct iscsi_iser_task *iser_task,
+			    enum iser_data_dir cmd_dir)
+{
+	struct iser_mem_reg *reg = &iser_task->rdma_regd[cmd_dir].reg;
+	struct iser_conn *iser_conn = iser_task->iser_conn;
+	struct ib_conn *ib_conn = &iser_conn->ib_conn;
+	struct fast_reg_descriptor *desc = reg->mem_h;
+
+	if (!desc)
+		return;
+
+	reg->mem_h = NULL;
+	spin_lock_bh(&ib_conn->lock);
+	list_add_tail(&desc->list, &ib_conn->fastreg.pool);
+	spin_unlock_bh(&ib_conn->lock);
+}
+
+/**
  * iser_reg_rdma_mem_fmr - Registers memory intended for RDMA,
  * using FMR (if possible) obtaining rkey and va
  *
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 7ee4926..986b5f4 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -992,93 +992,6 @@ connect_failure:
 	return err;
 }
 
-/**
- * iser_reg_page_vec - Register physical memory
- *
- * returns: 0 on success, errno code on failure
- */
-int iser_reg_page_vec(struct ib_conn *ib_conn,
-		      struct iser_page_vec *page_vec,
-		      struct iser_mem_reg  *mem_reg)
-{
-	struct ib_pool_fmr *mem;
-	u64		   io_addr;
-	u64		   *page_list;
-	int		   status;
-
-	page_list = page_vec->pages;
-	io_addr	  = page_list[0];
-
-	mem  = ib_fmr_pool_map_phys(ib_conn->fmr.pool,
-				    page_list,
-				    page_vec->length,
-				    io_addr);
-
-	if (IS_ERR(mem)) {
-		status = (int)PTR_ERR(mem);
-		iser_err("ib_fmr_pool_map_phys failed: %d\n", status);
-		return status;
-	}
-
-	mem_reg->lkey  = mem->fmr->lkey;
-	mem_reg->rkey  = mem->fmr->rkey;
-	mem_reg->len   = page_vec->length * SIZE_4K;
-	mem_reg->va    = io_addr;
-	mem_reg->mem_h = (void *)mem;
-
-	mem_reg->va   += page_vec->offset;
-	mem_reg->len   = page_vec->data_size;
-
-	iser_dbg("PHYSICAL Mem.register, [PHYS p_array: 0x%p, sz: %d, "
-		 "entry[0]: (0x%08lx,%ld)] -> "
-		 "[lkey: 0x%08X mem_h: 0x%p va: 0x%08lX sz: %ld]\n",
-		 page_vec, page_vec->length,
-		 (unsigned long)page_vec->pages[0],
-		 (unsigned long)page_vec->data_size,
-		 (unsigned int)mem_reg->lkey, mem_reg->mem_h,
-		 (unsigned long)mem_reg->va, (unsigned long)mem_reg->len);
-	return 0;
-}
-
-/**
- * Unregister (previosuly registered using FMR) memory.
- * If memory is non-FMR does nothing.
- */
-void iser_unreg_mem_fmr(struct iscsi_iser_task *iser_task,
-			enum iser_data_dir cmd_dir)
-{
-	struct iser_mem_reg *reg = &iser_task->rdma_regd[cmd_dir].reg;
-	int ret;
-
-	if (!reg->mem_h)
-		return;
-
-	iser_dbg("PHYSICAL Mem.Unregister mem_h %p\n",reg->mem_h);
-
-	ret = ib_fmr_pool_unmap((struct ib_pool_fmr *)reg->mem_h);
-	if (ret)
-		iser_err("ib_fmr_pool_unmap failed %d\n", ret);
-
-	reg->mem_h = NULL;
-}
-
-void iser_unreg_mem_fastreg(struct iscsi_iser_task *iser_task,
-			    enum iser_data_dir cmd_dir)
-{
-	struct iser_mem_reg *reg = &iser_task->rdma_regd[cmd_dir].reg;
-	struct iser_conn *iser_conn = iser_task->iser_conn;
-	struct ib_conn *ib_conn = &iser_conn->ib_conn;
-	struct fast_reg_descriptor *desc = reg->mem_h;
-
-	if (!desc)
-		return;
-
-	reg->mem_h = NULL;
-	spin_lock_bh(&ib_conn->lock);
-	list_add_tail(&desc->list, &ib_conn->fastreg.pool);
-	spin_unlock_bh(&ib_conn->lock);
-}
-
 int iser_post_recvl(struct iser_conn *iser_conn)
 {
 	struct ib_recv_wr rx_wr, *rx_wr_failed;
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 08/18] IB/iser: Remove redundant assignments in iser_reg_page_vec

[Sagi Grimberg]

Buffer length was assigned twice, and no reason to set va to
io_addr and then add the offset, just set va to io_addr + offset.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_memory.c |    7 ++-----
 1 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 4e0cbbb..cb30865 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -392,12 +392,9 @@ int iser_reg_page_vec(struct ib_conn *ib_conn,
 
 	mem_reg->lkey  = mem->fmr->lkey;
 	mem_reg->rkey  = mem->fmr->rkey;
-	mem_reg->len   = page_vec->length * SIZE_4K;
-	mem_reg->va    = io_addr;
-	mem_reg->mem_h = (void *)mem;
-
-	mem_reg->va   += page_vec->offset;
 	mem_reg->len   = page_vec->data_size;
+	mem_reg->va    = io_addr + page_vec->offset;
+	mem_reg->mem_h = (void *)mem;
 
 	iser_dbg("PHYSICAL Mem.register, [PHYS p_array: 0x%p, sz: %d, "
 		 "entry[0]: (0x%08lx,%ld)] -> "
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 09/18] IB/iser: Get rid of struct iser_rdma_regd

[Sagi Grimberg]

This struct members other than struct iser_mem_reg are unused,
so remove it altogether.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h     |   21 +---------
 drivers/infiniband/ulp/iser/iser_initiator.c |   44 ++++++++++----------
 drivers/infiniband/ulp/iser/iser_memory.c    |   56 +++++++++++++-------------
 drivers/infiniband/ulp/iser/iser_verbs.c     |    2 +-
 4 files changed, 53 insertions(+), 70 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index d5e5288..8133678 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -261,23 +261,6 @@ struct iser_mem_reg {
 	void *mem_h;
 };
 
-/**
- * struct iser_regd_buf - iSER buffer registration desc
- *
- * @reg:          memory registration info
- * @virt_addr:    virtual address of buffer
- * @device:       reference to iser device
- * @direction:    dma direction (for dma_unmap)
- * @data_size:    data buffer size in bytes
- */
-struct iser_regd_buf {
-	struct iser_mem_reg     reg;
-	void                    *virt_addr;
-	struct iser_device      *device;
-	enum dma_data_direction direction;
-	unsigned int            data_size;
-};
-
 enum iser_desc_type {
 	ISCSI_TX_CONTROL ,
 	ISCSI_TX_SCSI_COMMAND,
@@ -537,7 +520,7 @@ struct iser_conn {
  * @sc:               link to scsi command
  * @command_sent:     indicate if command was sent
  * @dir:              iser data direction
- * @rdma_regd:        task rdma registration desc
+ * @rdma_reg:         task rdma registration desc
  * @data:             iser data buffer desc
  * @prot:             iser protection buffer desc
  */
@@ -548,7 +531,7 @@ struct iscsi_iser_task {
 	struct scsi_cmnd	     *sc;
 	int                          command_sent;
 	int                          dir[ISER_DIRS_NUM];
-	struct iser_regd_buf         rdma_regd[ISER_DIRS_NUM];
+	struct iser_mem_reg          rdma_reg[ISER_DIRS_NUM];
 	struct iser_data_buf         data[ISER_DIRS_NUM];
 	struct iser_data_buf         prot[ISER_DIRS_NUM];
 };
diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c b/drivers/infiniband/ulp/iser/iser_initiator.c
index 0e414db..420a613 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -50,7 +50,7 @@ static int iser_prepare_read_cmd(struct iscsi_task *task)
 {
 	struct iscsi_iser_task *iser_task = task->dd_data;
 	struct iser_device  *device = iser_task->iser_conn->ib_conn.device;
-	struct iser_regd_buf *regd_buf;
+	struct iser_mem_reg *mem_reg;
 	int err;
 	struct iser_hdr *hdr = &iser_task->desc.iser_header;
 	struct iser_data_buf *buf_in = &iser_task->data[ISER_DIR_IN];
@@ -78,15 +78,15 @@ static int iser_prepare_read_cmd(struct iscsi_task *task)
 		iser_err("Failed to set up Data-IN RDMA\n");
 		return err;
 	}
-	regd_buf = &iser_task->rdma_regd[ISER_DIR_IN];
+	mem_reg = &iser_task->rdma_reg[ISER_DIR_IN];
 
 	hdr->flags    |= ISER_RSV;
-	hdr->read_stag = cpu_to_be32(regd_buf->reg.rkey);
-	hdr->read_va   = cpu_to_be64(regd_buf->reg.va);
+	hdr->read_stag = cpu_to_be32(mem_reg->rkey);
+	hdr->read_va   = cpu_to_be64(mem_reg->va);
 
 	iser_dbg("Cmd itt:%d READ tags RKEY:%#.4X VA:%#llX\n",
-		 task->itt, regd_buf->reg.rkey,
-		 (unsigned long long)regd_buf->reg.va);
+		 task->itt, mem_reg->rkey,
+		 (unsigned long long)mem_reg->va);
 
 	return 0;
 }
@@ -104,7 +104,7 @@ iser_prepare_write_cmd(struct iscsi_task *task,
 {
 	struct iscsi_iser_task *iser_task = task->dd_data;
 	struct iser_device  *device = iser_task->iser_conn->ib_conn.device;
-	struct iser_regd_buf *regd_buf;
+	struct iser_mem_reg *mem_reg;
 	int err;
 	struct iser_hdr *hdr = &iser_task->desc.iser_header;
 	struct iser_data_buf *buf_out = &iser_task->data[ISER_DIR_OUT];
@@ -134,25 +134,25 @@ iser_prepare_write_cmd(struct iscsi_task *task,
 		return err;
 	}
 
-	regd_buf = &iser_task->rdma_regd[ISER_DIR_OUT];
+	mem_reg = &iser_task->rdma_reg[ISER_DIR_OUT];
 
 	if (unsol_sz < edtl) {
 		hdr->flags     |= ISER_WSV;
-		hdr->write_stag = cpu_to_be32(regd_buf->reg.rkey);
-		hdr->write_va   = cpu_to_be64(regd_buf->reg.va + unsol_sz);
+		hdr->write_stag = cpu_to_be32(mem_reg->rkey);
+		hdr->write_va   = cpu_to_be64(mem_reg->va + unsol_sz);
 
 		iser_dbg("Cmd itt:%d, WRITE tags, RKEY:%#.4X "
 			 "VA:%#llX + unsol:%d\n",
-			 task->itt, regd_buf->reg.rkey,
-			 (unsigned long long)regd_buf->reg.va, unsol_sz);
+			 task->itt, mem_reg->rkey,
+			 (unsigned long long)mem_reg->va, unsol_sz);
 	}
 
 	if (imm_sz > 0) {
 		iser_dbg("Cmd itt:%d, WRITE, adding imm.data sz: %d\n",
 			 task->itt, imm_sz);
-		tx_dsg->addr   = regd_buf->reg.va;
+		tx_dsg->addr   = mem_reg->va;
 		tx_dsg->length = imm_sz;
-		tx_dsg->lkey   = regd_buf->reg.lkey;
+		tx_dsg->lkey   = mem_reg->lkey;
 		iser_task->desc.num_sge = 2;
 	}
 
@@ -450,7 +450,7 @@ int iser_send_data_out(struct iscsi_conn *conn,
 	struct iser_conn *iser_conn = conn->dd_data;
 	struct iscsi_iser_task *iser_task = task->dd_data;
 	struct iser_tx_desc *tx_desc = NULL;
-	struct iser_regd_buf *regd_buf;
+	struct iser_mem_reg *mem_reg;
 	unsigned long buf_offset;
 	unsigned long data_seg_len;
 	uint32_t itt;
@@ -477,11 +477,11 @@ int iser_send_data_out(struct iscsi_conn *conn,
 	/* build the tx desc */
 	iser_initialize_task_headers(task, tx_desc);
 
-	regd_buf = &iser_task->rdma_regd[ISER_DIR_OUT];
+	mem_reg = &iser_task->rdma_reg[ISER_DIR_OUT];
 	tx_dsg = &tx_desc->tx_sg[1];
-	tx_dsg->addr    = regd_buf->reg.va + buf_offset;
+	tx_dsg->addr    = mem_reg->va + buf_offset;
 	tx_dsg->length  = data_seg_len;
-	tx_dsg->lkey    = regd_buf->reg.lkey;
+	tx_dsg->lkey    = mem_reg->lkey;
 	tx_desc->num_sge = 2;
 
 	if (buf_offset + data_seg_len > iser_task->data[ISER_DIR_OUT].data_len) {
@@ -658,10 +658,10 @@ void iser_task_rdma_init(struct iscsi_iser_task *iser_task)
 	iser_task->prot[ISER_DIR_IN].data_len  = 0;
 	iser_task->prot[ISER_DIR_OUT].data_len = 0;
 
-	memset(&iser_task->rdma_regd[ISER_DIR_IN], 0,
-	       sizeof(struct iser_regd_buf));
-	memset(&iser_task->rdma_regd[ISER_DIR_OUT], 0,
-	       sizeof(struct iser_regd_buf));
+	memset(&iser_task->rdma_reg[ISER_DIR_IN], 0,
+	       sizeof(struct iser_mem_reg));
+	memset(&iser_task->rdma_reg[ISER_DIR_OUT], 0,
+	       sizeof(struct iser_mem_reg));
 }
 
 void iser_task_rdma_finalize(struct iscsi_iser_task *iser_task)
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index cb30865..0b8656f 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -414,7 +414,7 @@ int iser_reg_page_vec(struct ib_conn *ib_conn,
 void iser_unreg_mem_fmr(struct iscsi_iser_task *iser_task,
 			enum iser_data_dir cmd_dir)
 {
-	struct iser_mem_reg *reg = &iser_task->rdma_regd[cmd_dir].reg;
+	struct iser_mem_reg *reg = &iser_task->rdma_reg[cmd_dir];
 	int ret;
 
 	if (!reg->mem_h)
@@ -432,7 +432,7 @@ void iser_unreg_mem_fmr(struct iscsi_iser_task *iser_task,
 void iser_unreg_mem_fastreg(struct iscsi_iser_task *iser_task,
 			    enum iser_data_dir cmd_dir)
 {
-	struct iser_mem_reg *reg = &iser_task->rdma_regd[cmd_dir].reg;
+	struct iser_mem_reg *reg = &iser_task->rdma_reg[cmd_dir];
 	struct iser_conn *iser_conn = iser_task->iser_conn;
 	struct ib_conn *ib_conn = &iser_conn->ib_conn;
 	struct fast_reg_descriptor *desc = reg->mem_h;
@@ -459,13 +459,13 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 	struct iser_device   *device = ib_conn->device;
 	struct ib_device     *ibdev = device->ib_device;
 	struct iser_data_buf *mem = &iser_task->data[cmd_dir];
-	struct iser_regd_buf *regd_buf;
+	struct iser_mem_reg *mem_reg;
 	int aligned_len;
 	int err;
 	int i;
 	struct scatterlist *sg;
 
-	regd_buf = &iser_task->rdma_regd[cmd_dir];
+	mem_reg = &iser_task->rdma_reg[cmd_dir];
 
 	aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 	if (aligned_len != mem->dma_nents) {
@@ -481,21 +481,21 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 	if (mem->dma_nents == 1) {
 		sg = mem->sg;
 
-		regd_buf->reg.lkey = device->mr->lkey;
-		regd_buf->reg.rkey = device->mr->rkey;
-		regd_buf->reg.len  = ib_sg_dma_len(ibdev, &sg[0]);
-		regd_buf->reg.va   = ib_sg_dma_address(ibdev, &sg[0]);
+		mem_reg->lkey = device->mr->lkey;
+		mem_reg->rkey = device->mr->rkey;
+		mem_reg->len  = ib_sg_dma_len(ibdev, &sg[0]);
+		mem_reg->va   = ib_sg_dma_address(ibdev, &sg[0]);
 
 		iser_dbg("PHYSICAL Mem.register: lkey: 0x%08X rkey: 0x%08X  "
 			 "va: 0x%08lX sz: %ld]\n",
-			 (unsigned int)regd_buf->reg.lkey,
-			 (unsigned int)regd_buf->reg.rkey,
-			 (unsigned long)regd_buf->reg.va,
-			 (unsigned long)regd_buf->reg.len);
+			 (unsigned int)mem_reg->lkey,
+			 (unsigned int)mem_reg->rkey,
+			 (unsigned long)mem_reg->va,
+			 (unsigned long)mem_reg->len);
 	} else { /* use FMR for multiple dma entries */
 		iser_page_vec_build(mem, ib_conn->fmr.page_vec, ibdev);
 		err = iser_reg_page_vec(ib_conn, ib_conn->fmr.page_vec,
-					&regd_buf->reg);
+					mem_reg);
 		if (err && err != -EAGAIN) {
 			iser_data_buf_dump(mem, ibdev);
 			iser_err("mem->dma_nents = %d (dlength = 0x%x)\n",
@@ -652,12 +652,12 @@ err:
 }
 
 static int iser_fast_reg_mr(struct iscsi_iser_task *iser_task,
-			    struct iser_regd_buf *regd_buf,
+			    struct iser_mem_reg *mem_reg,
 			    struct iser_data_buf *mem,
 			    enum iser_reg_indicator ind,
 			    struct ib_sge *sge)
 {
-	struct fast_reg_descriptor *desc = regd_buf->reg.mem_h;
+	struct fast_reg_descriptor *desc = mem_reg->mem_h;
 	struct ib_conn *ib_conn = &iser_task->iser_conn->ib_conn;
 	struct iser_device *device = ib_conn->device;
 	struct ib_device *ibdev = device->ib_device;
@@ -746,7 +746,7 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 	struct iser_device *device = ib_conn->device;
 	struct ib_device *ibdev = device->ib_device;
 	struct iser_data_buf *mem = &iser_task->data[cmd_dir];
-	struct iser_regd_buf *regd_buf = &iser_task->rdma_regd[cmd_dir];
+	struct iser_mem_reg *mem_reg = &iser_task->rdma_reg[cmd_dir];
 	struct fast_reg_descriptor *desc = NULL;
 	struct ib_sge data_sge;
 	int err, aligned_len;
@@ -769,10 +769,10 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 					struct fast_reg_descriptor, list);
 		list_del(&desc->list);
 		spin_unlock_irqrestore(&ib_conn->lock, flags);
-		regd_buf->reg.mem_h = desc;
+		mem_reg->mem_h = desc;
 	}
 
-	err = iser_fast_reg_mr(iser_task, regd_buf, mem,
+	err = iser_fast_reg_mr(iser_task, mem_reg, mem,
 			       ISER_DATA_KEY_VALID, &data_sge);
 	if (err)
 		goto err_reg;
@@ -793,7 +793,7 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 				}
 			}
 
-			err = iser_fast_reg_mr(iser_task, regd_buf, mem,
+			err = iser_fast_reg_mr(iser_task, mem_reg, mem,
 					       ISER_PROT_KEY_VALID, &prot_sge);
 			if (err)
 				goto err_reg;
@@ -807,19 +807,19 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 		}
 		desc->reg_indicators |= ISER_FASTREG_PROTECTED;
 
-		regd_buf->reg.lkey = sig_sge.lkey;
-		regd_buf->reg.rkey = desc->pi_ctx->sig_mr->rkey;
-		regd_buf->reg.va = sig_sge.addr;
-		regd_buf->reg.len = sig_sge.length;
+		mem_reg->lkey = sig_sge.lkey;
+		mem_reg->rkey = desc->pi_ctx->sig_mr->rkey;
+		mem_reg->va = sig_sge.addr;
+		mem_reg->len = sig_sge.length;
 	} else {
 		if (desc)
-			regd_buf->reg.rkey = desc->data_mr->rkey;
+			mem_reg->rkey = desc->data_mr->rkey;
 		else
-			regd_buf->reg.rkey = device->mr->rkey;
+			mem_reg->rkey = device->mr->rkey;
 
-		regd_buf->reg.lkey = data_sge.lkey;
-		regd_buf->reg.va = data_sge.addr;
-		regd_buf->reg.len = data_sge.length;
+		mem_reg->lkey = data_sge.lkey;
+		mem_reg->va = data_sge.addr;
+		mem_reg->len = data_sge.length;
 	}
 
 	return 0;
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 986b5f4..20eec09 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -1220,7 +1220,7 @@ static void iser_cq_callback(struct ib_cq *cq, void *cq_context)
 u8 iser_check_task_pi_status(struct iscsi_iser_task *iser_task,
 			     enum iser_data_dir cmd_dir, sector_t *sector)
 {
-	struct iser_mem_reg *reg = &iser_task->rdma_regd[cmd_dir].reg;
+	struct iser_mem_reg *reg = &iser_task->rdma_reg[cmd_dir];
 	struct fast_reg_descriptor *desc = reg->mem_h;
 	unsigned long sector_size = iser_task->sc->device->sector_size;
 	struct ib_mr_status mr_status;
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 10/18] IB/iser: Merge build page-vec into register page-vec

[Sagi Grimberg]

No need for these two separate. Keep it in a single routine
like in the fastreg case. This will also make iser_reg_page_vec
closer to iser_fast_reg_mr arguments. This is a preparation
step for registration flow refactor.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_memory.c |   91 ++++++++++------------------
 1 files changed, 33 insertions(+), 58 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 0b8656f..6e6b753 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -280,31 +280,6 @@ static void iser_dump_page_vec(struct iser_page_vec *page_vec)
 		iser_err("%d %lx\n",i,(unsigned long)page_vec->pages[i]);
 }
 
-static void iser_page_vec_build(struct iser_data_buf *data,
-				struct iser_page_vec *page_vec,
-				struct ib_device *ibdev)
-{
-	int page_vec_len = 0;
-
-	page_vec->length = 0;
-	page_vec->offset = 0;
-
-	iser_dbg("Translating sg sz: %d\n", data->dma_nents);
-	page_vec_len = iser_sg_to_page_vec(data, ibdev, page_vec->pages,
-					   &page_vec->offset,
-					   &page_vec->data_size);
-	iser_dbg("sg len %d page_vec_len %d\n", data->dma_nents, page_vec_len);
-
-	page_vec->length = page_vec_len;
-
-	if (page_vec_len * SIZE_4K < page_vec->data_size) {
-		iser_err("page_vec too short to hold this SG\n");
-		iser_data_buf_dump(data, ibdev);
-		iser_dump_page_vec(page_vec);
-		BUG();
-	}
-}
-
 int iser_dma_map_task_data(struct iscsi_iser_task *iser_task,
 			    struct iser_data_buf *data,
 			    enum iser_data_dir iser_dir,
@@ -367,43 +342,44 @@ static int fall_to_bounce_buf(struct iscsi_iser_task *iser_task,
  * returns: 0 on success, errno code on failure
  */
 static
-int iser_reg_page_vec(struct ib_conn *ib_conn,
+int iser_reg_page_vec(struct iscsi_iser_task *iser_task,
+		      struct iser_data_buf *mem,
 		      struct iser_page_vec *page_vec,
-		      struct iser_mem_reg  *mem_reg)
+		      struct iser_mem_reg *mem_reg)
 {
-	struct ib_pool_fmr *mem;
-	u64		   io_addr;
-	u64		   *page_list;
-	int		   status;
-
-	page_list = page_vec->pages;
-	io_addr	  = page_list[0];
+	struct ib_conn *ib_conn = &iser_task->iser_conn->ib_conn;
+	struct iser_device *device = ib_conn->device;
+	struct ib_pool_fmr *fmr;
+	int ret, plen;
+
+	plen = iser_sg_to_page_vec(mem, device->ib_device,
+				   page_vec->pages,
+				   &page_vec->offset,
+				   &page_vec->data_size);
+	page_vec->length = plen;
+	if (plen * SIZE_4K < page_vec->data_size) {
+		iser_err("page vec too short to hold this SG\n");
+		iser_data_buf_dump(mem, device->ib_device);
+		iser_dump_page_vec(page_vec);
+		return -EINVAL;
+	}
 
-	mem  = ib_fmr_pool_map_phys(ib_conn->fmr.pool,
-				    page_list,
+	fmr  = ib_fmr_pool_map_phys(ib_conn->fmr.pool,
+				    page_vec->pages,
 				    page_vec->length,
-				    io_addr);
-
-	if (IS_ERR(mem)) {
-		status = (int)PTR_ERR(mem);
-		iser_err("ib_fmr_pool_map_phys failed: %d\n", status);
-		return status;
+				    page_vec->pages[0]);
+	if (IS_ERR(fmr)) {
+		ret = PTR_ERR(fmr);
+		iser_err("ib_fmr_pool_map_phys failed: %d\n", ret);
+		return ret;
 	}
 
-	mem_reg->lkey  = mem->fmr->lkey;
-	mem_reg->rkey  = mem->fmr->rkey;
-	mem_reg->len   = page_vec->data_size;
-	mem_reg->va    = io_addr + page_vec->offset;
-	mem_reg->mem_h = (void *)mem;
-
-	iser_dbg("PHYSICAL Mem.register, [PHYS p_array: 0x%p, sz: %d, "
-		 "entry[0]: (0x%08lx,%ld)] -> "
-		 "[lkey: 0x%08X mem_h: 0x%p va: 0x%08lX sz: %ld]\n",
-		 page_vec, page_vec->length,
-		 (unsigned long)page_vec->pages[0],
-		 (unsigned long)page_vec->data_size,
-		 (unsigned int)mem_reg->lkey, mem_reg->mem_h,
-		 (unsigned long)mem_reg->va, (unsigned long)mem_reg->len);
+	mem_reg->lkey = fmr->fmr->lkey;
+	mem_reg->rkey = fmr->fmr->rkey;
+	mem_reg->va = page_vec->pages[0] + page_vec->offset;
+	mem_reg->len = page_vec->data_size;
+	mem_reg->mem_h = fmr;
+
 	return 0;
 }
 
@@ -493,8 +469,7 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 			 (unsigned long)mem_reg->va,
 			 (unsigned long)mem_reg->len);
 	} else { /* use FMR for multiple dma entries */
-		iser_page_vec_build(mem, ib_conn->fmr.page_vec, ibdev);
-		err = iser_reg_page_vec(ib_conn, ib_conn->fmr.page_vec,
+		err = iser_reg_page_vec(iser_task, mem, ib_conn->fmr.page_vec,
 					mem_reg);
 		if (err && err != -EAGAIN) {
 			iser_data_buf_dump(mem, ibdev);
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 11/18] IB/iser: Move fastreg descriptor pool get/put to helper functions

[Sagi Grimberg]

Instead of open-coding connection fastreg pool get/put,
we introduce iser_reg_desc[get|put] helpers.

We aren't setting these static as this will be a per-device
routine later on. Also, cleanup iser_unreg_rdma_mem_fastreg
a bit.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h  |    5 +++
 drivers/infiniband/ulp/iser/iser_memory.c |   50 ++++++++++++++++++----------
 2 files changed, 37 insertions(+), 18 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index 8133678..185d2ec 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -644,4 +644,9 @@ int iser_create_fastreg_pool(struct ib_conn *ib_conn, unsigned cmds_max);
 void iser_free_fastreg_pool(struct ib_conn *ib_conn);
 u8 iser_check_task_pi_status(struct iscsi_iser_task *iser_task,
 			     enum iser_data_dir cmd_dir, sector_t *sector);
+struct fast_reg_descriptor *
+iser_reg_desc_get(struct ib_conn *ib_conn);
+void
+iser_reg_desc_put(struct ib_conn *ib_conn,
+		  struct fast_reg_descriptor *desc);
 #endif
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 6e6b753..17a5d70 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -41,6 +41,32 @@
 
 #define ISER_KMALLOC_THRESHOLD 0x20000 /* 128K - kmalloc limit */
 
+struct fast_reg_descriptor *
+iser_reg_desc_get(struct ib_conn *ib_conn)
+{
+	struct fast_reg_descriptor *desc;
+	unsigned long flags;
+
+	spin_lock_irqsave(&ib_conn->lock, flags);
+	desc = list_first_entry(&ib_conn->fastreg.pool,
+				struct fast_reg_descriptor, list);
+	list_del(&desc->list);
+	spin_unlock_irqrestore(&ib_conn->lock, flags);
+
+	return desc;
+}
+
+void
+iser_reg_desc_put(struct ib_conn *ib_conn,
+		  struct fast_reg_descriptor *desc)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&ib_conn->lock, flags);
+	list_add_tail(&desc->list, &ib_conn->fastreg.pool);
+	spin_unlock_irqrestore(&ib_conn->lock, flags);
+}
+
 /**
  * iser_start_rdma_unaligned_sg
  */
@@ -409,17 +435,13 @@ void iser_unreg_mem_fastreg(struct iscsi_iser_task *iser_task,
 			    enum iser_data_dir cmd_dir)
 {
 	struct iser_mem_reg *reg = &iser_task->rdma_reg[cmd_dir];
-	struct iser_conn *iser_conn = iser_task->iser_conn;
-	struct ib_conn *ib_conn = &iser_conn->ib_conn;
-	struct fast_reg_descriptor *desc = reg->mem_h;
 
-	if (!desc)
+	if (!reg->mem_h)
 		return;
 
+	iser_reg_desc_put(&iser_task->iser_conn->ib_conn,
+			  reg->mem_h);
 	reg->mem_h = NULL;
-	spin_lock_bh(&ib_conn->lock);
-	list_add_tail(&desc->list, &ib_conn->fastreg.pool);
-	spin_unlock_bh(&ib_conn->lock);
 }
 
 /**
@@ -725,7 +747,6 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 	struct fast_reg_descriptor *desc = NULL;
 	struct ib_sge data_sge;
 	int err, aligned_len;
-	unsigned long flags;
 
 	aligned_len = iser_data_buf_aligned_len(mem, ibdev);
 	if (aligned_len != mem->dma_nents) {
@@ -739,11 +760,7 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 
 	if (mem->dma_nents != 1 ||
 	    scsi_get_prot_op(iser_task->sc) != SCSI_PROT_NORMAL) {
-		spin_lock_irqsave(&ib_conn->lock, flags);
-		desc = list_first_entry(&ib_conn->fastreg.pool,
-					struct fast_reg_descriptor, list);
-		list_del(&desc->list);
-		spin_unlock_irqrestore(&ib_conn->lock, flags);
+		desc = iser_reg_desc_get(ib_conn);
 		mem_reg->mem_h = desc;
 	}
 
@@ -799,11 +816,8 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 
 	return 0;
 err_reg:
-	if (desc) {
-		spin_lock_irqsave(&ib_conn->lock, flags);
-		list_add_tail(&desc->list, &ib_conn->fastreg.pool);
-		spin_unlock_irqrestore(&ib_conn->lock, flags);
-	}
+	if (desc)
+		iser_reg_desc_put(ib_conn, desc);
 
 	return err;
 }
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 12/18] IB/iser: Move PI context alloc/free to routines

[Sagi Grimberg]

Make iser_[create|destroy]_fastreg_desc shorter, more
readable and easily extendable.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_verbs.c |  118 ++++++++++++++++--------------
 1 files changed, 63 insertions(+), 55 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 20eec09..cc2dd35 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -274,6 +274,65 @@ void iser_free_fmr_pool(struct ib_conn *ib_conn)
 }
 
 static int
+iser_alloc_pi_ctx(struct ib_device *ib_device, struct ib_pd *pd,
+		  struct fast_reg_descriptor *desc)
+{
+	struct iser_pi_context *pi_ctx = NULL;
+	struct ib_mr_init_attr mr_init_attr = {.max_reg_descriptors = 2,
+					       .flags = IB_MR_SIGNATURE_EN};
+	int ret = 0;
+
+	desc->pi_ctx = kzalloc(sizeof(*desc->pi_ctx), GFP_KERNEL);
+	if (!desc->pi_ctx)
+		return -ENOMEM;
+
+	pi_ctx = desc->pi_ctx;
+
+	pi_ctx->prot_frpl = ib_alloc_fast_reg_page_list(ib_device,
+					    ISCSI_ISER_SG_TABLESIZE);
+	if (IS_ERR(pi_ctx->prot_frpl)) {
+		ret = PTR_ERR(pi_ctx->prot_frpl);
+		goto prot_frpl_failure;
+	}
+
+	pi_ctx->prot_mr = ib_alloc_fast_reg_mr(pd,
+					ISCSI_ISER_SG_TABLESIZE + 1);
+	if (IS_ERR(pi_ctx->prot_mr)) {
+		ret = PTR_ERR(pi_ctx->prot_mr);
+		goto prot_mr_failure;
+	}
+	desc->reg_indicators |= ISER_PROT_KEY_VALID;
+
+	pi_ctx->sig_mr = ib_create_mr(pd, &mr_init_attr);
+	if (IS_ERR(pi_ctx->sig_mr)) {
+		ret = PTR_ERR(pi_ctx->sig_mr);
+		goto sig_mr_failure;
+	}
+	desc->reg_indicators |= ISER_SIG_KEY_VALID;
+	desc->reg_indicators &= ~ISER_FASTREG_PROTECTED;
+
+	return 0;
+
+sig_mr_failure:
+	ib_dereg_mr(desc->pi_ctx->prot_mr);
+prot_mr_failure:
+	ib_free_fast_reg_page_list(desc->pi_ctx->prot_frpl);
+prot_frpl_failure:
+	kfree(desc->pi_ctx);
+
+	return ret;
+}
+
+static void
+iser_free_pi_ctx(struct iser_pi_context *pi_ctx)
+{
+	ib_free_fast_reg_page_list(pi_ctx->prot_frpl);
+	ib_dereg_mr(pi_ctx->prot_mr);
+	ib_destroy_mr(pi_ctx->sig_mr);
+	kfree(pi_ctx);
+}
+
+static int
 iser_create_fastreg_desc(struct ib_device *ib_device, struct ib_pd *pd,
 			 bool pi_enable, struct fast_reg_descriptor *desc)
 {
@@ -297,59 +356,12 @@ iser_create_fastreg_desc(struct ib_device *ib_device, struct ib_pd *pd,
 	desc->reg_indicators |= ISER_DATA_KEY_VALID;
 
 	if (pi_enable) {
-		struct ib_mr_init_attr mr_init_attr = {0};
-		struct iser_pi_context *pi_ctx = NULL;
-
-		desc->pi_ctx = kzalloc(sizeof(*desc->pi_ctx), GFP_KERNEL);
-		if (!desc->pi_ctx) {
-			iser_err("Failed to allocate pi context\n");
-			ret = -ENOMEM;
+		ret = iser_alloc_pi_ctx(ib_device, pd, desc);
+		if (ret)
 			goto pi_ctx_alloc_failure;
-		}
-		pi_ctx = desc->pi_ctx;
-
-		pi_ctx->prot_frpl = ib_alloc_fast_reg_page_list(ib_device,
-						    ISCSI_ISER_SG_TABLESIZE);
-		if (IS_ERR(pi_ctx->prot_frpl)) {
-			ret = PTR_ERR(pi_ctx->prot_frpl);
-			iser_err("Failed to allocate prot frpl ret=%d\n",
-				 ret);
-			goto prot_frpl_failure;
-		}
-
-		pi_ctx->prot_mr = ib_alloc_fast_reg_mr(pd,
-						ISCSI_ISER_SG_TABLESIZE + 1);
-		if (IS_ERR(pi_ctx->prot_mr)) {
-			ret = PTR_ERR(pi_ctx->prot_mr);
-			iser_err("Failed to allocate prot frmr ret=%d\n",
-				 ret);
-			goto prot_mr_failure;
-		}
-		desc->reg_indicators |= ISER_PROT_KEY_VALID;
-
-		mr_init_attr.max_reg_descriptors = 2;
-		mr_init_attr.flags |= IB_MR_SIGNATURE_EN;
-		pi_ctx->sig_mr = ib_create_mr(pd, &mr_init_attr);
-		if (IS_ERR(pi_ctx->sig_mr)) {
-			ret = PTR_ERR(pi_ctx->sig_mr);
-			iser_err("Failed to allocate signature enabled mr err=%d\n",
-				 ret);
-			goto sig_mr_failure;
-		}
-		desc->reg_indicators |= ISER_SIG_KEY_VALID;
 	}
-	desc->reg_indicators &= ~ISER_FASTREG_PROTECTED;
-
-	iser_dbg("Create fr_desc %p page_list %p\n",
-		 desc, desc->data_frpl->page_list);
 
 	return 0;
-sig_mr_failure:
-	ib_dereg_mr(desc->pi_ctx->prot_mr);
-prot_mr_failure:
-	ib_free_fast_reg_page_list(desc->pi_ctx->prot_frpl);
-prot_frpl_failure:
-	kfree(desc->pi_ctx);
 pi_ctx_alloc_failure:
 	ib_dereg_mr(desc->data_mr);
 fast_reg_mr_failure:
@@ -416,12 +428,8 @@ void iser_free_fastreg_pool(struct ib_conn *ib_conn)
 		list_del(&desc->list);
 		ib_free_fast_reg_page_list(desc->data_frpl);
 		ib_dereg_mr(desc->data_mr);
-		if (desc->pi_ctx) {
-			ib_free_fast_reg_page_list(desc->pi_ctx->prot_frpl);
-			ib_dereg_mr(desc->pi_ctx->prot_mr);
-			ib_destroy_mr(desc->pi_ctx->sig_mr);
-			kfree(desc->pi_ctx);
-		}
+		if (desc->pi_ctx)
+			iser_free_pi_ctx(desc->pi_ctx);
 		kfree(desc);
 		++i;
 	}
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 13/18] IB/iser: Make fastreg pool cache friendly

[Sagi Grimberg]

Memory regions are resources that are saved
in the device caches. Increase the probability for
a cache hit by adding the MRU descriptor to pool
head.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_memory.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 17a5d70..45f5120 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -63,7 +63,7 @@ iser_reg_desc_put(struct ib_conn *ib_conn,
 	unsigned long flags;
 
 	spin_lock_irqsave(&ib_conn->lock, flags);
-	list_add_tail(&desc->list, &ib_conn->fastreg.pool);
+	list_add(&desc->list, &ib_conn->fastreg.pool);
 	spin_unlock_irqrestore(&ib_conn->lock, flags);
 }
 
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 14/18] IB/iser: Modify struct iser_mem_reg members

[Sagi Grimberg]

No need to keep lkey, va, len variables, we can keep
them as struct ib_sge. This will help when we change the
memory registration logic.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h     |   14 ++++-------
 drivers/infiniband/ulp/iser/iser_initiator.c |   18 +++++++-------
 drivers/infiniband/ulp/iser/iser_memory.c    |   30 +++++++++++++-------------
 3 files changed, 29 insertions(+), 33 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index 185d2ec..5fd0963 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -247,18 +247,14 @@ struct iscsi_endpoint;
 /**
  * struct iser_mem_reg - iSER memory registration info
  *
- * @lkey:         MR local key
- * @rkey:         MR remote key
- * @va:           MR start address (buffer va)
- * @len:          MR length
+ * @sge:          memory region sg element
+ * @rkey:         memory region remote key
  * @mem_h:        pointer to registration context (FMR/Fastreg)
  */
 struct iser_mem_reg {
-	u32  lkey;
-	u32  rkey;
-	u64  va;
-	u64  len;
-	void *mem_h;
+	struct ib_sge	 sge;
+	u32		 rkey;
+	void		*mem_h;
 };
 
 enum iser_desc_type {
diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c b/drivers/infiniband/ulp/iser/iser_initiator.c
index 420a613..b2e3b77 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -82,11 +82,11 @@ static int iser_prepare_read_cmd(struct iscsi_task *task)
 
 	hdr->flags    |= ISER_RSV;
 	hdr->read_stag = cpu_to_be32(mem_reg->rkey);
-	hdr->read_va   = cpu_to_be64(mem_reg->va);
+	hdr->read_va   = cpu_to_be64(mem_reg->sge.addr);
 
 	iser_dbg("Cmd itt:%d READ tags RKEY:%#.4X VA:%#llX\n",
 		 task->itt, mem_reg->rkey,
-		 (unsigned long long)mem_reg->va);
+		 (unsigned long long)mem_reg->sge.addr);
 
 	return 0;
 }
@@ -139,20 +139,20 @@ iser_prepare_write_cmd(struct iscsi_task *task,
 	if (unsol_sz < edtl) {
 		hdr->flags     |= ISER_WSV;
 		hdr->write_stag = cpu_to_be32(mem_reg->rkey);
-		hdr->write_va   = cpu_to_be64(mem_reg->va + unsol_sz);
+		hdr->write_va   = cpu_to_be64(mem_reg->sge.addr + unsol_sz);
 
 		iser_dbg("Cmd itt:%d, WRITE tags, RKEY:%#.4X "
 			 "VA:%#llX + unsol:%d\n",
 			 task->itt, mem_reg->rkey,
-			 (unsigned long long)mem_reg->va, unsol_sz);
+			 (unsigned long long)mem_reg->sge.addr, unsol_sz);
 	}
 
 	if (imm_sz > 0) {
 		iser_dbg("Cmd itt:%d, WRITE, adding imm.data sz: %d\n",
 			 task->itt, imm_sz);
-		tx_dsg->addr   = mem_reg->va;
+		tx_dsg->addr = mem_reg->sge.addr;
 		tx_dsg->length = imm_sz;
-		tx_dsg->lkey   = mem_reg->lkey;
+		tx_dsg->lkey = mem_reg->sge.lkey;
 		iser_task->desc.num_sge = 2;
 	}
 
@@ -479,9 +479,9 @@ int iser_send_data_out(struct iscsi_conn *conn,
 
 	mem_reg = &iser_task->rdma_reg[ISER_DIR_OUT];
 	tx_dsg = &tx_desc->tx_sg[1];
-	tx_dsg->addr    = mem_reg->va + buf_offset;
-	tx_dsg->length  = data_seg_len;
-	tx_dsg->lkey    = mem_reg->lkey;
+	tx_dsg->addr = mem_reg->sge.addr + buf_offset;
+	tx_dsg->length = data_seg_len;
+	tx_dsg->lkey = mem_reg->sge.lkey;
 	tx_desc->num_sge = 2;
 
 	if (buf_offset + data_seg_len > iser_task->data[ISER_DIR_OUT].data_len) {
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 45f5120..40d22d5 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -400,10 +400,10 @@ int iser_reg_page_vec(struct iscsi_iser_task *iser_task,
 		return ret;
 	}
 
-	mem_reg->lkey = fmr->fmr->lkey;
+	mem_reg->sge.lkey = fmr->fmr->lkey;
 	mem_reg->rkey = fmr->fmr->rkey;
-	mem_reg->va = page_vec->pages[0] + page_vec->offset;
-	mem_reg->len = page_vec->data_size;
+	mem_reg->sge.addr = page_vec->pages[0] + page_vec->offset;
+	mem_reg->sge.length = page_vec->data_size;
 	mem_reg->mem_h = fmr;
 
 	return 0;
@@ -479,17 +479,17 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 	if (mem->dma_nents == 1) {
 		sg = mem->sg;
 
-		mem_reg->lkey = device->mr->lkey;
+		mem_reg->sge.lkey = device->mr->lkey;
 		mem_reg->rkey = device->mr->rkey;
-		mem_reg->len  = ib_sg_dma_len(ibdev, &sg[0]);
-		mem_reg->va   = ib_sg_dma_address(ibdev, &sg[0]);
+		mem_reg->sge.length = ib_sg_dma_len(ibdev, &sg[0]);
+		mem_reg->sge.addr = ib_sg_dma_address(ibdev, &sg[0]);
 
 		iser_dbg("PHYSICAL Mem.register: lkey: 0x%08X rkey: 0x%08X  "
 			 "va: 0x%08lX sz: %ld]\n",
-			 (unsigned int)mem_reg->lkey,
+			 (unsigned int)mem_reg->sge.lkey,
 			 (unsigned int)mem_reg->rkey,
-			 (unsigned long)mem_reg->va,
-			 (unsigned long)mem_reg->len);
+			 (unsigned long)mem_reg->sge.addr,
+			 (unsigned long)mem_reg->sge.length);
 	} else { /* use FMR for multiple dma entries */
 		err = iser_reg_page_vec(iser_task, mem, ib_conn->fmr.page_vec,
 					mem_reg);
@@ -799,19 +799,19 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 		}
 		desc->reg_indicators |= ISER_FASTREG_PROTECTED;
 
-		mem_reg->lkey = sig_sge.lkey;
+		mem_reg->sge.lkey = sig_sge.lkey;
 		mem_reg->rkey = desc->pi_ctx->sig_mr->rkey;
-		mem_reg->va = sig_sge.addr;
-		mem_reg->len = sig_sge.length;
+		mem_reg->sge.addr = sig_sge.addr;
+		mem_reg->sge.length = sig_sge.length;
 	} else {
 		if (desc)
 			mem_reg->rkey = desc->data_mr->rkey;
 		else
 			mem_reg->rkey = device->mr->rkey;
 
-		mem_reg->lkey = data_sge.lkey;
-		mem_reg->va = data_sge.addr;
-		mem_reg->len = data_sge.length;
+		mem_reg->sge.lkey = data_sge.lkey;
+		mem_reg->sge.addr = data_sge.addr;
+		mem_reg->sge.length = data_sge.length;
 	}
 
 	return 0;
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 15/18] IB/iser: Pass struct iser_mem_reg to iser_fast_reg_mr and iser_reg_sig_mr

[Sagi Grimberg]

Instead of passing ib_sge as output variable, we pass the mem_reg
pointer to have the routines fill the rkey as well. This reduces
code duplication and extra assignments. This is a preparation step
to unify some registration logics together. Also, pass iser_fast_reg_mr
the fastreg descriptor directly.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_memory.c |   75 ++++++++++++----------------
 1 files changed, 32 insertions(+), 43 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 40d22d5..0575052 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -590,8 +590,10 @@ iser_inv_rkey(struct ib_send_wr *inv_wr, struct ib_mr *mr)
 
 static int
 iser_reg_sig_mr(struct iscsi_iser_task *iser_task,
-		struct fast_reg_descriptor *desc, struct ib_sge *data_sge,
-		struct ib_sge *prot_sge, struct ib_sge *sig_sge)
+		struct fast_reg_descriptor *desc,
+		struct iser_mem_reg *data_reg,
+		struct iser_mem_reg *prot_reg,
+		struct iser_mem_reg *sig_reg)
 {
 	struct ib_conn *ib_conn = &iser_task->iser_conn->ib_conn;
 	struct iser_pi_context *pi_ctx = desc->pi_ctx;
@@ -615,12 +617,12 @@ iser_reg_sig_mr(struct iscsi_iser_task *iser_task,
 	memset(&sig_wr, 0, sizeof(sig_wr));
 	sig_wr.opcode = IB_WR_REG_SIG_MR;
 	sig_wr.wr_id = ISER_FASTREG_LI_WRID;
-	sig_wr.sg_list = data_sge;
+	sig_wr.sg_list = &data_reg->sge;
 	sig_wr.num_sge = 1;
 	sig_wr.wr.sig_handover.sig_attrs = &sig_attrs;
 	sig_wr.wr.sig_handover.sig_mr = pi_ctx->sig_mr;
 	if (scsi_prot_sg_count(iser_task->sc))
-		sig_wr.wr.sig_handover.prot = prot_sge;
+		sig_wr.wr.sig_handover.prot = &prot_reg->sge;
 	sig_wr.wr.sig_handover.access_flags = IB_ACCESS_LOCAL_WRITE |
 					      IB_ACCESS_REMOTE_READ |
 					      IB_ACCESS_REMOTE_WRITE;
@@ -637,24 +639,24 @@ iser_reg_sig_mr(struct iscsi_iser_task *iser_task,
 	}
 	desc->reg_indicators &= ~ISER_SIG_KEY_VALID;
 
-	sig_sge->lkey = pi_ctx->sig_mr->lkey;
-	sig_sge->addr = 0;
-	sig_sge->length = scsi_transfer_length(iser_task->sc);
+	sig_reg->sge.lkey = pi_ctx->sig_mr->lkey;
+	sig_reg->rkey = pi_ctx->sig_mr->rkey;
+	sig_reg->sge.addr = 0;
+	sig_reg->sge.length = scsi_transfer_length(iser_task->sc);
 
-	iser_dbg("sig_sge: addr: 0x%llx  length: %u lkey: 0x%x\n",
-		 sig_sge->addr, sig_sge->length,
-		 sig_sge->lkey);
+	iser_dbg("sig_sge: lkey: 0x%x, rkey: 0x%x, addr: 0x%llx, length: %u\n",
+		 sig_reg->sge.lkey, sig_reg->rkey, sig_reg->sge.addr,
+		 sig_reg->sge.length);
 err:
 	return ret;
 }
 
 static int iser_fast_reg_mr(struct iscsi_iser_task *iser_task,
-			    struct iser_mem_reg *mem_reg,
 			    struct iser_data_buf *mem,
+			    struct fast_reg_descriptor *desc,
 			    enum iser_reg_indicator ind,
-			    struct ib_sge *sge)
+			    struct iser_mem_reg *reg)
 {
-	struct fast_reg_descriptor *desc = mem_reg->mem_h;
 	struct ib_conn *ib_conn = &iser_task->iser_conn->ib_conn;
 	struct iser_device *device = ib_conn->device;
 	struct ib_device *ibdev = device->ib_device;
@@ -668,12 +670,13 @@ static int iser_fast_reg_mr(struct iscsi_iser_task *iser_task,
 	if (mem->dma_nents == 1) {
 		struct scatterlist *sg = mem->sg;
 
-		sge->lkey = device->mr->lkey;
-		sge->addr   = ib_sg_dma_address(ibdev, &sg[0]);
-		sge->length  = ib_sg_dma_len(ibdev, &sg[0]);
+		reg->sge.lkey = device->mr->lkey;
+		reg->rkey = device->mr->rkey;
+		reg->sge.addr = ib_sg_dma_address(ibdev, &sg[0]);
+		reg->sge.length = ib_sg_dma_len(ibdev, &sg[0]);
 
 		iser_dbg("Single DMA entry: lkey=0x%x, addr=0x%llx, length=0x%x\n",
-			 sge->lkey, sge->addr, sge->length);
+			 reg->sge.lkey, reg->sge.addr, reg->sge.length);
 		return 0;
 	}
 
@@ -723,9 +726,10 @@ static int iser_fast_reg_mr(struct iscsi_iser_task *iser_task,
 	}
 	desc->reg_indicators &= ~ind;
 
-	sge->lkey = mr->lkey;
-	sge->addr = frpl->page_list[0] + offset;
-	sge->length = size;
+	reg->sge.lkey = mr->lkey;
+	reg->rkey = mr->rkey;
+	reg->sge.addr = frpl->page_list[0] + offset;
+	reg->sge.length = size;
 
 	return ret;
 }
@@ -745,7 +749,6 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 	struct iser_data_buf *mem = &iser_task->data[cmd_dir];
 	struct iser_mem_reg *mem_reg = &iser_task->rdma_reg[cmd_dir];
 	struct fast_reg_descriptor *desc = NULL;
-	struct ib_sge data_sge;
 	int err, aligned_len;
 
 	aligned_len = iser_data_buf_aligned_len(mem, ibdev);
@@ -764,15 +767,15 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 		mem_reg->mem_h = desc;
 	}
 
-	err = iser_fast_reg_mr(iser_task, mem_reg, mem,
-			       ISER_DATA_KEY_VALID, &data_sge);
+	err = iser_fast_reg_mr(iser_task, mem, desc,
+			       ISER_DATA_KEY_VALID, mem_reg);
 	if (err)
 		goto err_reg;
 
 	if (scsi_get_prot_op(iser_task->sc) != SCSI_PROT_NORMAL) {
-		struct ib_sge prot_sge, sig_sge;
+		struct iser_mem_reg prot_reg;
 
-		memset(&prot_sge, 0, sizeof(prot_sge));
+		memset(&prot_reg, 0, sizeof(prot_reg));
 		if (scsi_prot_sg_count(iser_task->sc)) {
 			mem = &iser_task->prot[cmd_dir];
 			aligned_len = iser_data_buf_aligned_len(mem, ibdev);
@@ -785,33 +788,19 @@ int iser_reg_rdma_mem_fastreg(struct iscsi_iser_task *iser_task,
 				}
 			}
 
-			err = iser_fast_reg_mr(iser_task, mem_reg, mem,
-					       ISER_PROT_KEY_VALID, &prot_sge);
+			err = iser_fast_reg_mr(iser_task, mem, desc,
+					       ISER_PROT_KEY_VALID, &prot_reg);
 			if (err)
 				goto err_reg;
 		}
 
-		err = iser_reg_sig_mr(iser_task, desc, &data_sge,
-				      &prot_sge, &sig_sge);
+		err = iser_reg_sig_mr(iser_task, desc, mem_reg,
+				      &prot_reg, mem_reg);
 		if (err) {
 			iser_err("Failed to register signature mr\n");
 			return err;
 		}
 		desc->reg_indicators |= ISER_FASTREG_PROTECTED;
-
-		mem_reg->sge.lkey = sig_sge.lkey;
-		mem_reg->rkey = desc->pi_ctx->sig_mr->rkey;
-		mem_reg->sge.addr = sig_sge.addr;
-		mem_reg->sge.length = sig_sge.length;
-	} else {
-		if (desc)
-			mem_reg->rkey = desc->data_mr->rkey;
-		else
-			mem_reg->rkey = device->mr->rkey;
-
-		mem_reg->sge.lkey = data_sge.lkey;
-		mem_reg->sge.addr = data_sge.addr;
-		mem_reg->sge.length = data_sge.length;
 	}
 
 	return 0;
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 16/18] IB/iser: Remove code duplication for a single DMA entry

[Sagi Grimberg]

In singleton scatterlists, DMA memory registration code
is taken both for Fastreg and FMR code paths. Move it to
a function.

This patch does not change any functionality.

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Adir Lev <adirl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iser_memory.c |   48 ++++++++++++----------------
 1 files changed, 21 insertions(+), 27 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 0575052..abc979e 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -334,6 +334,24 @@ void iser_dma_unmap_task_data(struct iscsi_iser_task *iser_task,
 	ib_dma_unmap_sg(dev, data->sg, data->size, dir);
 }
 
+static int
+iser_reg_dma(struct iser_device *device, struct iser_data_buf *mem,
+	     struct iser_mem_reg *reg)
+{
+	struct scatterlist *sg = mem->sg;
+
+	reg->sge.lkey = device->mr->lkey;
+	reg->rkey = device->mr->rkey;
+	reg->sge.addr = ib_sg_dma_address(device->ib_device, &sg[0]);
+	reg->sge.length = ib_sg_dma_len(device->ib_device, &sg[0]);
+
+	iser_dbg("Single DMA entry: lkey=0x%x, rkey=0x%x, addr=0x%llx,"
+		 " length=0x%x\n", reg->sge.lkey, reg->rkey,
+		 reg->sge.addr, reg->sge.length);
+
+	return 0;
+}
+
 static int fall_to_bounce_buf(struct iscsi_iser_task *iser_task,
 			      struct iser_data_buf *mem,
 			      enum iser_data_dir cmd_dir,
@@ -461,7 +479,6 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 	int aligned_len;
 	int err;
 	int i;
-	struct scatterlist *sg;
 
 	mem_reg = &iser_task->rdma_reg[cmd_dir];
 
@@ -477,19 +494,7 @@ int iser_reg_rdma_mem_fmr(struct iscsi_iser_task *iser_task,
 
 	/* if there a single dma entry, FMR is not needed */
 	if (mem->dma_nents == 1) {
-		sg = mem->sg;
-
-		mem_reg->sge.lkey = device->mr->lkey;
-		mem_reg->rkey = device->mr->rkey;
-		mem_reg->sge.length = ib_sg_dma_len(ibdev, &sg[0]);
-		mem_reg->sge.addr = ib_sg_dma_address(ibdev, &sg[0]);
-
-		iser_dbg("PHYSICAL Mem.register: lkey: 0x%08X rkey: 0x%08X  "
-			 "va: 0x%08lX sz: %ld]\n",
-			 (unsigned int)mem_reg->sge.lkey,
-			 (unsigned int)mem_reg->rkey,
-			 (unsigned long)mem_reg->sge.addr,
-			 (unsigned long)mem_reg->sge.length);
+		return iser_reg_dma(device, mem, mem_reg);
 	} else { /* use FMR for multiple dma entries */
 		err = iser_reg_page_vec(iser_task, mem, ib_conn->fmr.page_vec,
 					mem_reg);
@@ -659,7 +664,6 @@ static int iser_fast_reg_mr(struct iscsi_iser_task *iser_task,
 {
 	struct ib_conn *ib_conn = &iser_task->iser_conn->ib_conn;
 	struct iser_device *device = ib_conn->device;
-	struct ib_device *ibdev = device->ib_device;
 	struct ib_mr *mr;
 	struct ib_fast_reg_page_list *frpl;
 	struct ib_send_wr fastreg_wr, inv_wr;
@@ -667,18 +671,8 @@ static int iser_fast_reg_mr(struct iscsi_iser_task *iser_task,
 	int ret, offset, size, plen;
 
 	/* if there a single dma entry, dma mr suffices */
-	if (mem->dma_nents == 1) {
-		struct scatterlist *sg = mem->sg;
-
-		reg->sge.lkey = device->mr->lkey;
-		reg->rkey = device->mr->rkey;
-		reg->sge.addr = ib_sg_dma_address(ibdev, &sg[0]);
-		reg->sge.length = ib_sg_dma_len(ibdev, &sg[0]);
-
-		iser_dbg("Single DMA entry: lkey=0x%x, addr=0x%llx, length=0x%x\n",
-			 reg->sge.lkey, reg->sge.addr, reg->sge.length);
-		return 0;
-	}
+	if (mem->dma_nents == 1)
+		return iser_reg_dma(device, mem, reg);
 
 	if (ind == ISER_DATA_KEY_VALID) {
 		mr = desc->data_mr;
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 17/18] IB/iser: Bump version to 1.6

[Sagi Grimberg]

Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index 5fd0963..a39645a 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -69,7 +69,7 @@
 
 #define DRV_NAME	"iser"
 #define PFX		DRV_NAME ": "
-#define DRV_VER		"1.5"
+#define DRV_VER		"1.6"
 
 #define iser_dbg(fmt, arg...)				 \
 	do {						 \
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch v3 18/18] IB/iser: Rewrite bounce buffer code path

[Sagi Grimberg]

In some rare cases, IO operations may be not aligned to page
boundaries. This prevents iser from performing fast memory
registration. In order to overcome that iser uses a bounce
buffer to carry the transaction. We basically allocate a buffer
in the size of the transaction and perform a copy.

The buffer allocation using kmalloc is too restrictive since it
requires higher order (atomic) allocations for large transactions
(which may result in memory exhaustion fairly fast for some workloads).
We rewrite the bounce buffer code path to allocate scattered pages
and perform a copy between the transaction sg and the bounce sg.

Reported-by: Alex Lyakas <alex-liwybQIp9wHp1aIWIFLecAC/G2K4zDHf@public.gmane.org>
Signed-off-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/ulp/iser/iscsi_iser.h     |    8 +-
 drivers/infiniband/ulp/iser/iser_initiator.c |    8 +-
 drivers/infiniband/ulp/iser/iser_memory.c    |  211 ++++++++++++++++----------
 3 files changed, 138 insertions(+), 89 deletions(-)

diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h b/drivers/infiniband/ulp/iser/iscsi_iser.h
index a39645a..262ba1f 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -222,12 +222,9 @@ enum iser_data_dir {
  * @size:         num entries of this sg
  * @data_len:     total beffer byte len
  * @dma_nents:    returned by dma_map_sg
- * @copy_buf:     allocated copy buf for SGs unaligned
- *                for rdma which are copied
  * @orig_sg:      pointer to the original sg list (in case
  *                we used a copy)
- * @sg_single:    SG-ified clone of a non SG SC or
- *                unaligned SG
+ * @orig_size:    num entris of orig sg list
  */
 struct iser_data_buf {
 	struct scatterlist *sg;
@@ -235,8 +232,7 @@ struct iser_data_buf {
 	unsigned long      data_len;
 	unsigned int       dma_nents;
 	struct scatterlist *orig_sg;
-	char               *copy_buf;
-	struct scatterlist sg_single;
+	unsigned int       orig_size;
   };
 
 /* fwd declarations */
diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c b/drivers/infiniband/ulp/iser/iser_initiator.c
index b2e3b77..3e2118e 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -674,28 +674,28 @@ void iser_task_rdma_finalize(struct iscsi_iser_task *iser_task)
 	/* if we were reading, copy back to unaligned sglist,
 	 * anyway dma_unmap and free the copy
 	 */
-	if (iser_task->data[ISER_DIR_IN].copy_buf) {
+	if (iser_task->data[ISER_DIR_IN].orig_sg) {
 		is_rdma_data_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->data[ISER_DIR_IN],
 						ISER_DIR_IN);
 	}
 
-	if (iser_task->data[ISER_DIR_OUT].copy_buf) {
+	if (iser_task->data[ISER_DIR_OUT].orig_sg) {
 		is_rdma_data_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->data[ISER_DIR_OUT],
 						ISER_DIR_OUT);
 	}
 
-	if (iser_task->prot[ISER_DIR_IN].copy_buf) {
+	if (iser_task->prot[ISER_DIR_IN].orig_sg) {
 		is_rdma_prot_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->prot[ISER_DIR_IN],
 						ISER_DIR_IN);
 	}
 
-	if (iser_task->prot[ISER_DIR_OUT].copy_buf) {
+	if (iser_task->prot[ISER_DIR_OUT].orig_sg) {
 		is_rdma_prot_aligned = 0;
 		iser_finalize_rdma_unaligned_sg(iser_task,
 						&iser_task->prot[ISER_DIR_OUT],
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index abc979e..f0cdc96 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -39,7 +39,112 @@
 
 #include "iscsi_iser.h"
 
-#define ISER_KMALLOC_THRESHOLD 0x20000 /* 128K - kmalloc limit */
+static void
+iser_free_bounce_sg(struct iser_data_buf *data)
+{
+	struct scatterlist *sg;
+	int count;
+
+	for_each_sg(data->sg, sg, data->size, count)
+		__free_page(sg_page(sg));
+
+	kfree(data->sg);
+
+	data->sg = data->orig_sg;
+	data->size = data->orig_size;
+	data->orig_sg = NULL;
+	data->orig_size = 0;
+}
+
+static int
+iser_alloc_bounce_sg(struct iser_data_buf *data)
+{
+	struct scatterlist *sg;
+	struct page *page;
+	unsigned long length = data->data_len;
+	int i = 0, nents = DIV_ROUND_UP(length, PAGE_SIZE);
+
+	sg = kcalloc(nents, sizeof(*sg), GFP_ATOMIC);
+	if (!sg)
+		goto err;
+
+	sg_init_table(sg, nents);
+	while (length) {
+		u32 page_len = min_t(u32, length, PAGE_SIZE);
+
+		page = alloc_page(GFP_ATOMIC);
+		if (!page)
+			goto err;
+
+		sg_set_page(&sg[i], page, page_len, 0);
+		length -= page_len;
+		i++;
+	}
+
+	data->orig_sg = data->sg;
+	data->orig_size = data->size;
+	data->sg = sg;
+	data->size = nents;
+
+	return 0;
+
+err:
+	for (; i > 0; i--)
+		__free_page(sg_page(&sg[i - 1]));
+	kfree(sg);
+
+	return -ENOMEM;
+}
+
+static void
+iser_copy_bounce(struct iser_data_buf *data, bool to_buffer)
+{
+	struct scatterlist *osg, *bsg = data->sg;
+	void *oaddr, *baddr;
+	unsigned int left = data->data_len;
+	unsigned int bsg_off = 0;
+	int i;
+
+	for_each_sg(data->orig_sg, osg, data->orig_size, i) {
+		unsigned int copy_len, osg_off = 0;
+
+		oaddr = kmap_atomic(sg_page(osg)) + osg->offset;
+		copy_len = min(left, osg->length);
+		while (copy_len) {
+			unsigned int len = min(copy_len, bsg->length - bsg_off);
+
+			baddr = kmap_atomic(sg_page(bsg)) + bsg->offset;
+			if (to_buffer)
+				memcpy(baddr + bsg_off, oaddr + osg_off, len);
+			else
+				memcpy(oaddr + osg_off, baddr + bsg_off, len);
+
+			kunmap_atomic(baddr - bsg->offset);
+			osg_off += len;
+			bsg_off += len;
+			copy_len -= len;
+
+			if (bsg_off >= bsg->length) {
+				bsg = sg_next(bsg);
+				bsg_off = 0;
+			}
+		}
+		kunmap_atomic(oaddr - osg->offset);
+		left -= osg_off;
+	}
+}
+
+static inline void
+iser_copy_from_bounce(struct iser_data_buf *data)
+{
+	iser_copy_bounce(data, false);
+}
+
+static inline void
+iser_copy_to_bounce(struct iser_data_buf *data)
+{
+	iser_copy_bounce(data, true);
+}
 
 struct fast_reg_descriptor *
 iser_reg_desc_get(struct ib_conn *ib_conn)
@@ -75,52 +180,32 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 					enum iser_data_dir cmd_dir)
 {
 	struct ib_device *dev = iser_task->iser_conn->ib_conn.device->ib_device;
-	struct scatterlist *sgl = data->sg;
-	struct scatterlist *sg;
-	char *mem = NULL;
-	unsigned long  cmd_data_len = data->data_len;
-	int dma_nents, i;
-
-	if (cmd_data_len > ISER_KMALLOC_THRESHOLD)
-		mem = (void *)__get_free_pages(GFP_ATOMIC,
-		      ilog2(roundup_pow_of_two(cmd_data_len)) - PAGE_SHIFT);
-	else
-		mem = kmalloc(cmd_data_len, GFP_ATOMIC);
+	int rc;
 
-	if (mem == NULL) {
-		iser_err("Failed to allocate mem size %d %d for copying sglist\n",
-			 data->size, (int)cmd_data_len);
-		return -ENOMEM;
+	rc = iser_alloc_bounce_sg(data);
+	if (rc) {
+		iser_err("Failed to allocate bounce for data len %lu\n",
+			 data->data_len);
+		return rc;
 	}
 
-	if (cmd_dir == ISER_DIR_OUT) {
-		/* copy the unaligned sg the buffer which is used for RDMA */
-		char *p, *from;
-
-		sgl = data->sg;
-		p = mem;
-		for_each_sg(sgl, sg, data->size, i) {
-			from = kmap_atomic(sg_page(sg));
-			memcpy(p,
-			       from + sg->offset,
-			       sg->length);
-			kunmap_atomic(from);
-			p += sg->length;
-		}
-	}
-
-	sg_init_one(&data->sg_single, mem, cmd_data_len);
-	data->orig_sg = data->sg;
-	data->sg = &data->sg_single;
-	data->copy_buf = mem;
-	dma_nents = ib_dma_map_sg(dev, data->sg, 1,
-				  (cmd_dir == ISER_DIR_OUT) ?
-				  DMA_TO_DEVICE : DMA_FROM_DEVICE);
-	BUG_ON(dma_nents == 0);
+	if (cmd_dir == ISER_DIR_OUT)
+		iser_copy_to_bounce(data);
 
-	data->dma_nents = dma_nents;
+	data->dma_nents = ib_dma_map_sg(dev, data->sg, data->size,
+					(cmd_dir == ISER_DIR_OUT) ?
+					DMA_TO_DEVICE : DMA_FROM_DEVICE);
+	if (!data->dma_nents) {
+		iser_err("Got dma_nents %d, something went wrong...\n",
+			 data->dma_nents);
+		rc = -ENOMEM;
+		goto err;
+	}
 
 	return 0;
+err:
+	iser_free_bounce_sg(data);
+	return rc;
 }
 
 /**
@@ -131,48 +216,16 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_task *iser_task,
 				     struct iser_data_buf *data,
 				     enum iser_data_dir cmd_dir)
 {
-	struct ib_device *dev;
-	unsigned long  cmd_data_len;
-
-	dev = iser_task->iser_conn->ib_conn.device->ib_device;
+	struct ib_device *dev = iser_task->iser_conn->ib_conn.device->ib_device;
 
-	ib_dma_unmap_sg(dev, data->sg, 1,
+	ib_dma_unmap_sg(dev, data->sg, data->size,
 			(cmd_dir == ISER_DIR_OUT) ?
 			DMA_TO_DEVICE : DMA_FROM_DEVICE);
 
-	if (cmd_dir == ISER_DIR_IN) {
-		char *mem;
-		struct scatterlist *sgl, *sg;
-		unsigned char *p, *to;
-		unsigned int sg_size;
-		int i;
-
-		/* copy back read RDMA to unaligned sg */
-		mem = data->copy_buf;
-
-		sgl = data->sg;
-		sg_size = data->size;
-
-		p = mem;
-		for_each_sg(sgl, sg, sg_size, i) {
-			to = kmap_atomic(sg_page(sg));
-			memcpy(to + sg->offset,
-			       p,
-			       sg->length);
-			kunmap_atomic(to);
-			p += sg->length;
-		}
-	}
-
-	cmd_data_len = data->data_len;
-
-	if (cmd_data_len > ISER_KMALLOC_THRESHOLD)
-		free_pages((unsigned long)data->copy_buf,
-			   ilog2(roundup_pow_of_two(cmd_data_len)) - PAGE_SHIFT);
-	else
-		kfree(data->copy_buf);
+	if (cmd_dir == ISER_DIR_IN)
+		iser_copy_from_bounce(data);
 
-	data->copy_buf = NULL;
+	iser_free_bounce_sg(data);
 }
 
 #define IS_4K_ALIGNED(addr)	((((unsigned long)addr) & ~MASK_4K) == 0)
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: iser patches for kernel 4.1 [v3]

[Or Gerlitz]

On 4/14/2015 6:08 PM, Sagi Grimberg wrote:
> Hi Roland,
>
> This set contains bug fixes as well as code refactoring patches
>
> - Patches 1-3:	   Bug fixes (stable material)
> - Patches 4-6:     Bounce buffer related code cleanups
> - Patches 7-12,16: Refactoring
> - Patches 13-15:   Minor optimizations
> - Patch 17:        Version bump
> - Patch 18:        Bounce buffer rewrite to handle situations where
>                     large transfers are unaligned causing iser to
> 		   use high order allocations
>
> Changes from v2:
> - Fix regression in DIX mode (introduced in patch 15) by incorrectly
>    setting the protection information scatter-entry.

Sagi, OTOH Roland made a note that he applied V2, hence you were 
supposed to send incremental fix on top of it (for-next time), OTHO, he 
didn't make his applying of the code public, so sending the whole series 
makes sense too. Just to be on the safe side of things, since the merge 
window is open, please send also an incremental fix.

Or.

>
> Changes from v1:
> - Addressed review comments (patch 18) from Alex Lyakas on bounce buffer
>    re-write code (kmap/kunmap mismatch, possible use-after-free,
>    and double free).

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html