[PATCH for-next 0/7] FIXME and other fixes

[PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

Dean fixes the FIXME that was left by Jason in the code to use the interval
notifier. There are also fixes for other things like splitting our counter
allocation to better align with the way the core is moving. 

---

Dean Luick (7):
      IB/hfi1: Remove redundant pageidx variable
      IB/hfi1: Assign npages earlier
      IB/hfi1: Consolidate the creation of user TIDs
      IB/hfi1: Improve TID validity checking
      IB/hfi1: Split IB counter allocation
      IBh/hfi1: Update RMT size calculation
      IB/hfi1: Use dma_mmap_coherent for matching buffers


 drivers/infiniband/hw/hfi1/chip.c         | 59 +++++++++--------
 drivers/infiniband/hw/hfi1/exp_rcv.h      |  5 +-
 drivers/infiniband/hw/hfi1/file_ops.c     | 81 ++++++++++++++++-------
 drivers/infiniband/hw/hfi1/user_exp_rcv.c | 55 ++++++---------
 drivers/infiniband/hw/hfi1/verbs.c        | 81 +++++++++--------------
 5 files changed, 146 insertions(+), 135 deletions(-)

--
-Denny

[PATCH for-next 1/7] IB/hfi1: Remove redundant pageidx variable

[Dennis Dalessandro]

From: Dean Luick <dean.luick@cornelisnetworks.com>

In hfi1_user_exp_rcv_setup(), variable pageidx mirrors
variable tididx.  Remove pageidx and its use as an argument
to program_rcvarray().

Signed-off-by: Dean Luick <dean.luick@cornelisnetworks.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
 drivers/infiniband/hw/hfi1/user_exp_rcv.c |   22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
index b02f2f0809c8..c2099510817c 100644
--- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c
+++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
@@ -27,8 +27,7 @@ static bool tid_cover_invalidate(struct mmu_interval_notifier *mni,
 			         const struct mmu_notifier_range *range,
 			         unsigned long cur_seq);
 static int program_rcvarray(struct hfi1_filedata *fd, struct tid_user_buf *,
-			    struct tid_group *grp,
-			    unsigned int start, u16 count,
+			    struct tid_group *grp, u16 count,
 			    u32 *tidlist, unsigned int *tididx,
 			    unsigned int *pmapped);
 static int unprogram_rcvarray(struct hfi1_filedata *fd, u32 tidinfo);
@@ -256,7 +255,7 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd,
 	int ret = 0, need_group = 0, pinned;
 	struct hfi1_ctxtdata *uctxt = fd->uctxt;
 	struct hfi1_devdata *dd = uctxt->dd;
-	unsigned int ngroups, pageidx = 0, pageset_count,
+	unsigned int ngroups, pageset_count,
 		tididx = 0, mapped, mapped_pages = 0;
 	u32 *tidlist = NULL;
 	struct tid_user_buf *tidbuf;
@@ -337,7 +336,7 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd,
 			tid_group_pop(&uctxt->tid_group_list);
 
 		ret = program_rcvarray(fd, tidbuf, grp,
-				       pageidx, dd->rcv_entries.group_size,
+				       dd->rcv_entries.group_size,
 				       tidlist, &tididx, &mapped);
 		/*
 		 * If there was a failure to program the RcvArray
@@ -353,11 +352,10 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd,
 
 		tid_group_add_tail(grp, &uctxt->tid_full_list);
 		ngroups--;
-		pageidx += ret;
 		mapped_pages += mapped;
 	}
 
-	while (pageidx < pageset_count) {
+	while (tididx < pageset_count) {
 		struct tid_group *grp, *ptr;
 		/*
 		 * If we don't have any partially used tid groups, check
@@ -379,11 +377,11 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd,
 		 */
 		list_for_each_entry_safe(grp, ptr, &uctxt->tid_used_list.list,
 					 list) {
-			unsigned use = min_t(unsigned, pageset_count - pageidx,
+			unsigned use = min_t(unsigned, pageset_count - tididx,
 					     grp->size - grp->used);
 
 			ret = program_rcvarray(fd, tidbuf, grp,
-					       pageidx, use, tidlist,
+					       use, tidlist,
 					       &tididx, &mapped);
 			if (ret < 0) {
 				hfi1_cdbg(TID,
@@ -395,11 +393,10 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd,
 					tid_group_move(grp,
 						       &uctxt->tid_used_list,
 						       &uctxt->tid_full_list);
-				pageidx += ret;
 				mapped_pages += mapped;
 				need_group = 0;
 				/* Check if we are done so we break out early */
-				if (pageidx >= pageset_count)
+				if (tididx >= pageset_count)
 					break;
 			} else if (WARN_ON(ret == 0)) {
 				/*
@@ -643,7 +640,6 @@ static u32 find_phys_blocks(struct tid_user_buf *tidbuf, unsigned int npages)
  *	  struct tid_pageset holding information on physically contiguous
  *	  chunks from the user buffer), and other fields.
  * @grp: RcvArray group
- * @start: starting index into sets array
  * @count: number of struct tid_pageset's to program
  * @tidlist: the array of u32 elements when the information about the
  *           programmed RcvArray entries is to be encoded.
@@ -663,14 +659,14 @@ static u32 find_phys_blocks(struct tid_user_buf *tidbuf, unsigned int npages)
  * number of RcvArray entries programmed.
  */
 static int program_rcvarray(struct hfi1_filedata *fd, struct tid_user_buf *tbuf,
-			    struct tid_group *grp,
-			    unsigned int start, u16 count,
+			    struct tid_group *grp, u16 count,
 			    u32 *tidlist, unsigned int *tididx,
 			    unsigned int *pmapped)
 {
 	struct hfi1_ctxtdata *uctxt = fd->uctxt;
 	struct hfi1_devdata *dd = uctxt->dd;
 	u16 idx;
+	unsigned int start = *tididx;
 	u32 tidinfo = 0, rcventry, useidx = 0;
 	int mapped = 0;
 

[PATCH for-next 2/7] IB/hfi1: Assign npages earlier

[Dennis Dalessandro]

From: Dean Luick <dean.luick@cornelisnetworks.com>

Improve code clarity and enable earlier use of
tidbuf->npages by moving its assignment to
structure creation time.

Signed-off-by: Dean Luick <dean.luick@cornelisnetworks.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
 drivers/infiniband/hw/hfi1/user_exp_rcv.c |    9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
index c2099510817c..0e51c95d3f1d 100644
--- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c
+++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
@@ -159,16 +159,11 @@ static void unpin_rcv_pages(struct hfi1_filedata *fd,
 static int pin_rcv_pages(struct hfi1_filedata *fd, struct tid_user_buf *tidbuf)
 {
 	int pinned;
-	unsigned int npages;
+	unsigned int npages = tidbuf->npages;
 	unsigned long vaddr = tidbuf->vaddr;
 	struct page **pages = NULL;
 	struct hfi1_devdata *dd = fd->uctxt->dd;
 
-	/* Get the number of pages the user buffer spans */
-	npages = num_user_pages(vaddr, tidbuf->length);
-	if (!npages)
-		return -EINVAL;
-
 	if (npages > fd->uctxt->expected_count) {
 		dd_dev_err(dd, "Expected buffer too big\n");
 		return -EINVAL;
@@ -195,7 +190,6 @@ static int pin_rcv_pages(struct hfi1_filedata *fd, struct tid_user_buf *tidbuf)
 		return pinned;
 	}
 	tidbuf->pages = pages;
-	tidbuf->npages = npages;
 	fd->tid_n_pinned += pinned;
 	return pinned;
 }
@@ -273,6 +267,7 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd,
 	mutex_init(&tidbuf->cover_mutex);
 	tidbuf->vaddr = tinfo->vaddr;
 	tidbuf->length = tinfo->length;
+	tidbuf->npages = num_user_pages(tidbuf->vaddr, tidbuf->length);
 	tidbuf->psets = kcalloc(uctxt->expected_count, sizeof(*tidbuf->psets),
 				GFP_KERNEL);
 	if (!tidbuf->psets) {

[PATCH for-next 3/7] IB/hfi1: Consolidate the creation of user TIDs

[Dennis Dalessandro]

From: Dean Luick <dean.luick@cornelisnetworks.com>

The function rcventry2tidinfo() only creates part of
a TID and all calls to it are only used to make a user
TID.  Consolidate all usage into a single routine.

Signed-off-by: Dean Luick <dean.luick@cornelisnetworks.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
 drivers/infiniband/hw/hfi1/exp_rcv.h      |    5 +++--
 drivers/infiniband/hw/hfi1/user_exp_rcv.c |    8 +++-----
 2 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/exp_rcv.h b/drivers/infiniband/hw/hfi1/exp_rcv.h
index c6291bbf723c..41f7fe5d1839 100644
--- a/drivers/infiniband/hw/hfi1/exp_rcv.h
+++ b/drivers/infiniband/hw/hfi1/exp_rcv.h
@@ -133,12 +133,13 @@ static inline struct tid_group *tid_group_pop(struct exp_tid_set *set)
 	return grp;
 }
 
-static inline u32 rcventry2tidinfo(u32 rcventry)
+static inline u32 create_tid(u32 rcventry, u32 npages)
 {
 	u32 pair = rcventry & ~0x1;
 
 	return EXP_TID_SET(IDX, pair >> 1) |
-		EXP_TID_SET(CTRL, 1 << (rcventry - pair));
+		EXP_TID_SET(CTRL, 1 << (rcventry - pair)) |
+		EXP_TID_SET(LEN, npages);
 }
 
 /**
diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
index 0e51c95d3f1d..0229cf210431 100644
--- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c
+++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
@@ -706,8 +706,7 @@ static int program_rcvarray(struct hfi1_filedata *fd, struct tid_user_buf *tbuf,
 			return ret;
 		mapped += npages;
 
-		tidinfo = rcventry2tidinfo(rcventry - uctxt->expected_base) |
-			EXP_TID_SET(LEN, npages);
+		tidinfo = create_tid(rcventry - uctxt->expected_base, npages);
 		tidlist[(*tididx)++] = tidinfo;
 		grp->used++;
 		grp->map |= 1 << useidx++;
@@ -916,9 +915,8 @@ static bool tid_rb_invalidate(struct mmu_interval_notifier *mni,
 	spin_lock(&fdata->invalid_lock);
 	if (fdata->invalid_tid_idx < uctxt->expected_count) {
 		fdata->invalid_tids[fdata->invalid_tid_idx] =
-			rcventry2tidinfo(node->rcventry - uctxt->expected_base);
-		fdata->invalid_tids[fdata->invalid_tid_idx] |=
-			EXP_TID_SET(LEN, node->npages);
+			create_tid(node->rcventry - uctxt->expected_base,
+				   node->npages);
 		if (!fdata->invalid_tid_idx) {
 			unsigned long *ev;
 

[PATCH for-next 4/7] IB/hfi1: Improve TID validity checking

[Dennis Dalessandro]

From: Dean Luick <dean.luick@cornelisnetworks.com>

Correct and improve validity checking of user supplied TIDs.
A tidctrl value of 0 is invalid.  Verify that the final
index is in range, not an intermediate value.

Signed-off-by: Dean Luick <dean.luick@cornelisnetworks.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
 drivers/infiniband/hw/hfi1/user_exp_rcv.c |   16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
index 0229cf210431..96058baf36ed 100644
--- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c
+++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c
@@ -790,20 +790,20 @@ static int unprogram_rcvarray(struct hfi1_filedata *fd, u32 tidinfo)
 	struct hfi1_ctxtdata *uctxt = fd->uctxt;
 	struct hfi1_devdata *dd = uctxt->dd;
 	struct tid_rb_node *node;
-	u8 tidctrl = EXP_TID_GET(tidinfo, CTRL);
+	u32 tidctrl = EXP_TID_GET(tidinfo, CTRL);
 	u32 tididx = EXP_TID_GET(tidinfo, IDX) << 1, rcventry;
 
-	if (tididx >= uctxt->expected_count) {
-		dd_dev_err(dd, "Invalid RcvArray entry (%u) index for ctxt %u\n",
-			   tididx, uctxt->ctxt);
-		return -EINVAL;
-	}
-
-	if (tidctrl == 0x3)
+	if (tidctrl == 0x3 || tidctrl == 0x0)
 		return -EINVAL;
 
 	rcventry = tididx + (tidctrl - 1);
 
+	if (rcventry >= uctxt->expected_count) {
+		dd_dev_err(dd, "Invalid RcvArray entry (%u) index for ctxt %u\n",
+			   rcventry, uctxt->ctxt);
+		return -EINVAL;
+	}
+
 	node = fd->entry_to_rb[rcventry];
 	if (!node || node->rcventry != (uctxt->expected_base + rcventry))
 		return -EBADF;

[PATCH for-next 5/7] IB/hfi1: Split IB counter allocation

[Dennis Dalessandro]

From: Dean Luick <dean.luick@cornelisnetworks.com>

Split the IB device and port counter allocation.  Remove
the need for a lock.  Clean up pointer usage.

Signed-off-by: Dean Luick <dean.luick@cornelisnetworks.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
 drivers/infiniband/hw/hfi1/verbs.c |   81 ++++++++++++++----------------------
 1 file changed, 32 insertions(+), 49 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index e6e17984553c..7f6d7fc7951d 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -1598,13 +1598,11 @@ static const char * const driver_cntr_names[] = {
 	"DRIVER_EgrHdrFull"
 };
 
-static DEFINE_MUTEX(cntr_names_lock); /* protects the *_cntr_names bufers */
 static struct rdma_stat_desc *dev_cntr_descs;
 static struct rdma_stat_desc *port_cntr_descs;
 int num_driver_cntrs = ARRAY_SIZE(driver_cntr_names);
 static int num_dev_cntrs;
 static int num_port_cntrs;
-static int cntr_names_initialized;
 
 /*
  * Convert a list of names separated by '\n' into an array of NULL terminated
@@ -1615,8 +1613,8 @@ static int init_cntr_names(const char *names_in, const size_t names_len,
 			   int num_extra_names, int *num_cntrs,
 			   struct rdma_stat_desc **cntr_descs)
 {
-	struct rdma_stat_desc *q;
-	char *names_out, *p;
+	struct rdma_stat_desc *names_out;
+	char *p;
 	int i, n;
 
 	n = 0;
@@ -1624,65 +1622,45 @@ static int init_cntr_names(const char *names_in, const size_t names_len,
 		if (names_in[i] == '\n')
 			n++;
 
-	names_out =
-		kzalloc((n + num_extra_names) * sizeof(*q) + names_len,
-			GFP_KERNEL);
+	names_out = kzalloc((n + num_extra_names) * sizeof(*names_out)
+				+ names_len,
+			    GFP_KERNEL);
 	if (!names_out) {
 		*num_cntrs = 0;
 		*cntr_descs = NULL;
 		return -ENOMEM;
 	}
 
-	p = names_out + (n + num_extra_names) * sizeof(*q);
+	p = (char *)&names_out[n + num_extra_names];
 	memcpy(p, names_in, names_len);
 
-	q = (struct rdma_stat_desc *)names_out;
 	for (i = 0; i < n; i++) {
-		q[i].name = p;
+		names_out[i].name = p;
 		p = strchr(p, '\n');
 		*p++ = '\0';
 	}
 
 	*num_cntrs = n;
-	*cntr_descs = (struct rdma_stat_desc *)names_out;
+	*cntr_descs = names_out;
 	return 0;
 }
 
-static int init_counters(struct ib_device *ibdev)
-{
-	struct hfi1_devdata *dd = dd_from_ibdev(ibdev);
-	int i, err = 0;
-
-	mutex_lock(&cntr_names_lock);
-	if (cntr_names_initialized)
-		goto out_unlock;
-
-	err = init_cntr_names(dd->cntrnames, dd->cntrnameslen, num_driver_cntrs,
-			      &num_dev_cntrs, &dev_cntr_descs);
-	if (err)
-		goto out_unlock;
-
-	for (i = 0; i < num_driver_cntrs; i++)
-		dev_cntr_descs[num_dev_cntrs + i].name = driver_cntr_names[i];
-
-	err = init_cntr_names(dd->portcntrnames, dd->portcntrnameslen, 0,
-			      &num_port_cntrs, &port_cntr_descs);
-	if (err) {
-		kfree(dev_cntr_descs);
-		dev_cntr_descs = NULL;
-		goto out_unlock;
-	}
-	cntr_names_initialized = 1;
-
-out_unlock:
-	mutex_unlock(&cntr_names_lock);
-	return err;
-}
-
 static struct rdma_hw_stats *hfi1_alloc_hw_device_stats(struct ib_device *ibdev)
 {
-	if (init_counters(ibdev))
-		return NULL;
+	if (!dev_cntr_descs) {
+		struct hfi1_devdata *dd = dd_from_ibdev(ibdev);
+		int i, err;
+
+		err = init_cntr_names(dd->cntrnames, dd->cntrnameslen,
+				      num_driver_cntrs,
+				      &num_dev_cntrs, &dev_cntr_descs);
+		if (err)
+			return NULL;
+
+		for (i = 0; i < num_driver_cntrs; i++)
+			dev_cntr_descs[num_dev_cntrs + i].name =
+							driver_cntr_names[i];
+	}
 	return rdma_alloc_hw_stats_struct(dev_cntr_descs,
 					  num_dev_cntrs + num_driver_cntrs,
 					  RDMA_HW_STATS_DEFAULT_LIFESPAN);
@@ -1691,8 +1669,16 @@ static struct rdma_hw_stats *hfi1_alloc_hw_device_stats(struct ib_device *ibdev)
 static struct rdma_hw_stats *hfi_alloc_hw_port_stats(struct ib_device *ibdev,
 						     u32 port_num)
 {
-	if (init_counters(ibdev))
-		return NULL;
+	if (!port_cntr_descs) {
+		struct hfi1_devdata *dd = dd_from_ibdev(ibdev);
+		int err;
+
+		err = init_cntr_names(dd->portcntrnames, dd->portcntrnameslen,
+				      0,
+				      &num_port_cntrs, &port_cntr_descs);
+		if (err)
+			return NULL;
+	}
 	return rdma_alloc_hw_stats_struct(port_cntr_descs, num_port_cntrs,
 					  RDMA_HW_STATS_DEFAULT_LIFESPAN);
 }
@@ -1917,13 +1903,10 @@ void hfi1_unregister_ib_device(struct hfi1_devdata *dd)
 	del_timer_sync(&dev->mem_timer);
 	verbs_txreq_exit(dev);
 
-	mutex_lock(&cntr_names_lock);
 	kfree(dev_cntr_descs);
 	kfree(port_cntr_descs);
 	dev_cntr_descs = NULL;
 	port_cntr_descs = NULL;
-	cntr_names_initialized = 0;
-	mutex_unlock(&cntr_names_lock);
 }
 
 void hfi1_cnp_rcv(struct hfi1_packet *packet)

[PATCH for-next 6/7] IBh/hfi1: Update RMT size calculation

[Dennis Dalessandro]

From: Dean Luick <dean.luick@cornelisnetworks.com>

Fix possible RMT overflow:  Use the correct netdev size.
Don't allow adjusted user contexts to go negative.

Fix QOS calculation: Send kernel context count as an argument since
dd->n_krcv_queues is not yet set up in earliest call.  Do not include
the control context in the QOS calculation.  Use the same sized
variable to find the max of krcvq[] entries.

Update the RMT count explanation to make more sense.

Signed-off-by: Dean Luick <dean.luick@cornelisnetworks.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
 drivers/infiniband/hw/hfi1/chip.c |   59 ++++++++++++++++++++-----------------
 1 file changed, 32 insertions(+), 27 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index ebe970f76232..90b672feed83 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -1056,7 +1056,7 @@ static void read_link_down_reason(struct hfi1_devdata *dd, u8 *ldr);
 static void handle_temp_err(struct hfi1_devdata *dd);
 static void dc_shutdown(struct hfi1_devdata *dd);
 static void dc_start(struct hfi1_devdata *dd);
-static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp,
+static int qos_rmt_entries(unsigned int n_krcv_queues, unsigned int *mp,
 			   unsigned int *np);
 static void clear_full_mgmt_pkey(struct hfi1_pportdata *ppd);
 static int wait_link_transfer_active(struct hfi1_devdata *dd, int wait_ms);
@@ -13362,7 +13362,6 @@ static int set_up_context_variables(struct hfi1_devdata *dd)
 	int ret;
 	unsigned ngroups;
 	int rmt_count;
-	int user_rmt_reduced;
 	u32 n_usr_ctxts;
 	u32 send_contexts = chip_send_contexts(dd);
 	u32 rcv_contexts = chip_rcv_contexts(dd);
@@ -13421,28 +13420,34 @@ static int set_up_context_variables(struct hfi1_devdata *dd)
 					 (num_kernel_contexts + n_usr_ctxts),
 					 &node_affinity.real_cpu_mask);
 	/*
-	 * The RMT entries are currently allocated as shown below:
-	 * 1. QOS (0 to 128 entries);
-	 * 2. FECN (num_kernel_context - 1 + num_user_contexts +
-	 *    num_netdev_contexts);
-	 * 3. netdev (num_netdev_contexts).
-	 * It should be noted that FECN oversubscribe num_netdev_contexts
-	 * entries of RMT because both netdev and PSM could allocate any receive
-	 * context between dd->first_dyn_alloc_text and dd->num_rcv_contexts,
-	 * and PSM FECN must reserve an RMT entry for each possible PSM receive
-	 * context.
+	 * RMT entries are allocated as follows:
+	 * 1. QOS (0 to 128 entries)
+	 * 2. FECN (num_kernel_context - 1 [a] + num_user_contexts +
+	 *          num_netdev_contexts [b])
+	 * 3. netdev (NUM_NETDEV_MAP_ENTRIES)
+	 *
+	 * Notes:
+	 * [a] Kernel contexts (except control) are included in FECN if kernel
+	 *     TID_RDMA is active.
+	 * [b] Netdev and user contexts are randomly allocated from the same
+	 *     context pool, so FECN must cover all contexts in the pool.
 	 */
-	rmt_count = qos_rmt_entries(dd, NULL, NULL) + (num_netdev_contexts * 2);
-	if (HFI1_CAP_IS_KSET(TID_RDMA))
-		rmt_count += num_kernel_contexts - 1;
-	if (rmt_count + n_usr_ctxts > NUM_MAP_ENTRIES) {
-		user_rmt_reduced = NUM_MAP_ENTRIES - rmt_count;
-		dd_dev_err(dd,
-			   "RMT size is reducing the number of user receive contexts from %u to %d\n",
-			   n_usr_ctxts,
-			   user_rmt_reduced);
-		/* recalculate */
-		n_usr_ctxts = user_rmt_reduced;
+	rmt_count = qos_rmt_entries(num_kernel_contexts - 1, NULL, NULL)
+		    + (HFI1_CAP_IS_KSET(TID_RDMA) ? num_kernel_contexts - 1
+						  : 0)
+		    + n_usr_ctxts
+		    + num_netdev_contexts
+		    + NUM_NETDEV_MAP_ENTRIES;
+	if (rmt_count > NUM_MAP_ENTRIES) {
+		int over = rmt_count - NUM_MAP_ENTRIES;
+		/* try to squish user contexts, minimum of 1 */
+		if (over >= n_usr_ctxts) {
+			dd_dev_err(dd, "RMT overflow: reduce the requested number of contexts\n");
+			return -EINVAL;
+		}
+		dd_dev_err(dd, "RMT overflow: reducing # user contexts from %u to %u\n",
+			   n_usr_ctxts, n_usr_ctxts - over);
+		n_usr_ctxts -= over;
 	}
 
 	/* the first N are kernel contexts, the rest are user/netdev contexts */
@@ -14299,15 +14304,15 @@ static void clear_rsm_rule(struct hfi1_devdata *dd, u8 rule_index)
 }
 
 /* return the number of RSM map table entries that will be used for QOS */
-static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp,
+static int qos_rmt_entries(unsigned int n_krcv_queues, unsigned int *mp,
 			   unsigned int *np)
 {
 	int i;
 	unsigned int m, n;
-	u8 max_by_vl = 0;
+	uint max_by_vl = 0;
 
 	/* is QOS active at all? */
-	if (dd->n_krcv_queues <= MIN_KERNEL_KCTXTS ||
+	if (n_krcv_queues < MIN_KERNEL_KCTXTS ||
 	    num_vls == 1 ||
 	    krcvqsset <= 1)
 		goto no_qos;
@@ -14365,7 +14370,7 @@ static void init_qos(struct hfi1_devdata *dd, struct rsm_map_table *rmt)
 
 	if (!rmt)
 		goto bail;
-	rmt_entries = qos_rmt_entries(dd, &m, &n);
+	rmt_entries = qos_rmt_entries(dd->n_krcv_queues - 1, &m, &n);
 	if (rmt_entries == 0)
 		goto bail;
 	qpns_per_vl = 1 << m;

[PATCH for-next 7/7] IB/hfi1: Use dma_mmap_coherent for matching buffers

[Dennis Dalessandro]

From: Dean Luick <dean.luick@cornelisnetworks.com>

For memory allocated with dma_alloc_coherent(), use
dma_mmap_coherent() to mmap it into user space.

Signed-off-by: Dean Luick <dean.luick@cornelisnetworks.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
 drivers/infiniband/hw/hfi1/file_ops.c |   81 +++++++++++++++++++++++----------
 1 file changed, 57 insertions(+), 24 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c
index c9fc913db00c..9703e863ef06 100644
--- a/drivers/infiniband/hw/hfi1/file_ops.c
+++ b/drivers/infiniband/hw/hfi1/file_ops.c
@@ -306,6 +306,17 @@ static ssize_t hfi1_write_iter(struct kiocb *kiocb, struct iov_iter *from)
 	return reqs;
 }
 
+static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 vmf,
+			     u64 memaddr, void *memvirt, dma_addr_t memdma,
+			     ssize_t memlen, struct vm_area_struct *vma)
+{
+	hfi1_cdbg(PROC,
+		  "%u:%u type:%u io/vf/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx",
+		  ctxt, subctxt, type, mapio, vmf, !!memdma,
+		  memaddr ?: (u64)memvirt, memlen,
+		  vma->vm_end - vma->vm_start, vma->vm_flags);
+}
+
 static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 {
 	struct hfi1_filedata *fd = fp->private_data;
@@ -315,6 +326,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 	u64 token = vma->vm_pgoff << PAGE_SHIFT,
 		memaddr = 0;
 	void *memvirt = NULL;
+	dma_addr_t memdma = 0;
 	u8 subctxt, mapio = 0, vmf = 0, type;
 	ssize_t memlen = 0;
 	int ret = 0;
@@ -334,6 +346,11 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 		goto done;
 	}
 
+	/*
+	 * vm_pgoff is used as a buffer selector cookie.  Always mmap from
+	 * the beginning.
+	 */ 
+	vma->vm_pgoff = 0;
 	flags = vma->vm_flags;
 
 	switch (type) {
@@ -355,7 +372,8 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 		vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
 		mapio = 1;
 		break;
-	case PIO_CRED:
+	case PIO_CRED: {
+		u64 cr_page_offset;
 		if (flags & VM_WRITE) {
 			ret = -EPERM;
 			goto done;
@@ -365,10 +383,11 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 		 * second or third page allocated for credit returns (if number
 		 * of enabled contexts > 64 and 128 respectively).
 		 */
-		memvirt = dd->cr_base[uctxt->numa_id].va;
-		memaddr = virt_to_phys(memvirt) +
-			(((u64)uctxt->sc->hw_free -
-			  (u64)dd->cr_base[uctxt->numa_id].va) & PAGE_MASK);
+		cr_page_offset = ((u64)uctxt->sc->hw_free -
+			  	     (u64)dd->cr_base[uctxt->numa_id].va) &
+				   PAGE_MASK;
+		memvirt = dd->cr_base[uctxt->numa_id].va + cr_page_offset;
+		memdma = dd->cr_base[uctxt->numa_id].dma + cr_page_offset;
 		memlen = PAGE_SIZE;
 		flags &= ~VM_MAYWRITE;
 		flags |= VM_DONTCOPY | VM_DONTEXPAND;
@@ -378,14 +397,16 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 		 * memory been flagged as non-cached?
 		 */
 		/* vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); */
-		mapio = 1;
 		break;
+	}
 	case RCV_HDRQ:
 		memlen = rcvhdrq_size(uctxt);
 		memvirt = uctxt->rcvhdrq;
+		memdma = uctxt->rcvhdrq_dma;
 		break;
 	case RCV_EGRBUF: {
-		unsigned long addr;
+		unsigned long vm_start_save;
+		unsigned long vm_end_save;
 		int i;
 		/*
 		 * The RcvEgr buffer need to be handled differently
@@ -404,24 +425,34 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 			goto done;
 		}
 		vma->vm_flags &= ~VM_MAYWRITE;
-		addr = vma->vm_start;
+		/*
+		 * Mmap multiple separate allocations into a single vma.  From
+		 * here, dma_mmap_coherent() calls dma_direct_mmap(), which
+		 * requires the mmap to exactly fill the vma starting at
+		 * vma_start.  Adjust the vma start and end for each eager
+		 * buffer segment mapped.  Restore the originals when done.
+		 */
+		vm_start_save = vma->vm_start;
+		vm_end_save = vma->vm_end;
+		vma->vm_end = vma->vm_start;
 		for (i = 0 ; i < uctxt->egrbufs.numbufs; i++) {
 			memlen = uctxt->egrbufs.buffers[i].len;
 			memvirt = uctxt->egrbufs.buffers[i].addr;
-			ret = remap_pfn_range(
-				vma, addr,
-				/*
-				 * virt_to_pfn() does the same, but
-				 * it's not available on x86_64
-				 * when CONFIG_MMU is enabled.
-				 */
-				PFN_DOWN(__pa(memvirt)),
-				memlen,
-				vma->vm_page_prot);
-			if (ret < 0)
+			memdma = uctxt->egrbufs.buffers[i].dma;
+			vma->vm_end += memlen;
+			mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr,
+				  memvirt, memdma, memlen, vma);
+			ret = dma_mmap_coherent(&dd->pcidev->dev, vma,
+						memvirt, memdma, memlen);
+			if (ret < 0) {
+				vma->vm_start = vm_start_save;
+				vma->vm_end = vm_end_save;
 				goto done;
-			addr += memlen;
+			}
+			vma->vm_start += memlen;
 		}
+		vma->vm_start = vm_start_save;
+		vma->vm_end = vm_end_save;
 		ret = 0;
 		goto done;
 	}
@@ -481,6 +512,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 		}
 		memlen = PAGE_SIZE;
 		memvirt = (void *)hfi1_rcvhdrtail_kvaddr(uctxt);
+		memdma = uctxt->rcvhdrqtailaddr_dma;
 		flags &= ~VM_MAYWRITE;
 		break;
 	case SUBCTXT_UREGS:
@@ -529,14 +561,15 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 	}
 
 	vma->vm_flags = flags;
-	hfi1_cdbg(PROC,
-		  "%u:%u type:%u io/vf:%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx\n",
-		    ctxt, subctxt, type, mapio, vmf, memaddr, memlen,
-		    vma->vm_end - vma->vm_start, vma->vm_flags);
+	mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr, memvirt, memdma, 
+		  memlen, vma);
 	if (vmf) {
 		vma->vm_pgoff = PFN_DOWN(memaddr);
 		vma->vm_ops = &vm_ops;
 		ret = 0;
+	} else if (memdma) {
+		ret = dma_mmap_coherent(&dd->pcidev->dev, vma,
+					memvirt, memdma, memlen);
 	} else if (mapio) {
 		ret = io_remap_pfn_range(vma, vma->vm_start,
 					 PFN_DOWN(memaddr),

Re: [PATCH for-next 0/7] FIXME and other fixes

[Leon Romanovsky]

On Mon, 09 Jan 2023 14:03:58 -0500, Dennis Dalessandro wrote:
> Dean fixes the FIXME that was left by Jason in the code to use the interval
> notifier. There are also fixes for other things like splitting our counter
> allocation to better align with the way the core is moving.
> 

Applied, thanks!

[1/7] IB/hfi1: Remove redundant pageidx variable
      https://git.kernel.org/rdma/rdma/c/3c49eef3897822
[2/7] IB/hfi1: Assign npages earlier
      https://git.kernel.org/rdma/rdma/c/a479433a6b7a2b
[3/7] IB/hfi1: Consolidate the creation of user TIDs
      https://git.kernel.org/rdma/rdma/c/d8f4ab01c6d0d5
[4/7] IB/hfi1: Improve TID validity checking
      https://git.kernel.org/rdma/rdma/c/845127ed8717e0
[5/7] IB/hfi1: Split IB counter allocation
      https://git.kernel.org/rdma/rdma/c/ef90f0a1913e8b
[6/7] IBh/hfi1: Update RMT size calculation
      https://git.kernel.org/rdma/rdma/c/131268e2558f1f
[7/7] IB/hfi1: Use dma_mmap_coherent for matching buffers
      https://git.kernel.org/rdma/rdma/c/5d3fcb45a374a7

Best regards,
-- 
Leon Romanovsky <leon@kernel.org>

Re: [PATCH for-next 0/7] FIXME and other fixes

[Jason Gunthorpe]

On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
> Dean fixes the FIXME that was left by Jason in the code to use the interval
> notifier.

? Which patch did that?

Jason

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/10/23 9:58 AM, Jason Gunthorpe wrote:
> On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
>> Dean fixes the FIXME that was left by Jason in the code to use the interval
>> notifier.
> 
> ? Which patch did that?

My fault. The last patch in the previous series [1] was meant to go first here.
I got off by 1 when I was splitting the patches out for submit.

[1]
https://lore.kernel.org/linux-rdma/167328549178.1472310.9867497376936699488.stgit@awfm-02.cornelisnetworks.com/T/#u

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/10/23 4:03 PM, Dennis Dalessandro wrote:
> On 1/10/23 9:58 AM, Jason Gunthorpe wrote:
>> On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
>>> Dean fixes the FIXME that was left by Jason in the code to use the interval
>>> notifier.
>>
>> ? Which patch did that?
> 
> My fault. The last patch in the previous series [1] was meant to go first here.
> I got off by 1 when I was splitting the patches out for submit.
> 
> [1]
> https://lore.kernel.org/linux-rdma/167328549178.1472310.9867497376936699488.stgit@awfm-02.cornelisnetworks.com/T/#u

As a side effect of this, can we pull patch 2/7 from this series into the RC?

[PATCH for-next 2/7] IB/hfi1: Assign npages earlier

should go with:

[PATCH for-rc 6/6] IB/hfi1: Remove user expected buffer invalidate race

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Leon Romanovsky]

On Fri, Jan 13, 2023 at 12:21:50PM -0500, Dennis Dalessandro wrote:
> On 1/10/23 4:03 PM, Dennis Dalessandro wrote:
> > On 1/10/23 9:58 AM, Jason Gunthorpe wrote:
> >> On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
> >>> Dean fixes the FIXME that was left by Jason in the code to use the interval
> >>> notifier.
> >>
> >> ? Which patch did that?
> > 
> > My fault. The last patch in the previous series [1] was meant to go first here.
> > I got off by 1 when I was splitting the patches out for submit.
> > 
> > [1]
> > https://lore.kernel.org/linux-rdma/167328549178.1472310.9867497376936699488.stgit@awfm-02.cornelisnetworks.com/T/#u
> 
> As a side effect of this, can we pull patch 2/7 from this series into the RC?

No, everything is in for-rc/for-next now.

Thanks

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/15/23 6:46 AM, Leon Romanovsky wrote:
> On Fri, Jan 13, 2023 at 12:21:50PM -0500, Dennis Dalessandro wrote:
>> On 1/10/23 4:03 PM, Dennis Dalessandro wrote:
>>> On 1/10/23 9:58 AM, Jason Gunthorpe wrote:
>>>> On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
>>>>> Dean fixes the FIXME that was left by Jason in the code to use the interval
>>>>> notifier.
>>>>
>>>> ? Which patch did that?
>>>
>>> My fault. The last patch in the previous series [1] was meant to go first here.
>>> I got off by 1 when I was splitting the patches out for submit.
>>>
>>> [1]
>>> https://lore.kernel.org/linux-rdma/167328549178.1472310.9867497376936699488.stgit@awfm-02.cornelisnetworks.com/T/#u
>>
>> As a side effect of this, can we pull patch 2/7 from this series into the RC?
> 
> No, everything is in for-rc/for-next now.

Without that patch there will be a regression in 6.2. Is there a reason it can't
merge into -rc?

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Leon Romanovsky]

On Mon, Jan 16, 2023 at 12:36:51AM -0500, Dennis Dalessandro wrote:
> On 1/15/23 6:46 AM, Leon Romanovsky wrote:
> > On Fri, Jan 13, 2023 at 12:21:50PM -0500, Dennis Dalessandro wrote:
> >> On 1/10/23 4:03 PM, Dennis Dalessandro wrote:
> >>> On 1/10/23 9:58 AM, Jason Gunthorpe wrote:
> >>>> On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
> >>>>> Dean fixes the FIXME that was left by Jason in the code to use the interval
> >>>>> notifier.
> >>>>
> >>>> ? Which patch did that?
> >>>
> >>> My fault. The last patch in the previous series [1] was meant to go first here.
> >>> I got off by 1 when I was splitting the patches out for submit.
> >>>
> >>> [1]
> >>> https://lore.kernel.org/linux-rdma/167328549178.1472310.9867497376936699488.stgit@awfm-02.cornelisnetworks.com/T/#u
> >>
> >> As a side effect of this, can we pull patch 2/7 from this series into the RC?
> > 
> > No, everything is in for-rc/for-next now.
> 
> Without that patch there will be a regression in 6.2. 

I'm lost here. You are saying above that you wanted patch from -rc to be
in -next series. However, you wrote about regression in 6.2, which is -rc.

> Is there a reason it can't merge into -rc?

Here you are asking to bring -next patches to -rc.

So please help me, what do you want to do with these branches?
1. -rc
2. -next

Options are:
1. keep as is
2. revert
3. anything else?

What we won't do:
1. backmerge -next to -rc
2. merge -rc into -next without strong justification, as it is not
needed in general because such merge happens during merge window.

Thanks

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/16/23 2:16 AM, Leon Romanovsky wrote:
> On Mon, Jan 16, 2023 at 12:36:51AM -0500, Dennis Dalessandro wrote:
>> On 1/15/23 6:46 AM, Leon Romanovsky wrote:
>>> On Fri, Jan 13, 2023 at 12:21:50PM -0500, Dennis Dalessandro wrote:
>>>> On 1/10/23 4:03 PM, Dennis Dalessandro wrote:
>>>>> On 1/10/23 9:58 AM, Jason Gunthorpe wrote:
>>>>>> On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
>>>>>>> Dean fixes the FIXME that was left by Jason in the code to use the interval
>>>>>>> notifier.
>>>>>>
>>>>>> ? Which patch did that?
>>>>>
>>>>> My fault. The last patch in the previous series [1] was meant to go first here.
>>>>> I got off by 1 when I was splitting the patches out for submit.
>>>>>
>>>>> [1]
>>>>> https://lore.kernel.org/linux-rdma/167328549178.1472310.9867497376936699488.stgit@awfm-02.cornelisnetworks.com/T/#u
>>>>
>>>> As a side effect of this, can we pull patch 2/7 from this series into the RC?
>>>
>>> No, everything is in for-rc/for-next now.
>>
>> Without that patch there will be a regression in 6.2. 
> 
> I'm lost here. You are saying above that you wanted patch from -rc to be
> in -next series. However, you wrote about regression in 6.2, which is -rc.

Originally I did not mean to send:
[PATCH for-rc 6/6] IB/hfi1: Remove user expected buffer invalidate race
for -rc.

I didn't realize, it has a functional dependency on:
[PATCH for-next 2/7] IB/hfi1: Assign npages earlier

Ideally either they both go to -rc or they both go to -next.

>> Is there a reason it can't merge into -rc?
> 
> Here you are asking to bring -next patches to -rc.

One patch.

> So please help me, what do you want to do with these branches?
> 1. -rc
> 2. -next
> 
> Options are:
> 1. keep as is
> 2. revert

Let me do some build testing. If we revert the -rc patch and then reapply to
-next we may encounter conflicts and/or build issues and just make things worse.

> 3. anything else?Will get back to you if I come up with something else.

> What we won't do:
> 1. backmerge -next to -rc

So why is this not an option? Well ok so I don't mean we should merge. I guess
I'm more looking to cherry-pick.

> 2. merge -rc into -next without strong justification, as it is not
> needed in general because such merge happens during merge window.

Agree, not needed.

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Leon Romanovsky]

On Wed, Jan 18, 2023 at 08:04:52AM -0500, Dennis Dalessandro wrote:
> On 1/16/23 2:16 AM, Leon Romanovsky wrote:
> > On Mon, Jan 16, 2023 at 12:36:51AM -0500, Dennis Dalessandro wrote:
> >> On 1/15/23 6:46 AM, Leon Romanovsky wrote:
> >>> On Fri, Jan 13, 2023 at 12:21:50PM -0500, Dennis Dalessandro wrote:
> >>>> On 1/10/23 4:03 PM, Dennis Dalessandro wrote:
> >>>>> On 1/10/23 9:58 AM, Jason Gunthorpe wrote:
> >>>>>> On Mon, Jan 09, 2023 at 02:03:58PM -0500, Dennis Dalessandro wrote:
> >>>>>>> Dean fixes the FIXME that was left by Jason in the code to use the interval
> >>>>>>> notifier.
> >>>>>>
> >>>>>> ? Which patch did that?
> >>>>>
> >>>>> My fault. The last patch in the previous series [1] was meant to go first here.
> >>>>> I got off by 1 when I was splitting the patches out for submit.
> >>>>>
> >>>>> [1]
> >>>>> https://lore.kernel.org/linux-rdma/167328549178.1472310.9867497376936699488.stgit@awfm-02.cornelisnetworks.com/T/#u
> >>>>
> >>>> As a side effect of this, can we pull patch 2/7 from this series into the RC?
> >>>
> >>> No, everything is in for-rc/for-next now.
> >>
> >> Without that patch there will be a regression in 6.2. 
> > 
> > I'm lost here. You are saying above that you wanted patch from -rc to be
> > in -next series. However, you wrote about regression in 6.2, which is -rc.
> 
> Originally I did not mean to send:
> [PATCH for-rc 6/6] IB/hfi1: Remove user expected buffer invalidate race
> for -rc.
> 
> I didn't realize, it has a functional dependency on:
> [PATCH for-next 2/7] IB/hfi1: Assign npages earlier
> 
> Ideally either they both go to -rc or they both go to -next.
> 
> >> Is there a reason it can't merge into -rc?
> > 
> > Here you are asking to bring -next patches to -rc.
> 
> One patch.
> 
> > So please help me, what do you want to do with these branches?
> > 1. -rc
> > 2. -next
> > 
> > Options are:
> > 1. keep as is
> > 2. revert
> 
> Let me do some build testing. If we revert the -rc patch and then reapply to
> -next we may encounter conflicts and/or build issues and just make things worse.
> 
> > 3. anything else?Will get back to you if I come up with something else.
> 
> > What we won't do:
> > 1. backmerge -next to -rc
> 
> So why is this not an option? Well ok so I don't mean we should merge. I guess
> I'm more looking to cherry-pick.

Backmerge will cause to the situation where features are brought to -rc.
The cherry-pick will be:
1. Revert [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] from -next
2. Apply [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] to -rc

Thanks

Re: [PATCH for-next 0/7] FIXME and other fixes

[Jason Gunthorpe]

On Fri, Jan 20, 2023 at 07:09:43PM +0200, Leon Romanovsky wrote:

> Backmerge will cause to the situation where features are brought to -rc.
> The cherry-pick will be:
> 1. Revert [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] from -next
> 2. Apply [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] to -rc

You don't need to revert, we just need to merge a RC release to -next
and deal with the conflict, if any.

Jason

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/20/23 12:42 PM, Jason Gunthorpe wrote:
> On Fri, Jan 20, 2023 at 07:09:43PM +0200, Leon Romanovsky wrote:
> 
>> Backmerge will cause to the situation where features are brought to -rc.
>> The cherry-pick will be:
>> 1. Revert [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] from -next
>> 2. Apply [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] to -rc
> 
> You don't need to revert, we just need to merge a RC release to -next
> and deal with the conflict, if any.

Thanks this sounds like a good way to go.

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Leon Romanovsky]

On Fri, Jan 20, 2023 at 12:50:35PM -0500, Dennis Dalessandro wrote:
> On 1/20/23 12:42 PM, Jason Gunthorpe wrote:
> > On Fri, Jan 20, 2023 at 07:09:43PM +0200, Leon Romanovsky wrote:
> > 
> >> Backmerge will cause to the situation where features are brought to -rc.
> >> The cherry-pick will be:
> >> 1. Revert [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] from -next
> >> 2. Apply [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] to -rc
> > 
> > You don't need to revert, we just need to merge a RC release to -next
> > and deal with the conflict, if any.
> 
> Thanks this sounds like a good way to go.

You talked about broken -rc, but here wants to fix -next.
https://lore.kernel.org/all/bce1ab36-66e4-465c-e051-94e397d108ba@cornelisnetworks.com/
https://lore.kernel.org/all/Y8T5602bNhscGixb@unreal/

Thanks

> 
> -Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/22/23 4:14 AM, Leon Romanovsky wrote:
> On Fri, Jan 20, 2023 at 12:50:35PM -0500, Dennis Dalessandro wrote:
>> On 1/20/23 12:42 PM, Jason Gunthorpe wrote:
>>> On Fri, Jan 20, 2023 at 07:09:43PM +0200, Leon Romanovsky wrote:
>>>
>>>> Backmerge will cause to the situation where features are brought to -rc.
>>>> The cherry-pick will be:
>>>> 1. Revert [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] from -next
>>>> 2. Apply [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] to -rc
>>>
>>> You don't need to revert, we just need to merge a RC release to -next
>>> and deal with the conflict, if any.
>>
>> Thanks this sounds like a good way to go.
> 
> You talked about broken -rc, but here wants to fix -next.
> https://lore.kernel.org/all/bce1ab36-66e4-465c-e051-94e397d108ba@cornelisnetworks.com/
> https://lore.kernel.org/all/Y8T5602bNhscGixb@unreal/


> >> As a side effect of this, can we pull patch 2/7 from this series into the RC?
> >
> > No, everything is in for-rc/for-next now.
>
> Without that patch there will be a regression in 6.2.

Sorry it's not clear. Want to move or include patch to keep -rc from being
broken. Your #2 above. I'm not concerned about #1 b/c it will fix itself in time
after merging with 6.2-rc.

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/23/23 11:49 AM, Dennis Dalessandro wrote:
> On 1/22/23 4:14 AM, Leon Romanovsky wrote:
>> On Fri, Jan 20, 2023 at 12:50:35PM -0500, Dennis Dalessandro wrote:
>>> On 1/20/23 12:42 PM, Jason Gunthorpe wrote:
>>>> On Fri, Jan 20, 2023 at 07:09:43PM +0200, Leon Romanovsky wrote:
>>>>
>>>>> Backmerge will cause to the situation where features are brought to -rc.
>>>>> The cherry-pick will be:
>>>>> 1. Revert [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] from -next
>>>>> 2. Apply [PATCH for-next 2/7] IB/hfi1: Assign npages earlier] to -rc
>>>>
>>>> You don't need to revert, we just need to merge a RC release to -next
>>>> and deal with the conflict, if any.
>>>
>>> Thanks this sounds like a good way to go.
>>
>> You talked about broken -rc, but here wants to fix -next.
>> https://lore.kernel.org/all/bce1ab36-66e4-465c-e051-94e397d108ba@cornelisnetworks.com/
>> https://lore.kernel.org/all/Y8T5602bNhscGixb@unreal/
> 
> 
>>>> As a side effect of this, can we pull patch 2/7 from this series into the RC?
>>>
>>> No, everything is in for-rc/for-next now.
>>
>> Without that patch there will be a regression in 6.2.
> 
> Sorry it's not clear. Want to move or include patch to keep -rc from being
> broken. Your #2 above. I'm not concerned about #1 b/c it will fix itself in time
> after merging with 6.2-rc.

I didn't see this make it out yet. Can this still make it in for -rc? Based on
Jason's reply [1] sounds like it will just work itself out in for-next.

[1] https://lore.kernel.org/linux-rdma/Y8rSiD5s+ZFV666t@nvidia.com/

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Jason Gunthorpe]

On Mon, Jan 30, 2023 at 04:57:56PM -0500, Dennis Dalessandro wrote:

> I didn't see this make it out yet. Can this still make it in for -rc? Based on
> Jason's reply [1] sounds like it will just work itself out in for-next.
> 
> [1] https://lore.kernel.org/linux-rdma/Y8rSiD5s+ZFV666t@nvidia.com/

Well, it looks OK to me, you should do a test merge yourself and
confirm nothing got mangled

Jason

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 1/31/23 9:58 AM, Jason Gunthorpe wrote:
> On Mon, Jan 30, 2023 at 04:57:56PM -0500, Dennis Dalessandro wrote:
> 
>> I didn't see this make it out yet. Can this still make it in for -rc? Based on
>> Jason's reply [1] sounds like it will just work itself out in for-next.
>>
>> [1] https://lore.kernel.org/linux-rdma/Y8rSiD5s+ZFV666t@nvidia.com/
> 
> Well, it looks OK to me, you should do a test merge yourself and
> confirm nothing got mangled

I tested this last week by cherry picking:

a479433a6b7a ("IB/hfi1: Assign npages earlier")

onto for-rc, then merged for-rc into for-next. Looks ok. No conflicts.

-Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Leon Romanovsky]

On Tue, Jan 31, 2023 at 11:57:22AM -0500, Dennis Dalessandro wrote:
> On 1/31/23 9:58 AM, Jason Gunthorpe wrote:
> > On Mon, Jan 30, 2023 at 04:57:56PM -0500, Dennis Dalessandro wrote:
> > 
> >> I didn't see this make it out yet. Can this still make it in for -rc? Based on
> >> Jason's reply [1] sounds like it will just work itself out in for-next.
> >>
> >> [1] https://lore.kernel.org/linux-rdma/Y8rSiD5s+ZFV666t@nvidia.com/
> > 
> > Well, it looks OK to me, you should do a test merge yourself and
> > confirm nothing got mangled
> 
> I tested this last week by cherry picking:
> 
> a479433a6b7a ("IB/hfi1: Assign npages earlier")
> 
> onto for-rc, then merged for-rc into for-next. Looks ok. No conflicts.

Jason took this patch to his wip/jgg-for-rc branch.

Thanks

> 
> -Denny

Re: [PATCH for-next 0/7] FIXME and other fixes

[Dennis Dalessandro]

On 2/1/23 3:40 AM, Leon Romanovsky wrote:
> On Tue, Jan 31, 2023 at 11:57:22AM -0500, Dennis Dalessandro wrote:
>> On 1/31/23 9:58 AM, Jason Gunthorpe wrote:
>>> On Mon, Jan 30, 2023 at 04:57:56PM -0500, Dennis Dalessandro wrote:
>>>
>>>> I didn't see this make it out yet. Can this still make it in for -rc? Based on
>>>> Jason's reply [1] sounds like it will just work itself out in for-next.
>>>>
>>>> [1] https://lore.kernel.org/linux-rdma/Y8rSiD5s+ZFV666t@nvidia.com/
>>>
>>> Well, it looks OK to me, you should do a test merge yourself and
>>> confirm nothing got mangled
>>
>> I tested this last week by cherry picking:
>>
>> a479433a6b7a ("IB/hfi1: Assign npages earlier")
>>
>> onto for-rc, then merged for-rc into for-next. Looks ok. No conflicts.
> 
> Jason took this patch to his wip/jgg-for-rc branch.

Ok cool. Jason if you want a fixes lines to tack onto the commit message...

Fixes:  b3deec25847b ("IB/hfi1: Remove user expected buffer invalidate race")

-Denny