[PATCH 0/6] cxlflash: Miscellaneous patches

[PATCH 0/6] cxlflash: Miscellaneous patches

[Uma Krishnan]

This patch series contains miscellaneous fixes. The first patch fixes a
bug while the rest improve the code structure and prepare the code for
future enhancements.

This series is intended for 4.16 and is bisectable.

Matthew R. Ochs (3):
  cxlflash: Explicitly cache number of interrupts per context
  cxlflash: Remove embedded CXL work structures
  cxlflash: Staging to support future accelerators

Uma Krishnan (3):
  cxlflash: Reset command ioasc
  cxlflash: Update cxl-specific arguments to generic cookie
  cxlflash: Adapter context init can return error

 drivers/scsi/cxlflash/Makefile    |   2 +-
 drivers/scsi/cxlflash/backend.h   |  41 ++++++++++
 drivers/scsi/cxlflash/common.h    |   8 +-
 drivers/scsi/cxlflash/cxl_hw.c    | 168 ++++++++++++++++++++++++++++++++++++++
 drivers/scsi/cxlflash/main.c      | 100 +++++++++--------------
 drivers/scsi/cxlflash/superpipe.c |  64 +++++++--------
 drivers/scsi/cxlflash/superpipe.h |   4 +-
 7 files changed, 288 insertions(+), 99 deletions(-)
 create mode 100644 drivers/scsi/cxlflash/backend.h
 create mode 100644 drivers/scsi/cxlflash/cxl_hw.c

-- 
2.1.0

[PATCH 1/6] cxlflash: Reset command ioasc

[Uma Krishnan]

In the event of a command failure, cxlflash returns the failure to the
upper layers to process. After processing the error, when the command is
queued again, the private command structure will not be zeroed and the
ioasc could be stale. Per the SISLite specification, the AFU only sets the
ioasc in the presence of a failure. Thus, even though the original command
succeeds the second time, the command is considered a failure due to stale
ioasc. This cycle repeats indefinitely and can cause a hang or IO failure.

To fix the issue, clear the ioasc before queuing any command.

Fixes: 479ad8e9d48c ("scsi: cxlflash: Remove zeroing of private command
data")
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 38b3a9c..48d3663 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -620,6 +620,7 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	cmd->parent = afu;
 	cmd->hwq_index = hwq_index;
 
+	cmd->sa.ioasc = 0;
 	cmd->rcb.ctx_id = hwq->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
 	cmd->rcb.port_sel = CHAN2PORTMASK(scp->device->channel);
-- 
2.1.0

[PATCH 2/6] cxlflash: Update cxl-specific arguments to generic cookie

[Uma Krishnan]

Convert cxl-specific pointers to generic cookies to facilitate future
enhancements.

Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h    |  4 ++--
 drivers/scsi/cxlflash/main.c      | 38 +++++++++++++++++++-------------------
 drivers/scsi/cxlflash/superpipe.c | 15 +++++++--------
 drivers/scsi/cxlflash/superpipe.h |  2 +-
 4 files changed, 29 insertions(+), 30 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 6d95e8e..d2a180d 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -129,7 +129,7 @@ struct cxlflash_cfg {
 	int lr_port;
 	atomic_t scan_host_needed;
 
-	struct cxl_afu *cxl_afu;
+	void *afu_cookie;
 
 	atomic_t recovery_threads;
 	struct mutex ctx_recovery_mutex;
@@ -203,7 +203,7 @@ struct hwq {
 	 * fields after this point
 	 */
 	struct afu *afu;
-	struct cxl_context *ctx;
+	void *ctx_cookie;
 	struct cxl_ioctl_start_work work;
 	struct sisl_host_map __iomem *host_map;		/* MC host map */
 	struct sisl_ctrl_map __iomem *ctrl_map;		/* MC control map */
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 48d3663..3880d52 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -739,7 +739,7 @@ static void term_intr(struct cxlflash_cfg *cfg, enum undo_level level,
 
 	hwq = get_hwq(afu, index);
 
-	if (!hwq->ctx) {
+	if (!hwq->ctx_cookie) {
 		dev_err(dev, "%s: returning with NULL MC\n", __func__);
 		return;
 	}
@@ -748,13 +748,13 @@ static void term_intr(struct cxlflash_cfg *cfg, enum undo_level level,
 	case UNMAP_THREE:
 		/* SISL_MSI_ASYNC_ERROR is setup only for the primary HWQ */
 		if (index == PRIMARY_HWQ)
-			cxl_unmap_afu_irq(hwq->ctx, 3, hwq);
+			cxl_unmap_afu_irq(hwq->ctx_cookie, 3, hwq);
 	case UNMAP_TWO:
-		cxl_unmap_afu_irq(hwq->ctx, 2, hwq);
+		cxl_unmap_afu_irq(hwq->ctx_cookie, 2, hwq);
 	case UNMAP_ONE:
-		cxl_unmap_afu_irq(hwq->ctx, 1, hwq);
+		cxl_unmap_afu_irq(hwq->ctx_cookie, 1, hwq);
 	case FREE_IRQ:
-		cxl_free_afu_irqs(hwq->ctx);
+		cxl_free_afu_irqs(hwq->ctx_cookie);
 		/* fall through */
 	case UNDO_NOOP:
 		/* No action required */
@@ -783,15 +783,15 @@ static void term_mc(struct cxlflash_cfg *cfg, u32 index)
 
 	hwq = get_hwq(afu, index);
 
-	if (!hwq->ctx) {
+	if (!hwq->ctx_cookie) {
 		dev_err(dev, "%s: returning with NULL MC\n", __func__);
 		return;
 	}
 
-	WARN_ON(cxl_stop_context(hwq->ctx));
+	WARN_ON(cxl_stop_context(hwq->ctx_cookie));
 	if (index != PRIMARY_HWQ)
-		WARN_ON(cxl_release_context(hwq->ctx));
-	hwq->ctx = NULL;
+		WARN_ON(cxl_release_context(hwq->ctx_cookie));
+	hwq->ctx_cookie = NULL;
 
 	spin_lock_irqsave(&hwq->hsq_slock, lock_flags);
 	flush_pending_cmds(hwq);
@@ -1611,7 +1611,7 @@ static int start_context(struct cxlflash_cfg *cfg, u32 index)
 	struct hwq *hwq = get_hwq(cfg->afu, index);
 	int rc = 0;
 
-	rc = cxl_start_context(hwq->ctx,
+	rc = cxl_start_context(hwq->ctx_cookie,
 			       hwq->work.work_element_descriptor,
 			       NULL);
 
@@ -1748,7 +1748,7 @@ static void init_pcr(struct cxlflash_cfg *cfg)
 	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
 
-		hwq->ctx_hndl = (u16) cxl_process_element(hwq->ctx);
+		hwq->ctx_hndl = (u16) cxl_process_element(hwq->ctx_cookie);
 		hwq->host_map = &afu->afu_map->hosts[hwq->ctx_hndl].host;
 		hwq->ctrl_map = &afu->afu_map->ctrls[hwq->ctx_hndl].ctrl;
 
@@ -1926,7 +1926,7 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 				 struct hwq *hwq)
 {
 	struct device *dev = &cfg->dev->dev;
-	struct cxl_context *ctx = hwq->ctx;
+	void *ctx = hwq->ctx_cookie;
 	int rc = 0;
 	enum undo_level level = UNDO_NOOP;
 	bool is_primary_hwq = (hwq->index == PRIMARY_HWQ);
@@ -1980,7 +1980,7 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
  */
 static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 {
-	struct cxl_context *ctx;
+	void *ctx;
 	struct device *dev = &cfg->dev->dev;
 	struct hwq *hwq = get_hwq(cfg->afu, index);
 	int rc = 0;
@@ -1999,8 +1999,8 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 		goto err1;
 	}
 
-	WARN_ON(hwq->ctx);
-	hwq->ctx = ctx;
+	WARN_ON(hwq->ctx_cookie);
+	hwq->ctx_cookie = ctx;
 
 	/* Set it up as a master with the CXL */
 	cxl_set_master(ctx);
@@ -2040,7 +2040,7 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 	if (index != PRIMARY_HWQ)
 		cxl_release_context(ctx);
 err1:
-	hwq->ctx = NULL;
+	hwq->ctx_cookie = NULL;
 	goto out;
 }
 
@@ -2095,7 +2095,7 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	struct hwq *hwq;
 	int i;
 
-	cxl_perst_reloads_same_image(cfg->cxl_afu, true);
+	cxl_perst_reloads_same_image(cfg->afu_cookie, true);
 
 	afu->num_hwqs = afu->desired_hwqs;
 	for (i = 0; i < afu->num_hwqs; i++) {
@@ -2109,7 +2109,7 @@ static int init_afu(struct cxlflash_cfg *cfg)
 
 	/* Map the entire MMIO space of the AFU using the first context */
 	hwq = get_hwq(afu, PRIMARY_HWQ);
-	afu->afu_map = cxl_psa_map(hwq->ctx);
+	afu->afu_map = cxl_psa_map(hwq->ctx_cookie);
 	if (!afu->afu_map) {
 		dev_err(dev, "%s: cxl_psa_map failed\n", __func__);
 		rc = -ENOMEM;
@@ -3702,7 +3702,7 @@ static int cxlflash_probe(struct pci_dev *pdev,
 
 	pci_set_drvdata(pdev, cfg);
 
-	cfg->cxl_afu = cxl_pci_to_afu(pdev);
+	cfg->afu_cookie = cxl_pci_to_afu(pdev);
 
 	rc = init_pci(cfg);
 	if (rc) {
diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index 170fff5..18f6240 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -810,14 +810,13 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg)
  * init_context() - initializes a previously allocated context
  * @ctxi:	Previously allocated context
  * @cfg:	Internal structure associated with the host.
- * @ctx:	Previously obtained CXL context reference.
+ * @ctx:	Previously obtained context cookie.
  * @ctxid:	Previously obtained process element associated with CXL context.
  * @file:	Previously obtained file associated with CXL context.
  * @perms:	User-specified permissions.
  */
 static void init_context(struct ctx_info *ctxi, struct cxlflash_cfg *cfg,
-			 struct cxl_context *ctx, int ctxid, struct file *file,
-			 u32 perms)
+			 void *ctx, int ctxid, struct file *file, u32 perms)
 {
 	struct afu *afu = cfg->afu;
 
@@ -976,7 +975,7 @@ static int cxlflash_disk_detach(struct scsi_device *sdev,
  */
 static int cxlflash_cxl_release(struct inode *inode, struct file *file)
 {
-	struct cxl_context *ctx = cxl_fops_get_context(file);
+	void *ctx = cxl_fops_get_context(file);
 	struct cxlflash_cfg *cfg = container_of(file->f_op, struct cxlflash_cfg,
 						cxl_fops);
 	struct device *dev = &cfg->dev->dev;
@@ -1089,7 +1088,7 @@ static int cxlflash_mmap_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	struct file *file = vma->vm_file;
-	struct cxl_context *ctx = cxl_fops_get_context(file);
+	void *ctx = cxl_fops_get_context(file);
 	struct cxlflash_cfg *cfg = container_of(file->f_op, struct cxlflash_cfg,
 						cxl_fops);
 	struct device *dev = &cfg->dev->dev;
@@ -1162,7 +1161,7 @@ static const struct vm_operations_struct cxlflash_mmap_vmops = {
  */
 static int cxlflash_cxl_mmap(struct file *file, struct vm_area_struct *vma)
 {
-	struct cxl_context *ctx = cxl_fops_get_context(file);
+	void *ctx = cxl_fops_get_context(file);
 	struct cxlflash_cfg *cfg = container_of(file->f_op, struct cxlflash_cfg,
 						cxl_fops);
 	struct device *dev = &cfg->dev->dev;
@@ -1317,7 +1316,7 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 	u64 rctxid = 0UL;
 	struct file *file = NULL;
 
-	struct cxl_context *ctx = NULL;
+	void *ctx = NULL;
 
 	int fd = -1;
 
@@ -1529,7 +1528,7 @@ static int recover_context(struct cxlflash_cfg *cfg,
 	int fd = -1;
 	int ctxid = -1;
 	struct file *file;
-	struct cxl_context *ctx;
+	void *ctx;
 	struct afu *afu = cfg->afu;
 
 	ctx = cxl_dev_context_init(cfg->dev);
diff --git a/drivers/scsi/cxlflash/superpipe.h b/drivers/scsi/cxlflash/superpipe.h
index 0b59768..62097df 100644
--- a/drivers/scsi/cxlflash/superpipe.h
+++ b/drivers/scsi/cxlflash/superpipe.h
@@ -104,7 +104,7 @@ struct ctx_info {
 	bool err_recovery_active;
 	struct mutex mutex; /* Context protection */
 	struct kref kref;
-	struct cxl_context *ctx;
+	void *ctx;
 	struct cxlflash_cfg *cfg;
 	struct list_head luns;	/* LUNs attached to this context */
 	const struct vm_operations_struct *cxl_mmap_vmops;
-- 
2.1.0

[PATCH 3/6] cxlflash: Explicitly cache number of interrupts per context

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

The number of interrupts a user requests during a context attach is
presently stored within the CXL work ioctl structure that is nested
alongside the per context metadata. Keeping this data in a structure
that is tied to a particular hardware implementation (CXL) will only
complicate matters when supporting newer accelerator transports.

Instead of relying upon the number of interrupts being cached within
a CXL-specific structure, explicitly cache the value within the context
information structure.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/superpipe.c | 14 +++++++++-----
 drivers/scsi/cxlflash/superpipe.h |  1 +
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index 18f6240..ecfa553 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -814,15 +814,18 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg)
  * @ctxid:	Previously obtained process element associated with CXL context.
  * @file:	Previously obtained file associated with CXL context.
  * @perms:	User-specified permissions.
+ * @irqs:	User-specified number of interrupts.
  */
 static void init_context(struct ctx_info *ctxi, struct cxlflash_cfg *cfg,
-			 void *ctx, int ctxid, struct file *file, u32 perms)
+			 void *ctx, int ctxid, struct file *file, u32 perms,
+			 u64 irqs)
 {
 	struct afu *afu = cfg->afu;
 
 	ctxi->rht_perms = perms;
 	ctxi->ctrl_map = &afu->afu_map->ctrls[ctxid].ctrl;
 	ctxi->ctxid = ENCODE_CTXID(ctxi, ctxid);
+	ctxi->irqs = irqs;
 	ctxi->pid = task_tgid_nr(current); /* tgid = pid */
 	ctxi->ctx = ctx;
 	ctxi->cfg = cfg;
@@ -1312,6 +1315,7 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 	int rc = 0;
 	u32 perms;
 	int ctxid = -1;
+	u64 irqs = attach->num_interrupts;
 	u64 flags = 0UL;
 	u64 rctxid = 0UL;
 	struct file *file = NULL;
@@ -1320,9 +1324,9 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 
 	int fd = -1;
 
-	if (attach->num_interrupts > 4) {
+	if (irqs > 4) {
 		dev_dbg(dev, "%s: Cannot support this many interrupts %llu\n",
-			__func__, attach->num_interrupts);
+			__func__, irqs);
 		rc = -EINVAL;
 		goto out;
 	}
@@ -1402,7 +1406,7 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 	}
 
 	work = &ctxi->work;
-	work->num_interrupts = attach->num_interrupts;
+	work->num_interrupts = irqs;
 	work->flags = CXL_START_WORK_NUM_IRQS;
 
 	rc = cxl_start_work(ctx, work);
@@ -1430,7 +1434,7 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 	perms = SISL_RHT_PERM(attach->hdr.flags + 1);
 
 	/* Context mutex is locked upon return */
-	init_context(ctxi, cfg, ctx, ctxid, file, perms);
+	init_context(ctxi, cfg, ctx, ctxid, file, perms, irqs);
 
 	rc = afu_attach(cfg, ctxi);
 	if (unlikely(rc)) {
diff --git a/drivers/scsi/cxlflash/superpipe.h b/drivers/scsi/cxlflash/superpipe.h
index 62097df..b761293 100644
--- a/drivers/scsi/cxlflash/superpipe.h
+++ b/drivers/scsi/cxlflash/superpipe.h
@@ -98,6 +98,7 @@ struct ctx_info {
 
 	struct cxl_ioctl_start_work work;
 	u64 ctxid;
+	u64 irqs; /* Number of interrupts requested for context */
 	pid_t pid;
 	bool initialized;
 	bool unavail;
-- 
2.1.0

[PATCH 4/6] cxlflash: Remove embedded CXL work structures

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

The CXL-specific work structure used to request the number of interrupts
currently resides as a nested member of both the context information and
hardware queue structures. It is used to cache values (specifically the
number of interrupts) required by the CXL layer when starting a context.

To facilitate staging that will ultimately allow the cxlflash core to
become agnostic of the underlying accelerator transport, remove these
embedded work structures.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h    |  1 -
 drivers/scsi/cxlflash/main.c      |  4 +---
 drivers/scsi/cxlflash/superpipe.c | 15 +++++++++------
 drivers/scsi/cxlflash/superpipe.h |  1 -
 4 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index d2a180d..48df89f 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -204,7 +204,6 @@ struct hwq {
 	 */
 	struct afu *afu;
 	void *ctx_cookie;
-	struct cxl_ioctl_start_work work;
 	struct sisl_host_map __iomem *host_map;		/* MC host map */
 	struct sisl_ctrl_map __iomem *ctrl_map;		/* MC control map */
 	ctx_hndl_t ctx_hndl;	/* master's context handle */
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 3880d52..b6cadeb 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1611,9 +1611,7 @@ static int start_context(struct cxlflash_cfg *cfg, u32 index)
 	struct hwq *hwq = get_hwq(cfg->afu, index);
 	int rc = 0;
 
-	rc = cxl_start_context(hwq->ctx_cookie,
-			       hwq->work.work_element_descriptor,
-			       NULL);
+	rc = cxl_start_context(hwq->ctx_cookie, 0, NULL);
 
 	dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc);
 	return rc;
diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index ecfa553..51f67dc 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -1309,7 +1309,7 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 	struct afu *afu = cfg->afu;
 	struct llun_info *lli = sdev->hostdata;
 	struct glun_info *gli = lli->parent;
-	struct cxl_ioctl_start_work *work;
+	struct cxl_ioctl_start_work work = { 0 };
 	struct ctx_info *ctxi = NULL;
 	struct lun_access *lun_access = NULL;
 	int rc = 0;
@@ -1405,11 +1405,10 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 		goto err;
 	}
 
-	work = &ctxi->work;
-	work->num_interrupts = irqs;
-	work->flags = CXL_START_WORK_NUM_IRQS;
+	work.num_interrupts = irqs;
+	work.flags = CXL_START_WORK_NUM_IRQS;
 
-	rc = cxl_start_work(ctx, work);
+	rc = cxl_start_work(ctx, &work);
 	if (unlikely(rc)) {
 		dev_dbg(dev, "%s: Could not start context rc=%d\n",
 			__func__, rc);
@@ -1534,6 +1533,7 @@ static int recover_context(struct cxlflash_cfg *cfg,
 	struct file *file;
 	void *ctx;
 	struct afu *afu = cfg->afu;
+	struct cxl_ioctl_start_work work = { 0 };
 
 	ctx = cxl_dev_context_init(cfg->dev);
 	if (IS_ERR_OR_NULL(ctx)) {
@@ -1543,7 +1543,10 @@ static int recover_context(struct cxlflash_cfg *cfg,
 		goto out;
 	}
 
-	rc = cxl_start_work(ctx, &ctxi->work);
+	work.num_interrupts = ctxi->irqs;
+	work.flags = CXL_START_WORK_NUM_IRQS;
+
+	rc = cxl_start_work(ctx, &work);
 	if (unlikely(rc)) {
 		dev_dbg(dev, "%s: Could not start context rc=%d\n",
 			__func__, rc);
diff --git a/drivers/scsi/cxlflash/superpipe.h b/drivers/scsi/cxlflash/superpipe.h
index b761293..35c3cbf 100644
--- a/drivers/scsi/cxlflash/superpipe.h
+++ b/drivers/scsi/cxlflash/superpipe.h
@@ -96,7 +96,6 @@ struct ctx_info {
 	struct llun_info **rht_lun;       /* Mapping of RHT entries to LUNs */
 	u8 *rht_needs_ws;	/* User-desired write-same function per RHTE */
 
-	struct cxl_ioctl_start_work work;
 	u64 ctxid;
 	u64 irqs; /* Number of interrupts requested for context */
 	pid_t pid;
-- 
2.1.0

[PATCH 5/6] cxlflash: Adapter context init can return error

[Uma Krishnan]

Adapter context creation can return either NULL or an error pointer.
Updating the check condition to reflect this.

Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index b6cadeb..32014e8 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1992,7 +1992,7 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 		ctx = cxl_get_context(cfg->dev);
 	else
 		ctx = cxl_dev_context_init(cfg->dev);
-	if (unlikely(!ctx)) {
+	if (IS_ERR_OR_NULL(ctx)) {
 		rc = -ENOMEM;
 		goto err1;
 	}
-- 
2.1.0

[PATCH 6/6] cxlflash: Staging to support future accelerators

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

As staging to support future accelerator transports, add a shim layer
such that the underlying services the cxlflash driver requires can be
conditional upon the accelerator infrastructure.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/Makefile    |   2 +-
 drivers/scsi/cxlflash/backend.h   |  41 ++++++++++
 drivers/scsi/cxlflash/common.h    |   3 +
 drivers/scsi/cxlflash/cxl_hw.c    | 168 ++++++++++++++++++++++++++++++++++++++
 drivers/scsi/cxlflash/main.c      |  79 +++++++-----------
 drivers/scsi/cxlflash/superpipe.c |  48 +++++------
 6 files changed, 263 insertions(+), 78 deletions(-)
 create mode 100644 drivers/scsi/cxlflash/backend.h
 create mode 100644 drivers/scsi/cxlflash/cxl_hw.c

diff --git a/drivers/scsi/cxlflash/Makefile b/drivers/scsi/cxlflash/Makefile
index 9e39866..7ec3f6b 100644
--- a/drivers/scsi/cxlflash/Makefile
+++ b/drivers/scsi/cxlflash/Makefile
@@ -1,2 +1,2 @@
 obj-$(CONFIG_CXLFLASH) += cxlflash.o
-cxlflash-y += main.o superpipe.o lunmgt.o vlun.o
+cxlflash-y += main.o superpipe.o lunmgt.o vlun.o cxl_hw.o
diff --git a/drivers/scsi/cxlflash/backend.h b/drivers/scsi/cxlflash/backend.h
new file mode 100644
index 0000000..339e42b
--- /dev/null
+++ b/drivers/scsi/cxlflash/backend.h
@@ -0,0 +1,41 @@
+/*
+ * CXL Flash Device Driver
+ *
+ * Written by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>, IBM Corporation
+ *             Uma Krishnan <ukrishn@linux.vnet.ibm.com>, IBM Corporation
+ *
+ * Copyright (C) 2018 IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+extern const struct cxlflash_backend_ops cxlflash_cxl_ops;
+
+struct cxlflash_backend_ops {
+	struct module *module;
+	void __iomem * (*psa_map)(void *);
+	void (*psa_unmap)(void __iomem *);
+	int (*process_element)(void *);
+	int (*map_afu_irq)(void *, int, irq_handler_t, void *, char *);
+	void (*unmap_afu_irq)(void *, int, void *);
+	int (*start_context)(void *);
+	int (*stop_context)(void *);
+	int (*afu_reset)(void *);
+	void (*set_master)(void *);
+	void * (*get_context)(struct pci_dev *, void *);
+	void * (*dev_context_init)(struct pci_dev *, void *);
+	int (*release_context)(void *);
+	void (*perst_reloads_same_image)(void *, bool);
+	ssize_t (*read_adapter_vpd)(struct pci_dev *, void *, size_t);
+	int (*allocate_afu_irqs)(void *, int);
+	void (*free_afu_irqs)(void *);
+	void * (*create_afu)(struct pci_dev *);
+	struct file * (*get_fd)(void *, struct file_operations *, int *);
+	void * (*fops_get_context)(struct file *);
+	int (*start_work)(void *, u64);
+	int (*fd_mmap)(struct file *, struct vm_area_struct *);
+	int (*fd_release)(struct inode *, struct file *);
+};
diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 48df89f..102fd26 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -25,6 +25,8 @@
 #include <scsi/scsi_cmnd.h>
 #include <scsi/scsi_device.h>
 
+#include "backend.h"
+
 extern const struct file_operations cxlflash_cxl_fops;
 
 #define MAX_CONTEXT	CXLFLASH_MAX_CONTEXT	/* num contexts per afu */
@@ -114,6 +116,7 @@ enum cxlflash_hwq_mode {
 struct cxlflash_cfg {
 	struct afu *afu;
 
+	const struct cxlflash_backend_ops *ops;
 	struct pci_dev *dev;
 	struct pci_device_id *dev_id;
 	struct Scsi_Host *host;
diff --git a/drivers/scsi/cxlflash/cxl_hw.c b/drivers/scsi/cxlflash/cxl_hw.c
new file mode 100644
index 0000000..db1cada
--- /dev/null
+++ b/drivers/scsi/cxlflash/cxl_hw.c
@@ -0,0 +1,168 @@
+/*
+ * CXL Flash Device Driver
+ *
+ * Written by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>, IBM Corporation
+ *             Uma Krishnan <ukrishn@linux.vnet.ibm.com>, IBM Corporation
+ *
+ * Copyright (C) 2018 IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <misc/cxl.h>
+
+#include "backend.h"
+
+/*
+ * The following routines map the cxlflash backend operations to existing CXL
+ * kernel API function and are largely simple shims that provide an abstraction
+ * for converting generic context and AFU cookies into cxl_context or cxl_afu
+ * pointers.
+ */
+
+static void __iomem *cxlflash_psa_map(void *ctx_cookie)
+{
+	return cxl_psa_map(ctx_cookie);
+}
+
+static void cxlflash_psa_unmap(void __iomem *addr)
+{
+	cxl_psa_unmap(addr);
+}
+
+static int cxlflash_process_element(void *ctx_cookie)
+{
+	return cxl_process_element(ctx_cookie);
+}
+
+static int cxlflash_map_afu_irq(void *ctx_cookie, int num,
+				irq_handler_t handler, void *cookie, char *name)
+{
+	return cxl_map_afu_irq(ctx_cookie, num, handler, cookie, name);
+}
+
+static void cxlflash_unmap_afu_irq(void *ctx_cookie, int num, void *cookie)
+{
+	cxl_unmap_afu_irq(ctx_cookie, num, cookie);
+}
+
+static int cxlflash_start_context(void *ctx_cookie)
+{
+	return cxl_start_context(ctx_cookie, 0, NULL);
+}
+
+static int cxlflash_stop_context(void *ctx_cookie)
+{
+	return cxl_stop_context(ctx_cookie);
+}
+
+static int cxlflash_afu_reset(void *ctx_cookie)
+{
+	return cxl_afu_reset(ctx_cookie);
+}
+
+static void cxlflash_set_master(void *ctx_cookie)
+{
+	cxl_set_master(ctx_cookie);
+}
+
+static void *cxlflash_get_context(struct pci_dev *dev, void *afu_cookie)
+{
+	return cxl_get_context(dev);
+}
+
+static void *cxlflash_dev_context_init(struct pci_dev *dev, void *afu_cookie)
+{
+	return cxl_dev_context_init(dev);
+}
+
+static int cxlflash_release_context(void *ctx_cookie)
+{
+	return cxl_release_context(ctx_cookie);
+}
+
+static void cxlflash_perst_reloads_same_image(void *afu_cookie, bool image)
+{
+	cxl_perst_reloads_same_image(afu_cookie, image);
+}
+
+static ssize_t cxlflash_read_adapter_vpd(struct pci_dev *dev,
+					 void *buf, size_t count)
+{
+	return cxl_read_adapter_vpd(dev, buf, count);
+}
+
+static int cxlflash_allocate_afu_irqs(void *ctx_cookie, int num)
+{
+	return cxl_allocate_afu_irqs(ctx_cookie, num);
+}
+
+static void cxlflash_free_afu_irqs(void *ctx_cookie)
+{
+	cxl_free_afu_irqs(ctx_cookie);
+}
+
+static void *cxlflash_create_afu(struct pci_dev *dev)
+{
+	return cxl_pci_to_afu(dev);
+}
+
+static struct file *cxlflash_get_fd(void *ctx_cookie,
+				    struct file_operations *fops, int *fd)
+{
+	return cxl_get_fd(ctx_cookie, fops, fd);
+}
+
+static void *cxlflash_fops_get_context(struct file *file)
+{
+	return cxl_fops_get_context(file);
+}
+
+static int cxlflash_start_work(void *ctx_cookie, u64 irqs)
+{
+	struct cxl_ioctl_start_work work = { 0 };
+
+	work.num_interrupts = irqs;
+	work.flags = CXL_START_WORK_NUM_IRQS;
+
+	return cxl_start_work(ctx_cookie, &work);
+}
+
+static int cxlflash_fd_mmap(struct file *file, struct vm_area_struct *vm)
+{
+	return cxl_fd_mmap(file, vm);
+}
+
+static int cxlflash_fd_release(struct inode *inode, struct file *file)
+{
+	return cxl_fd_release(inode, file);
+}
+
+const struct cxlflash_backend_ops cxlflash_cxl_ops = {
+	.module			= THIS_MODULE,
+	.psa_map		= cxlflash_psa_map,
+	.psa_unmap		= cxlflash_psa_unmap,
+	.process_element	= cxlflash_process_element,
+	.map_afu_irq		= cxlflash_map_afu_irq,
+	.unmap_afu_irq		= cxlflash_unmap_afu_irq,
+	.start_context		= cxlflash_start_context,
+	.stop_context		= cxlflash_stop_context,
+	.afu_reset		= cxlflash_afu_reset,
+	.set_master		= cxlflash_set_master,
+	.get_context		= cxlflash_get_context,
+	.dev_context_init	= cxlflash_dev_context_init,
+	.release_context	= cxlflash_release_context,
+	.perst_reloads_same_image = cxlflash_perst_reloads_same_image,
+	.read_adapter_vpd	= cxlflash_read_adapter_vpd,
+	.allocate_afu_irqs	= cxlflash_allocate_afu_irqs,
+	.free_afu_irqs		= cxlflash_free_afu_irqs,
+	.create_afu		= cxlflash_create_afu,
+	.get_fd			= cxlflash_get_fd,
+	.fops_get_context	= cxlflash_fops_get_context,
+	.start_work		= cxlflash_start_work,
+	.fd_mmap		= cxlflash_fd_mmap,
+	.fd_release		= cxlflash_fd_release,
+};
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 32014e8..d8fe7ab8 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -711,7 +711,7 @@ static void stop_afu(struct cxlflash_cfg *cfg)
 		}
 
 		if (likely(afu->afu_map)) {
-			cxl_psa_unmap((void __iomem *)afu->afu_map);
+			cfg->ops->psa_unmap(afu->afu_map);
 			afu->afu_map = NULL;
 		}
 	}
@@ -748,13 +748,13 @@ static void term_intr(struct cxlflash_cfg *cfg, enum undo_level level,
 	case UNMAP_THREE:
 		/* SISL_MSI_ASYNC_ERROR is setup only for the primary HWQ */
 		if (index == PRIMARY_HWQ)
-			cxl_unmap_afu_irq(hwq->ctx_cookie, 3, hwq);
+			cfg->ops->unmap_afu_irq(hwq->ctx_cookie, 3, hwq);
 	case UNMAP_TWO:
-		cxl_unmap_afu_irq(hwq->ctx_cookie, 2, hwq);
+		cfg->ops->unmap_afu_irq(hwq->ctx_cookie, 2, hwq);
 	case UNMAP_ONE:
-		cxl_unmap_afu_irq(hwq->ctx_cookie, 1, hwq);
+		cfg->ops->unmap_afu_irq(hwq->ctx_cookie, 1, hwq);
 	case FREE_IRQ:
-		cxl_free_afu_irqs(hwq->ctx_cookie);
+		cfg->ops->free_afu_irqs(hwq->ctx_cookie);
 		/* fall through */
 	case UNDO_NOOP:
 		/* No action required */
@@ -788,9 +788,9 @@ static void term_mc(struct cxlflash_cfg *cfg, u32 index)
 		return;
 	}
 
-	WARN_ON(cxl_stop_context(hwq->ctx_cookie));
+	WARN_ON(cfg->ops->stop_context(hwq->ctx_cookie));
 	if (index != PRIMARY_HWQ)
-		WARN_ON(cxl_release_context(hwq->ctx_cookie));
+		WARN_ON(cfg->ops->release_context(hwq->ctx_cookie));
 	hwq->ctx_cookie = NULL;
 
 	spin_lock_irqsave(&hwq->hsq_slock, lock_flags);
@@ -1599,25 +1599,6 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 }
 
 /**
- * start_context() - starts the master context
- * @cfg:	Internal structure associated with the host.
- * @index:	Index of the hardware queue.
- *
- * Return: A success or failure value from CXL services.
- */
-static int start_context(struct cxlflash_cfg *cfg, u32 index)
-{
-	struct device *dev = &cfg->dev->dev;
-	struct hwq *hwq = get_hwq(cfg->afu, index);
-	int rc = 0;
-
-	rc = cxl_start_context(hwq->ctx_cookie, 0, NULL);
-
-	dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc);
-	return rc;
-}
-
-/**
  * read_vpd() - obtains the WWPNs from VPD
  * @cfg:	Internal structure associated with the host.
  * @wwpn:	Array of size MAX_FC_PORTS to pass back WWPNs
@@ -1639,7 +1620,7 @@ static int read_vpd(struct cxlflash_cfg *cfg, u64 wwpn[])
 	const char *wwpn_vpd_tags[MAX_FC_PORTS] = { "V5", "V6", "V7", "V8" };
 
 	/* Get the VPD data from the device */
-	vpd_size = cxl_read_adapter_vpd(pdev, vpd_data, sizeof(vpd_data));
+	vpd_size = cfg->ops->read_adapter_vpd(pdev, vpd_data, sizeof(vpd_data));
 	if (unlikely(vpd_size <= 0)) {
 		dev_err(dev, "%s: Unable to read VPD (size = %ld)\n",
 			__func__, vpd_size);
@@ -1731,6 +1712,7 @@ static void init_pcr(struct cxlflash_cfg *cfg)
 	struct afu *afu = cfg->afu;
 	struct sisl_ctrl_map __iomem *ctrl_map;
 	struct hwq *hwq;
+	void *cookie;
 	int i;
 
 	for (i = 0; i < MAX_CONTEXT; i++) {
@@ -1745,8 +1727,9 @@ static void init_pcr(struct cxlflash_cfg *cfg)
 	/* Copy frequently used fields into hwq */
 	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
+		cookie = hwq->ctx_cookie;
 
-		hwq->ctx_hndl = (u16) cxl_process_element(hwq->ctx_cookie);
+		hwq->ctx_hndl = (u16) cfg->ops->process_element(cookie);
 		hwq->host_map = &afu->afu_map->hosts[hwq->ctx_hndl].host;
 		hwq->ctrl_map = &afu->afu_map->ctrls[hwq->ctx_hndl].ctrl;
 
@@ -1930,7 +1913,7 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 	bool is_primary_hwq = (hwq->index == PRIMARY_HWQ);
 	int num_irqs = is_primary_hwq ? 3 : 2;
 
-	rc = cxl_allocate_afu_irqs(ctx, num_irqs);
+	rc = cfg->ops->allocate_afu_irqs(ctx, num_irqs);
 	if (unlikely(rc)) {
 		dev_err(dev, "%s: allocate_afu_irqs failed rc=%d\n",
 			__func__, rc);
@@ -1938,16 +1921,16 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 		goto out;
 	}
 
-	rc = cxl_map_afu_irq(ctx, 1, cxlflash_sync_err_irq, hwq,
-			     "SISL_MSI_SYNC_ERROR");
+	rc = cfg->ops->map_afu_irq(ctx, 1, cxlflash_sync_err_irq, hwq,
+				   "SISL_MSI_SYNC_ERROR");
 	if (unlikely(rc <= 0)) {
 		dev_err(dev, "%s: SISL_MSI_SYNC_ERROR map failed\n", __func__);
 		level = FREE_IRQ;
 		goto out;
 	}
 
-	rc = cxl_map_afu_irq(ctx, 2, cxlflash_rrq_irq, hwq,
-			     "SISL_MSI_RRQ_UPDATED");
+	rc = cfg->ops->map_afu_irq(ctx, 2, cxlflash_rrq_irq, hwq,
+				   "SISL_MSI_RRQ_UPDATED");
 	if (unlikely(rc <= 0)) {
 		dev_err(dev, "%s: SISL_MSI_RRQ_UPDATED map failed\n", __func__);
 		level = UNMAP_ONE;
@@ -1958,8 +1941,8 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 	if (!is_primary_hwq)
 		goto out;
 
-	rc = cxl_map_afu_irq(ctx, 3, cxlflash_async_err_irq, hwq,
-			     "SISL_MSI_ASYNC_ERROR");
+	rc = cfg->ops->map_afu_irq(ctx, 3, cxlflash_async_err_irq, hwq,
+				   "SISL_MSI_ASYNC_ERROR");
 	if (unlikely(rc <= 0)) {
 		dev_err(dev, "%s: SISL_MSI_ASYNC_ERROR map failed\n", __func__);
 		level = UNMAP_TWO;
@@ -1989,9 +1972,9 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 	INIT_LIST_HEAD(&hwq->pending_cmds);
 
 	if (index == PRIMARY_HWQ)
-		ctx = cxl_get_context(cfg->dev);
+		ctx = cfg->ops->get_context(cfg->dev, cfg->afu_cookie);
 	else
-		ctx = cxl_dev_context_init(cfg->dev);
+		ctx = cfg->ops->dev_context_init(cfg->dev, cfg->afu_cookie);
 	if (IS_ERR_OR_NULL(ctx)) {
 		rc = -ENOMEM;
 		goto err1;
@@ -2001,11 +1984,11 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 	hwq->ctx_cookie = ctx;
 
 	/* Set it up as a master with the CXL */
-	cxl_set_master(ctx);
+	cfg->ops->set_master(ctx);
 
 	/* Reset AFU when initializing primary context */
 	if (index == PRIMARY_HWQ) {
-		rc = cxl_afu_reset(ctx);
+		rc = cfg->ops->afu_reset(ctx);
 		if (unlikely(rc)) {
 			dev_err(dev, "%s: AFU reset failed rc=%d\n",
 				      __func__, rc);
@@ -2019,11 +2002,8 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 		goto err2;
 	}
 
-	/* This performs the equivalent of the CXL_IOCTL_START_WORK.
-	 * The CXL_IOCTL_GET_PROCESS_ELEMENT is implicit in the process
-	 * element (pe) that is embedded in the context (ctx)
-	 */
-	rc = start_context(cfg, index);
+	/* Finally, activate the context by starting it */
+	rc = cfg->ops->start_context(hwq->ctx_cookie);
 	if (unlikely(rc)) {
 		dev_err(dev, "%s: start context failed rc=%d\n", __func__, rc);
 		level = UNMAP_THREE;
@@ -2036,7 +2016,7 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 err2:
 	term_intr(cfg, level, index);
 	if (index != PRIMARY_HWQ)
-		cxl_release_context(ctx);
+		cfg->ops->release_context(ctx);
 err1:
 	hwq->ctx_cookie = NULL;
 	goto out;
@@ -2093,7 +2073,7 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	struct hwq *hwq;
 	int i;
 
-	cxl_perst_reloads_same_image(cfg->afu_cookie, true);
+	cfg->ops->perst_reloads_same_image(cfg->afu_cookie, true);
 
 	afu->num_hwqs = afu->desired_hwqs;
 	for (i = 0; i < afu->num_hwqs; i++) {
@@ -2107,9 +2087,9 @@ static int init_afu(struct cxlflash_cfg *cfg)
 
 	/* Map the entire MMIO space of the AFU using the first context */
 	hwq = get_hwq(afu, PRIMARY_HWQ);
-	afu->afu_map = cxl_psa_map(hwq->ctx_cookie);
+	afu->afu_map = cfg->ops->psa_map(hwq->ctx_cookie);
 	if (!afu->afu_map) {
-		dev_err(dev, "%s: cxl_psa_map failed\n", __func__);
+		dev_err(dev, "%s: psa_map failed\n", __func__);
 		rc = -ENOMEM;
 		goto err1;
 	}
@@ -3669,6 +3649,7 @@ static int cxlflash_probe(struct pci_dev *pdev,
 
 	cfg->init_state = INIT_STATE_NONE;
 	cfg->dev = pdev;
+	cfg->ops = &cxlflash_cxl_ops;
 	cfg->cxl_fops = cxlflash_cxl_fops;
 
 	/*
@@ -3700,7 +3681,7 @@ static int cxlflash_probe(struct pci_dev *pdev,
 
 	pci_set_drvdata(pdev, cfg);
 
-	cfg->afu_cookie = cxl_pci_to_afu(pdev);
+	cfg->afu_cookie = cfg->ops->create_afu(pdev);
 
 	rc = init_pci(cfg);
 	if (rc) {
diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index 51f67dc..2fe79df 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -978,9 +978,9 @@ static int cxlflash_disk_detach(struct scsi_device *sdev,
  */
 static int cxlflash_cxl_release(struct inode *inode, struct file *file)
 {
-	void *ctx = cxl_fops_get_context(file);
 	struct cxlflash_cfg *cfg = container_of(file->f_op, struct cxlflash_cfg,
 						cxl_fops);
+	void *ctx = cfg->ops->fops_get_context(file);
 	struct device *dev = &cfg->dev->dev;
 	struct ctx_info *ctxi = NULL;
 	struct dk_cxlflash_detach detach = { { 0 }, 0 };
@@ -988,7 +988,7 @@ static int cxlflash_cxl_release(struct inode *inode, struct file *file)
 	enum ctx_ctrl ctrl = CTX_CTRL_ERR_FALLBACK | CTX_CTRL_FILE;
 	int ctxid;
 
-	ctxid = cxl_process_element(ctx);
+	ctxid = cfg->ops->process_element(ctx);
 	if (unlikely(ctxid < 0)) {
 		dev_err(dev, "%s: Context %p was closed ctxid=%d\n",
 			__func__, ctx, ctxid);
@@ -1016,7 +1016,7 @@ static int cxlflash_cxl_release(struct inode *inode, struct file *file)
 	list_for_each_entry_safe(lun_access, t, &ctxi->luns, list)
 		_cxlflash_disk_detach(lun_access->sdev, ctxi, &detach);
 out_release:
-	cxl_fd_release(inode, file);
+	cfg->ops->fd_release(inode, file);
 out:
 	dev_dbg(dev, "%s: returning\n", __func__);
 	return 0;
@@ -1091,9 +1091,9 @@ static int cxlflash_mmap_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	struct file *file = vma->vm_file;
-	void *ctx = cxl_fops_get_context(file);
 	struct cxlflash_cfg *cfg = container_of(file->f_op, struct cxlflash_cfg,
 						cxl_fops);
+	void *ctx = cfg->ops->fops_get_context(file);
 	struct device *dev = &cfg->dev->dev;
 	struct ctx_info *ctxi = NULL;
 	struct page *err_page = NULL;
@@ -1101,7 +1101,7 @@ static int cxlflash_mmap_fault(struct vm_fault *vmf)
 	int rc = 0;
 	int ctxid;
 
-	ctxid = cxl_process_element(ctx);
+	ctxid = cfg->ops->process_element(ctx);
 	if (unlikely(ctxid < 0)) {
 		dev_err(dev, "%s: Context %p was closed ctxid=%d\n",
 			__func__, ctx, ctxid);
@@ -1164,16 +1164,16 @@ static const struct vm_operations_struct cxlflash_mmap_vmops = {
  */
 static int cxlflash_cxl_mmap(struct file *file, struct vm_area_struct *vma)
 {
-	void *ctx = cxl_fops_get_context(file);
 	struct cxlflash_cfg *cfg = container_of(file->f_op, struct cxlflash_cfg,
 						cxl_fops);
+	void *ctx = cfg->ops->fops_get_context(file);
 	struct device *dev = &cfg->dev->dev;
 	struct ctx_info *ctxi = NULL;
 	enum ctx_ctrl ctrl = CTX_CTRL_ERR_FALLBACK | CTX_CTRL_FILE;
 	int ctxid;
 	int rc = 0;
 
-	ctxid = cxl_process_element(ctx);
+	ctxid = cfg->ops->process_element(ctx);
 	if (unlikely(ctxid < 0)) {
 		dev_err(dev, "%s: Context %p was closed ctxid=%d\n",
 			__func__, ctx, ctxid);
@@ -1190,7 +1190,7 @@ static int cxlflash_cxl_mmap(struct file *file, struct vm_area_struct *vma)
 
 	dev_dbg(dev, "%s: mmap for context %d\n", __func__, ctxid);
 
-	rc = cxl_fd_mmap(file, vma);
+	rc = cfg->ops->fd_mmap(file, vma);
 	if (likely(!rc)) {
 		/* Insert ourself in the mmap fault handler path */
 		ctxi->cxl_mmap_vmops = vma->vm_ops;
@@ -1309,7 +1309,6 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 	struct afu *afu = cfg->afu;
 	struct llun_info *lli = sdev->hostdata;
 	struct glun_info *gli = lli->parent;
-	struct cxl_ioctl_start_work work = { 0 };
 	struct ctx_info *ctxi = NULL;
 	struct lun_access *lun_access = NULL;
 	int rc = 0;
@@ -1397,7 +1396,7 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 		goto err;
 	}
 
-	ctx = cxl_dev_context_init(cfg->dev);
+	ctx = cfg->ops->dev_context_init(cfg->dev, cfg->afu_cookie);
 	if (IS_ERR_OR_NULL(ctx)) {
 		dev_err(dev, "%s: Could not initialize context %p\n",
 			__func__, ctx);
@@ -1405,24 +1404,21 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 		goto err;
 	}
 
-	work.num_interrupts = irqs;
-	work.flags = CXL_START_WORK_NUM_IRQS;
-
-	rc = cxl_start_work(ctx, &work);
+	rc = cfg->ops->start_work(ctx, irqs);
 	if (unlikely(rc)) {
 		dev_dbg(dev, "%s: Could not start context rc=%d\n",
 			__func__, rc);
 		goto err;
 	}
 
-	ctxid = cxl_process_element(ctx);
+	ctxid = cfg->ops->process_element(ctx);
 	if (unlikely((ctxid >= MAX_CONTEXT) || (ctxid < 0))) {
 		dev_err(dev, "%s: ctxid=%d invalid\n", __func__, ctxid);
 		rc = -EPERM;
 		goto err;
 	}
 
-	file = cxl_get_fd(ctx, &cfg->cxl_fops, &fd);
+	file = cfg->ops->get_fd(ctx, &cfg->cxl_fops, &fd);
 	if (unlikely(fd < 0)) {
 		rc = -ENODEV;
 		dev_err(dev, "%s: Could not get file descriptor\n", __func__);
@@ -1481,8 +1477,8 @@ static int cxlflash_disk_attach(struct scsi_device *sdev,
 err:
 	/* Cleanup CXL context; okay to 'stop' even if it was not started */
 	if (!IS_ERR_OR_NULL(ctx)) {
-		cxl_stop_context(ctx);
-		cxl_release_context(ctx);
+		cfg->ops->stop_context(ctx);
+		cfg->ops->release_context(ctx);
 		ctx = NULL;
 	}
 
@@ -1533,9 +1529,8 @@ static int recover_context(struct cxlflash_cfg *cfg,
 	struct file *file;
 	void *ctx;
 	struct afu *afu = cfg->afu;
-	struct cxl_ioctl_start_work work = { 0 };
 
-	ctx = cxl_dev_context_init(cfg->dev);
+	ctx = cfg->ops->dev_context_init(cfg->dev, cfg->afu_cookie);
 	if (IS_ERR_OR_NULL(ctx)) {
 		dev_err(dev, "%s: Could not initialize context %p\n",
 			__func__, ctx);
@@ -1543,24 +1538,21 @@ static int recover_context(struct cxlflash_cfg *cfg,
 		goto out;
 	}
 
-	work.num_interrupts = ctxi->irqs;
-	work.flags = CXL_START_WORK_NUM_IRQS;
-
-	rc = cxl_start_work(ctx, &work);
+	rc = cfg->ops->start_work(ctx, ctxi->irqs);
 	if (unlikely(rc)) {
 		dev_dbg(dev, "%s: Could not start context rc=%d\n",
 			__func__, rc);
 		goto err1;
 	}
 
-	ctxid = cxl_process_element(ctx);
+	ctxid = cfg->ops->process_element(ctx);
 	if (unlikely((ctxid >= MAX_CONTEXT) || (ctxid < 0))) {
 		dev_err(dev, "%s: ctxid=%d invalid\n", __func__, ctxid);
 		rc = -EPERM;
 		goto err2;
 	}
 
-	file = cxl_get_fd(ctx, &cfg->cxl_fops, &fd);
+	file = cfg->ops->get_fd(ctx, &cfg->cxl_fops, &fd);
 	if (unlikely(fd < 0)) {
 		rc = -ENODEV;
 		dev_err(dev, "%s: Could not get file descriptor\n", __func__);
@@ -1607,9 +1599,9 @@ static int recover_context(struct cxlflash_cfg *cfg,
 	fput(file);
 	put_unused_fd(fd);
 err2:
-	cxl_stop_context(ctx);
+	cfg->ops->stop_context(ctx);
 err1:
-	cxl_release_context(ctx);
+	cfg->ops->release_context(ctx);
 	goto out;
 }
 
-- 
2.1.0

Re: [PATCH 1/6] cxlflash: Reset command ioasc

[Andrew Donnellan]

On 04/01/18 09:54, Uma Krishnan wrote:
> In the event of a command failure, cxlflash returns the failure to the
> upper layers to process. After processing the error, when the command is
> queued again, the private command structure will not be zeroed and the
> ioasc could be stale. Per the SISLite specification, the AFU only sets the
> ioasc in the presence of a failure. Thus, even though the original command
> succeeds the second time, the command is considered a failure due to stale
> ioasc. This cycle repeats indefinitely and can cause a hang or IO failure.
> 
> To fix the issue, clear the ioasc before queuing any command.
> 
> Fixes: 479ad8e9d48c ("scsi: cxlflash: Remove zeroing of private command
> data")
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>

Should this go to stable?


> ---
>   drivers/scsi/cxlflash/main.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
> index 38b3a9c..48d3663 100644
> --- a/drivers/scsi/cxlflash/main.c
> +++ b/drivers/scsi/cxlflash/main.c
> @@ -620,6 +620,7 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
>   	cmd->parent = afu;
>   	cmd->hwq_index = hwq_index;
> 
> +	cmd->sa.ioasc = 0;
>   	cmd->rcb.ctx_id = hwq->ctx_hndl;
>   	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
>   	cmd->rcb.port_sel = CHAN2PORTMASK(scp->device->channel);
> 

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: [PATCH 1/6] cxlflash: Reset command ioasc

[Matthew R. Ochs]

On Wed, Jan 03, 2018 at 04:54:02PM -0600, Uma Krishnan wrote:
> In the event of a command failure, cxlflash returns the failure to the
> upper layers to process. After processing the error, when the command is
> queued again, the private command structure will not be zeroed and the
> ioasc could be stale. Per the SISLite specification, the AFU only sets the
> ioasc in the presence of a failure. Thus, even though the original command
> succeeds the second time, the command is considered a failure due to stale
> ioasc. This cycle repeats indefinitely and can cause a hang or IO failure.
> 
> To fix the issue, clear the ioasc before queuing any command.
> 
> Fixes: 479ad8e9d48c ("scsi: cxlflash: Remove zeroing of private command
> data")
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>

Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>

Re: [PATCH 1/6] cxlflash: Reset command ioasc

[Matthew R. Ochs]

On Thu, Jan 04, 2018 at 05:33:48PM +1100, Andrew Donnellan wrote:
> On 04/01/18 09:54, Uma Krishnan wrote:
> >In the event of a command failure, cxlflash returns the failure to the
> >upper layers to process. After processing the error, when the command is
> >queued again, the private command structure will not be zeroed and the
> >ioasc could be stale. Per the SISLite specification, the AFU only sets the
> >ioasc in the presence of a failure. Thus, even though the original command
> >succeeds the second time, the command is considered a failure due to stale
> >ioasc. This cycle repeats indefinitely and can cause a hang or IO failure.
> >
> >To fix the issue, clear the ioasc before queuing any command.
> >
> >Fixes: 479ad8e9d48c ("scsi: cxlflash: Remove zeroing of private command
> >data")
> >Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
> 
> Should this go to stable?

Not a bad idea.

Re: [PATCH 2/6] cxlflash: Update cxl-specific arguments to generic cookie

[Matthew R. Ochs]

On Wed, Jan 03, 2018 at 04:54:25PM -0600, Uma Krishnan wrote:
> Convert cxl-specific pointers to generic cookies to facilitate future
> enhancements.
> 
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>

Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>

Re: [PATCH 5/6] cxlflash: Adapter context init can return error

[Matthew R. Ochs]

On Wed, Jan 03, 2018 at 04:55:04PM -0600, Uma Krishnan wrote:
> Adapter context creation can return either NULL or an error pointer.
> Updating the check condition to reflect this.
> 
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>

Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>

Re: [PATCH 1/6] cxlflash: Reset command ioasc

[Uma Krishnan]

[-- Attachment #1: Type: text/html, Size: 1529 bytes --]

Re: [PATCH 1/6] cxlflash: Reset command ioasc

[Uma Krishnan]

> On Jan 7, 2018, at 1:36 PM, Matthew R. Ochs =
<mrochs@linux.vnet.ibm.com> wrote:
>=20
> On Thu, Jan 04, 2018 at 05:33:48PM +1100, Andrew Donnellan wrote:
>> On 04/01/18 09:54, Uma Krishnan wrote:
>>> In the event of a command failure, cxlflash returns the failure to =
the
>>> upper layers to process. After processing the error, when the =
command is
>>> queued again, the private command structure will not be zeroed and =
the
>>> ioasc could be stale. Per the SISLite specification, the AFU only =
sets the
>>> ioasc in the presence of a failure. Thus, even though the original =
command
>>> succeeds the second time, the command is considered a failure due to =
stale
>>> ioasc. This cycle repeats indefinitely and can cause a hang or IO =
failure.
>>>=20
>>> To fix the issue, clear the ioasc before queuing any command.
>>>=20
>>> Fixes: 479ad8e9d48c ("scsi: cxlflash: Remove zeroing of private =
command
>>> data")
>>> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
>>=20
>> Should this go to stable?
>=20
> Not a bad idea.
>=20

I will forward the patch to stable once the patch is in next/master.=20
Unless there is a need to send a V2. Thanks for pointing out Andrew !!=

Re: [PATCH 0/6] cxlflash: Miscellaneous patches

[Martin K. Petersen]

Uma,

> This patch series contains miscellaneous fixes. The first patch fixes
> a bug while the rest improve the code structure and prepare the code
> for future enhancements.

Added stable tag to first patch and applied series to
4.16/scsi-queue. Thank you!

-- 
Martin K. Petersen	Oracle Linux Engineering