[RFC] FC pass thru - Rev IV

[RFC] FC pass thru - Rev IV

[James Smart]

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect
  to have us stuff into the bsg cmd/sense data areas. These
  are now genericized so we can essentially pass any kind of
  transaction. It can be a request that has no transmit or
  receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the
  request/response data structures that should be shared
  between the application and the kernel entities. 

- I stripped out some things that were in the request
  structure that were actually LLD fields. Instead, I added
  a dd_bsgsize structure to the template, so the transport
  will allocate LLD work space along with the job structure.
  I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request
  has all the information necessary for the LLD, so that
  there is no need to have the LLD remap the transmit payload
  to figure things out. Granted, this comes at the cost of
  replicating some data items.

  Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity
  better. The "service" struct is now a job. I still have
  headaches with "request" (is it the blk request, or the job
  request, or what..)

- The CT/ELS response is a bit funky. I've noted that the
  way Emulex returns a response, vs Qlogic is a bit different,
  thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown,
  error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to
  use when verifying that the recipient knows how to decode
  vendor-specific message. I didn't do this with the netlink
  things as I was prepping it to not break kabi in existing
  and older kernels. But, I believe this is a good time to
  add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt
  documentation, but punted finishing it in lieu of sending
  this RFC. I'm starting from Seokman's original emails and
  will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to
make sure I'm calling the right blk_xx completion functions
relative to the setup flow, and to ensure that the "goose"
when I jump out while the rport is blocked is correct.

Comments welcome

-- james s



 Signed-off-by: James Smart <james.smart@emulex.com>

 ---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   53 ++
 6 files changed, 946 insertions(+), 4 deletions(-)


diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt
--- a/Documentation/scsi/scsi_fc_transport.txt	2008-09-23 15:11:57.000000000 -0400
+++ b/Documentation/scsi/scsi_fc_transport.txt	2008-11-18 15:54:25.000000000 -0500
@@ -1,10 +1,11 @@
                              SCSI FC Tansport
                  =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008
 Kernel Revisions for features:
   rports : <<TBS>>
   vports : 2.6.22 (? TBD)
+  bsg support : 2.6.29 (?TBD)
 
 
 Introduction
@@ -15,6 +16,7 @@ The FC transport can be found at:
   drivers/scsi/scsi_transport_fc.c
   include/scsi/scsi_transport_fc.h
   include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 
 This file is found at Documentation/scsi/scsi_fc_transport.txt
 
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+========================================================================
+<< To Be Supplied >>
+
+
+
+
 Credits
 =======
 The following people have contributed to this document:
diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
--- a/Documentation/scsi/scsi_mid_low_api.txt	2008-09-23 15:11:57.000000000 -0400
+++ b/Documentation/scsi/scsi_mid_low_api.txt	2008-11-18 15:54:25.000000000 -0500
@@ -1271,6 +1271,11 @@ of interest:
     hostdata[0]  - area reserved for LLD at end of struct Scsi_Host. Size
                    is set by the second argument (named 'xtr_bytes') to
                    scsi_host_alloc() or scsi_register().
+    vendor_id    - a unique value that identifies the vendor supplying
+                   the LLD for the Scsi_Host.  Used most often in validating
+                   vendor-specific message requests.  Value consists of an
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid formats.
 
 The scsi_host structure is defined in include/scsi/scsi_host.h
 
diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c
--- a/drivers/scsi/scsi_transport_fc.c	2008-10-18 10:32:52.000000000 -0400
+++ b/drivers/scsi/scsi_transport_fc.c	2008-11-18 15:58:38.000000000 -0500
@@ -35,6 +35,7 @@
 #include <linux/netlink.h>
 #include <net/netlink.h>
 #include <scsi/scsi_netlink_fc.h>
+#include <scsi/scsi_bsg_fc.h>
 #include "scsi_priv.h"
 #include "scsi_transport_fc_internal.h"
 
@@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
 static int fc_vport_setup(struct Scsi_Host *shost, int channel,
 	struct device *pdev, struct fc_vport_identifiers  *ids,
 	struct fc_vport **vport);
+static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs *);
+static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
+static void fc_bsg_remove(struct request_queue *);
+static void fc_bsg_goose_queue(struct fc_rport *);
 
 /*
  * Redefine so that we can have same named attributes in the
@@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
 		return -ENOMEM;
 	}
 
+	fc_bsg_hostadd(shost, fc_host);
+	/* ignore any bsg add error - we just can't do sgio */
+
+	return 0;
+}
+
+static int fc_host_remove(struct transport_container *tc, struct device *dev,
+			 struct device *cdev)
+{
+	struct Scsi_Host *shost = dev_to_shost(dev);
+	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
+
+	fc_bsg_remove(fc_host->rqst_q);
 	return 0;
 }
 
 static DECLARE_TRANSPORT_CLASS(fc_host_class,
 			       "fc_host",
 			       fc_host_setup,
-			       NULL,
+			       fc_host_remove,
 			       NULL);
 
 /*
@@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
 		scsi_flush_work(shost);
 
 	fc_terminate_rport_io(rport);
+
 	/*
 	 * Cancel any outstanding timers. These should really exist
 	 * only when rmmod'ing the LLDD and we're asking for
@@ -2411,6 +2430,8 @@ fc_rport_final_delete(struct work_struct
 	if (i->f->dev_loss_tmo_callbk)
 		i->f->dev_loss_tmo_callbk(rport);
 
+	fc_bsg_remove(rport->rqst_q);
+
 	transport_remove_device(dev);
 	device_del(dev);
 	transport_destroy_device(dev);
@@ -2498,6 +2519,9 @@ fc_rport_create(struct Scsi_Host *shost,
 	transport_add_device(dev);
 	transport_configure_device(dev);
 
+	fc_bsg_rportadd(shost, rport);
+	/* ignore any bsg add error - we just can't do sgio */
+
 	if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
 		/* initiate a scan of the target */
 		rport->flags |= FC_RPORT_SCAN_PENDING;
@@ -2661,6 +2685,8 @@ fc_remote_port_add(struct Scsi_Host *sho
 					spin_unlock_irqrestore(shost->host_lock,
 							flags);
 
+				fc_bsg_goose_queue(rport);
+
 				return rport;
 			}
 		}
@@ -3327,6 +3353,559 @@ fc_vport_sched_delete(struct work_struct
 }
 
 
+/*
+ * BSG support
+ */
+
+
+/**
+ * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
+ * @job:	fc_bsg_job that is to be torn down
+ */
+static void
+fc_destroy_bsgjob(struct fc_bsg_job *job)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->ref_cnt) {
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		return;
+	}
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	put_device(job->dev);	/* release reference for the request */
+
+	kfree(job->request_payload.sg_list);
+	kfree(job->reply_payload.sg_list);
+	kfree(job);
+}
+
+
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req->next_rq;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));
+
+	fc_destroy_bsgjob(job);
+}
+
+
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:	request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+	struct fc_bsg_job *job = (void *) req->special;
+	struct Scsi_Host *shost = job->shost;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	unsigned long flags;
+	int err = 0, done = 0;
+
+	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
+		return BLK_EH_RESET_TIMER;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->state_flags & FC_RQST_STATE_DONE)
+		done = 1;
+	else
+		job->ref_cnt++;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	if (!done && i->f->bsg_timeout) {
+		/* call LLDD to abort the i/o as it has timed out */
+		err = i->f->bsg_timeout(job);
+		if (err)
+			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
+				"abort failed with status %d\n", err);
+	}
+
+	if (!done) {
+		spin_lock_irqsave(&job->job_lock, flags);
+		job->ref_cnt--;
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		fc_destroy_bsgjob(job);
+	}
+
+	/* the blk_end_sync_io() doesn't check the error */
+	return BLK_EH_HANDLED;
+}
+
+
+
+static void
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+}
+
+
+/**
+ * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for the
+ *                   bsg request
+ * @shost:	SCSI Host corresponding to the bsg object
+ * @rport:	(optional) FC Remote Port corresponding to the bsg object
+ * @req:	BSG request that needs a job structure
+ */
+static int
+fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
+	struct request *req)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request *rsp = req->next_rq;
+	struct fc_bsg_job *job;
+
+	BUG_ON(req->special);
+
+	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
+			GFP_KERNEL);
+	if (!job)
+		return -ENOMEM;
+
+	/*
+	 * Note: this is a bit silly.
+	 * The request gets formatted as a SGIO v4 ioctl request, which
+	 * then gets reformatted as a blk request, which then gets
+	 * reformatted as a fc bsg request. And on completion, we have
+	 * to wrap return results such that SGIO v4 thinks it was a scsi
+	 * status.  I hope this was all worth it.
+	 */
+
+	req->special = job;
+	job->shost = shost;
+	job->rport = rport;
+	job->req = req;
+	if (i->f->dd_bsg_size)
+		job->dd_data = (void *)&job[1];
+	spin_lock_init(&job->job_lock);
+	job->request = (struct fc_bsg_request *)req->cmd;
+	job->request_len = req->cmd_len;
+	job->reply = req->sense;
+	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
+						 * allocated */
+	if (req->bio)
+		fc_bsg_map_buffer(&job->request_payload, req);
+	if (rsp && rsp->bio)
+		fc_bsg_map_buffer(&job->reply_payload, rsp);
+	job->job_done = fc_bsg_jobdone;
+	if (rport)
+		job->dev = &rport->dev;
+	else
+		job->dev = &shost->shost_gendev;
+	get_device(job->dev);		/* take a reference for the request */
+
+	job->ref_cnt = 1;
+
+	return 0;
+}
+
+
+enum fc_dispatch_result {
+	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
+	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
+	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
+};
+
+
+/**
+ * fc_bsg_host_dispatch - process fc host bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the host command */
+	switch (job->request->msgcode) {
+	case FC_BSG_HST_ADD_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_add_rport);
+		break;
+
+	case FC_BSG_HST_DEL_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_del_rport);
+		break;
+
+	case FC_BSG_HST_ELS_NOLOGIN:
+		cmdlen += sizeof(struct fc_bsg_host_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_host_msg;
+		}
+		break;
+
+	case FC_BSG_HST_VENDOR:
+		cmdlen += sizeof(struct fc_bsg_host_vendor);
+		if ((shost->hostt->vendor_id == 0L) || 
+		    (job->request->rqst_data.h_vendor.vendor_id !=
+			shost->hostt->vendor_id)) {
+			ret = -ESRCH;
+			goto fail_host_msg;
+		}
+		break;
+
+	default:
+		ret = -EBADR;
+		goto fail_host_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_host_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_host_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/*
+ * fc_bsg_goose_queue - restart rport queue in case it was stopped
+ * @rport:	rport to be restarted
+ */
+static void
+fc_bsg_goose_queue(struct fc_rport *rport)
+{
+	int flagset;
+
+	if (!rport->rqst_q)
+		return;
+
+	get_device(&rport->dev);
+	
+	spin_lock(rport->rqst_q->queue_lock);
+	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags) &&
+		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
+	if (flagset)
+		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
+	__blk_run_queue(rport->rqst_q);
+	if (flagset)
+		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
+	spin_unlock(rport->rqst_q->queue_lock);
+
+	put_device(&rport->dev);
+}
+
+
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @rport:	rport request destined to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_rport *rport, struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the rport command */
+	switch (job->request->msgcode) {
+	case FC_BSG_RPT_ELS:
+	case FC_BSG_RPT_CT:
+		cmdlen += sizeof(struct fc_bsg_rport_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_rport_msg;
+		}
+		break;
+	default:
+		ret = -EBADR;
+		goto fail_rport_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_rport_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_rport_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:		request queue to manage
+ * @shost:	Scsi_Host related to the bsg object
+ * @rport:	FC remote port related to the bsg object (optional)
+ * @dev:	device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host *shost,
+		       struct fc_rport *rport, struct device *dev)
+{
+	struct request *req;
+	struct fc_bsg_job *job;
+	enum fc_dispatch_result ret;
+
+	if (!get_device(dev))
+		return;
+
+	while (!blk_queue_plugged(q)) {
+		req = elv_next_request(q);
+		if (!req)
+			break;
+
+		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
+				break;
+
+		blkdev_dequeue_request(req);
+
+		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
+			req->errors = -ENXIO;
+			blk_complete_request(req);
+			continue;
+		}
+
+		spin_unlock_irq(q->queue_lock);
+
+		ret = fc_req_to_bsgjob(shost, rport, req);
+		if (ret) {
+			req->errors = ret;
+			spin_lock_irq(q->queue_lock);
+			blk_complete_request(req);
+			continue;
+		}
+
+		job = req->special;
+
+		/* check if we have the msgcode value at least */
+		if (job->request_len < sizeof(uint32_t)) {
+			BUG_ON(job->reply_len < sizeof(uint32_t));
+			job->reply->result = -ENOMSG;
+			job->reply_len = sizeof(uint32_t);
+			fc_bsg_jobdone(job);
+			spin_lock_irq(q->queue_lock);
+			continue;
+		}
+		
+		/* the dispatch routines will unlock the queue_lock */
+		if (rport)
+			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
+		else
+			ret = fc_bsg_host_dispatch(q, shost, job);
+
+		/* did dispatcher hit state that can't process any more */
+		if (ret == FC_DISPATCH_BREAK)
+			break;
+
+		/* did dispatcher had released the lock */
+		if (ret == FC_DISPATCH_UNLOCKED)
+			spin_lock_irq(q->queue_lock);
+	}
+
+	spin_unlock_irq(q->queue_lock);
+	put_device(dev);
+	spin_lock_irq(q->queue_lock);
+}
+
+
+/**
+ * fc_bsg_host_handler - handler for bsg requests for a fc host
+ * @q:		fc host request queue 
+ */
+static void
+fc_bsg_host_handler(struct request_queue *q)
+{
+	struct Scsi_Host *shost = q->queuedata;
+
+	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
+}
+
+
+/**
+ * fc_bsg_rport_handler - handler for bsg requests for a fc rport
+ * @q:		rport request queue 
+ */
+static void
+fc_bsg_rport_handler(struct request_queue *q)
+{
+	struct fc_rport *rport = q->queuedata;
+	struct Scsi_Host *shost = rport_to_shost(rport);
+
+	fc_bsg_request_handler(q, shost, rport, &rport->dev);
+}
+
+
+/**
+ * fc_bsg_hostadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost for fc_host
+ * @fc_host:	fc_host adding the structures to
+ */
+static int
+fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs *fc_host)
+{
+	struct device *dev = &shost->shost_gendev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+
+	fc_host->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	snprintf(fc_host->bsg_name, sizeof(fc_host->bsg_name),
+		 "fc_host%d", shost->host_no);
+
+	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
+	if (!q) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - no request queue\n",
+				 shost->host_no);
+		return -ENOMEM;
+	} 
+
+	q->queuedata = shost;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, fc_host->bsg_name, NULL);
+	if (err) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - register queue\n",
+			 	shost->host_no);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	fc_host->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_rportadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost that rport is attached to
+ * @rport:	rport that the bsg hooks are being attached to
+ */
+static int
+fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
+{
+	struct device *dev = &rport->dev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+
+	rport->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
+	if (!q) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - no request queue\n",
+				 dev->bus_id);
+		return -ENOMEM;
+	} 
+
+	q->queuedata = rport;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, dev->bus_id, NULL);
+	if (err) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - register queue\n",
+				 dev->bus_id);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	rport->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
+ * @q:	the request_queue that is to be torn down.
+ */
+static void
+fc_bsg_remove(struct request_queue *q)
+{
+	if (q) {
+		bsg_unregister_queue(q);
+		blk_cleanup_queue(q);
+	}
+}
+
+
 /* Original Author:  Martin Hicks */
 MODULE_AUTHOR("James Smart");
 MODULE_DESCRIPTION("FC Transport Attributes");
diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
--- a/include/scsi/scsi_bsg_fc.h	1969-12-31 19:00:00.000000000 -0500
+++ b/include/scsi/scsi_bsg_fc.h	2008-11-18 15:54:25.000000000 -0500
@@ -0,0 +1,291 @@
+/*
+ *  FC Transport BSG Interface
+ *
+ *  Copyright (C) 2008   James Smart, Emulex Corporation
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ *
+ */
+
+#ifndef SCSI_BSG_FC_H
+#define SCSI_BSG_FC_H
+
+/*
+ * This file intended to be included by both kernel and user space
+ */
+
+#include <scsi/scsi.h>
+#include <scsi/sg.h>
+
+/*
+ * FC Transport SGIO v4 BSG Message Support
+ */
+
+/* Default BSG request timeout (in seconds) */
+#define FC_DEFAULT_BSG_TIMEOUT		10
+
+
+/*
+ * Request Message Codes supported by the FC Transport
+ */
+
+/* define the class masks for the message codes */
+#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
+#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
+#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
+
+	/* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
+#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
+#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
+
+	/* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
+#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
+
+
+
+/*
+ * FC Address Identifiers in Message Structures :
+ *
+ *   Whenever a command payload contains a FC Address Identifier
+ *   (aka port_id), the value is effectively in big-endian
+ *   order, thus the array elements are decoded as follows:
+ *     element [0] is bits 23:16 of the FC Address Identifier
+ *     element [1] is bits 15:8 of the FC Address Identifier
+ *     element [2] is bits 7:0 of the FC Address Identifier
+ */
+
+
+/*
+ * FC Host Messages
+ */
+
+/* FC_BSG_HST_ADDR_PORT : */
+
+/* Request:
+ * This message requests the FC host to login to the remote port
+ * at the specified N_Port_Id.  The remote port is to be enumerated
+ * with the transport upon completion of the login.
+ */
+struct fc_bsg_host_add_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to login to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient
+ */
+
+
+/* FC_BSG_HST_DEL_RPORT : */
+
+/* Request:
+ * This message requests the FC host to remove an enumerated
+ * remote port and to terminate the login to it.
+ *
+ * Note: The driver is free to reject this request if it desires to
+ * remain logged in with the remote port.
+ */
+struct fc_bsg_host_del_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to logout of */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient 
+ */
+
+
+/* FC_BSG_HST_ELS_NOLOGIN : */
+
+/* Request:
+ * This message requests the FC_Host to send an ELS to a specific
+ * N_Port_ID. The host does not need to log into the remote port,
+ * nor does it need to enumerate the rport for further traffic
+ * (although, the FC host is free to do so if it desires).
+ */
+struct fc_bsg_host_els {
+	/* 
+	 * ELS Command Code being sent (must be the same as byte 0
+	 * of the payload)
+	 */
+	uint8_t 	command_code;
+
+	/* FC Address Identier of the remote port to send the ELS to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ */
+/* fc_bsg_ctels_reply->status values */
+#define FC_CTELS_STATUS_OK	0x00000000
+#define FC_CTELS_STATUS_REJECT	0x00000001
+#define FC_CTELS_STATUS_P_RJT	0x00000002
+#define FC_CTELS_STATUS_F_RJT	0x00000003
+#define FC_CTELS_STATUS_P_BSY	0x00000004
+#define FC_CTELS_STATUS_F_BSY	0x00000006
+struct fc_bsg_ctels_reply {
+	/*
+	 * Note: An ELS LS_RJT may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the ELS receive payload to determine
+	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
+	 *     data will be in word 1.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
+	 *   the receive payload word 0 indicates LS_ACC
+	 *   (e.g. value is 0x02xxxxxx).
+	 *
+	 * Note: Similarly, a CT Reject may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the CT receive payload to determine
+	 *     Accept or Reject (by contents of word 2). The reject
+	 *     data will be in word 3.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: x_RJT/BSY status will indicae that the rjt_data field
+	 *   is valid and contains the reason/explanation values.
+	 */
+	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
+
+	/* valid if status is not FC_CTELS_STATUS_OK */
+	struct	{
+		uint8_t	action;		/* fragment_id for CT REJECT */
+		uint8_t	reason_code;
+		uint8_t	reason_explanation;
+		uint8_t	vendor_unique;
+	} rjt_data;
+};
+
+
+/* FC_BSG_HST_VENDOR : */
+
+/* Request:
+ * Note: When specifying vendor_id, be sure to read the Vendor Type and ID
+ *   formatting requirements specified in scsi_netlink.h
+ */
+struct fc_bsg_host_vendor {
+	/*
+	 * Identifies the vendor that the message is formatted for. This
+	 * should be the recipient of the message.
+	 */
+	uint64_t vendor_id;
+
+	/* start of vendor command area */
+	uint32_t vendor_cmd[0];
+};
+
+/* Response:
+ */
+struct fc_bsg_host_vendor_reply {
+	/* start of vendor response area */
+	uint32_t vendor_rsp[0];
+};
+
+
+
+/*
+ * FC Remote Port Messages
+ */
+
+/* FC_BSG_RPT_ELS : */
+
+/* Request:
+ * This message requests that an ELS be performed with the rport.
+ */
+struct fc_bsg_rport_els {
+	/* 
+	 * ELS Command Code being sent (must be the same as
+	 * byte 0 of the payload)
+	 */
+	uint8_t els_code;
+};
+
+/* Response:
+ * 
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_RPT_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the rport.
+ */
+struct fc_bsg_rport_ct {
+	/* 
+	 * We need words 0-2 of the generic preamble for the LLD's
+	 */
+	uint32_t	preamble_word0;	/* revision & IN_ID */
+	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
+	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+
+
+/* request (CDB) structure of the sg_io_v4 */
+struct fc_bsg_request {
+	uint32_t msgcode;
+	union {
+		struct fc_bsg_host_add_rport	h_addrport;
+		struct fc_bsg_host_del_rport	h_delrport;
+		struct fc_bsg_host_els	h_els;
+		struct fc_bsg_host_vendor	h_vendor;
+
+		struct fc_bsg_rport_els		r_els;
+		struct fc_bsg_rport_ct		r_ct;
+	} rqst_data;
+};
+
+
+/* response (request sense data) structure of the sg_io_v4 */
+struct fc_bsg_reply {
+	/*
+	 * The completion result. Result exists in two forms:
+	 *  if negative, it is an -Exxx system errno value. There will
+	 *    be no further reply information supplied.
+	 *  else, it's the 4-byte scsi error result, with driver, host,
+	 *    msg and status fields. The per-msgcode reply structure
+	 *    will contain valid data.
+	 */
+	uint32_t result;
+
+	/* If there was reply_payload, how much was recevied ? */
+	uint32_t reply_payload_rcv_len;
+
+	union {
+		struct fc_bsg_host_vendor_reply		vendor_reply;
+
+		struct fc_bsg_ctels_reply		ctels_reply;
+	} reply_data;
+};
+
+
+#endif /* SCSI_BSG_FC_H */
+
diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
--- a/include/scsi/scsi_host.h	2008-10-18 10:32:54.000000000 -0400
+++ b/include/scsi/scsi_host.h	2008-11-18 15:54:25.000000000 -0500
@@ -478,6 +478,15 @@ struct scsi_host_template {
 	 * module_init/module_exit.
 	 */
 	struct list_head legacy_hosts;
+
+	/*
+	 * Vendor Identifier associated with the host
+	 *
+	 * Note: When specifying vendor_id, be sure to read the
+	 *   Vendor Type and ID formatting requirements specified in
+	 *   scsi_netlink.h
+	 */
+	u64 vendor_id;
 };
 
 /*
diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h
--- a/include/scsi/scsi_transport_fc.h	2008-10-18 10:32:54.000000000 -0400
+++ b/include/scsi/scsi_transport_fc.h	2008-11-18 15:54:25.000000000 -0500
@@ -33,7 +33,6 @@
 
 struct scsi_transport_template;
 
-
 /*
  * FC Port definitions - Following FC HBAAPI guidelines
  *
@@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
  	struct delayed_work fail_io_work;
  	struct work_struct stgt_delete_work;
 	struct work_struct rport_delete_work;
+	struct request_queue *rqst_q;	/* bsg support */
 } __attribute__((aligned(sizeof(unsigned long))));
 
 /* bit field values for struct fc_rport "flags" field: */
@@ -513,6 +513,10 @@ struct fc_host_attrs {
 	struct workqueue_struct *work_q;
 	char devloss_work_q_name[20];
 	struct workqueue_struct *devloss_work_q;
+
+	/* bsg support */
+	char bsg_name[20];
+	struct request_queue *rqst_q;
 };
 
 #define shost_to_fc_host(x) \
@@ -578,6 +582,47 @@ struct fc_host_attrs {
 	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
 
 
+struct fc_bsg_buffer {
+	unsigned int payload_len;
+	int sg_cnt;
+	struct scatterlist *sg_list;
+};
+
+/* Values for fc_bsg_job->state_flags (bitflags) */
+#define FC_RQST_STATE_INPROGRESS	0
+#define FC_RQST_STATE_DONE		1
+
+struct fc_bsg_job {
+	struct Scsi_Host *shost;
+	struct fc_rport *rport;
+	struct device *dev;
+	struct request *req;
+	spinlock_t job_lock;
+	unsigned int state_flags;
+	unsigned int ref_cnt;
+	void (*job_done)(struct fc_bsg_job *);
+
+	struct fc_bsg_request *request;
+	struct fc_bsg_reply *reply;
+	unsigned int request_len;
+	unsigned int reply_len;
+	/*
+	 * On entry : reply_len indicates the buffer size allocated for
+	 * the reply.
+	 *
+	 * Upon completion : the message handler must set reply_len
+	 *  to indicates the size of the reply to be returned to the
+	 *  caller.
+	 */
+
+	/* DMA payloads for the request/response */
+	struct fc_bsg_buffer request_payload;
+	struct fc_bsg_buffer reply_payload;
+
+	void *dd_data;			/* Used for driver-specific storage */
+};
+
+
 /* The functions by which the transport class and the driver communicate */
 struct fc_function_template {
 	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
@@ -613,9 +658,14 @@ struct fc_function_template {
 	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
 	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
 
+	/* bsg support */
+	int	(* bsg_request)(struct fc_bsg_job *);
+	int	(* bsg_timeout)(struct fc_bsg_job *);
+
 	/* allocation lengths for host-specific data */
 	u32	 			dd_fcrport_size;
 	u32	 			dd_fcvport_size;
+	u32				dd_bsg_size;
 
 	/*
 	 * The driver sets these to tell the transport class it
@@ -736,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
 	vport->vport_state = new_state;
 }
 
-
 struct scsi_transport_template *fc_attach_transport(
 			struct fc_function_template *);
 void fc_release_transport(struct scsi_transport_template *);

Re: [RFC] FC pass thru - Rev IV

[Sven Schuetz]

Hi James,

thanks for the rework, looks pretty good two me :)
One first question below.

James Smart wrote:
<snip>
> +	/* fc_host Message Codes */
> +#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
> +#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
> +#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
> +#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
> +

Shouldn't we add FC_BSG_HST_CT_NOLOGIN as well?
As far as I can see from looking over the patch (correct me when I am wrong),
the way to issue a CT request now is to send an FC_BSG_HST_ADD_RPORT for the wka
port in question (if not enumerated by the driver, which zfcp does not do), then
send the FC_BSG_RPT_CT followed by a FC_BSG_HST_DEL_RPORT. I was under the
impression the CT stuff was one of the main reasons to have devices for the
fc_host. What do you think?

> +	/* fc_rport Message Codes */
> +#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
> +#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
> +
</snip>

Sven

Re: [RFC] FC pass thru - Rev IV

[James Smart]

Sven Schuetz wrote:
> Shouldn't we add FC_BSG_HST_CT_NOLOGIN as well?

I didn't add this as FC-GS-6 r9.21, which defines the CT protocol, says:
  4.5.2.1 Fabric login and N_Port login
   An Nx_Port shall perform Fabric Login, and shall perform N_Port Login
   with the WKA or N_Port Identifier where the Service is offered, in the
   manner that is specified in FC-FS, before making any requests
   of a Server provided by the Service. An Nx_Port that has completed its
   registration with a Server shall perform explicit N_Port Logout with
   the Service. An Nx_Port that has completed any other requests with a
   Server should also perform explicit N_Port Logout with the Service.

So - a login for CT requests is a must (at least as I read it).

That's not the case for ELS's, which can usually survive on an implicit 
login - especially things like ADISC/PDISC.   Login is also the
distinction in my opinion for whether something should or should not be
exported via an rport in the transport.

> As far as I can see from looking over the patch (correct me when I am wrong),
> the way to issue a CT request now is to send an FC_BSG_HST_ADD_RPORT for the wka
> port in question (if not enumerated by the driver, which zfcp does not do), then
> send the FC_BSG_RPT_CT followed by a FC_BSG_HST_DEL_RPORT. I was under the
> impression the CT stuff was one of the main reasons to have devices for the
> fc_host. What do you think?

Your flow is exactly what I intended...

I also did not involve all the rport build-up, etc in the transport for
the adding of the rport.  Just as today, all of this occurs within the
LLDD for normal discovery (deciding when to send the ELS's, when to make
the transport calls, etc) - I'm assuming the same holds true for the
add rport.

Also not addressed - how do we enable CT-receive and ELS-recieve. This,
due to the buffering and routing definitions (what process gets what)
is a fun problem.

Thanks Sven.

-- james s

Re: [RFC] FC pass thru - Rev IV

[Seokmann Ju]

On Nov 18, 2008, at 1:24 PM, James Smart wrote:

> All,
>
> I've reworked Seokmann's patch for the following items:
Thanks for the update. Overall, it looks good to me.
It looks much clear on what the feature support should looks
like, esp., for the "...some request types to the fc_host.."
as you mentioned in the earlier mails.
A couple of questions.
1. Should 'loopback' considered at this time?
2. Would  it be valuable to consider sending requests with
    WWNN:WWPN?

I'm looking through the patch and some questions are inlined.

Thank you,
Seokmann
>
> - Add an fchost interface for bsg requests
>
> - Formalized the request/response structures that I expect
>  to have us stuff into the bsg cmd/sense data areas. These
>  are now genericized so we can essentially pass any kind of
>  transaction. It can be a request that has no transmit or
>  receive payload, and simply returns a response.
>
> - A new file was created, scsi_bsg_fc.h, which contains the
>  request/response data structures that should be shared
>  between the application and the kernel entities.
>
> - I stripped out some things that were in the request
>  structure that were actually LLD fields. Instead, I added
>  a dd_bsgsize structure to the template, so the transport
>  will allocate LLD work space along with the job structure.
>  I expect the missing fields to move to this area.
>
> - I've made a strong attempt at ensuring that the request
>  has all the information necessary for the LLD, so that
>  there is no need to have the LLD remap the transmit payload
>  to figure things out. Granted, this comes at the cost of
>  replicating some data items.
>
>  Sven, I've added the CT information you needed as part of this.
>
> - I've renamed things quite a bit, hoping to make it clarity
>  better. The "service" struct is now a job. I still have
>  headaches with "request" (is it the blk request, or the job
>  request, or what..)
In my thought, the 'job' is preferred over the 'blk' as
contents the 'request' is nothing but FC specific service
requests.

>
>
> - The CT/ELS response is a bit funky. I've noted that the
>  way Emulex returns a response, vs Qlogic is a bit different,
>  thus the 2 ways to indicate "reject".
>
> - fixed a couple of bugs in Seokmann's code, in the teardown,
>  error flows, request que dma settings, etc.
>
> - I added a "vendor_id" field to the scsi_host_template to
>  use when verifying that the recipient knows how to decode
>  vendor-specific message. I didn't do this with the netlink
>  things as I was prepping it to not break kabi in existing
>  and older kernels. But, I believe this is a good time to
>  add it.
>
> - I've started the Documentation/scsi/scsi_transport_fc.txt
>  documentation, but punted finishing it in lieu of sending
>  this RFC. I'm starting from Seokman's original emails and
>  will be updating for this reformat.
>
> I'm only starting to debug this, so user beware.
>
> I could really use some code review from Fujita or Boaz, to
> make sure I'm calling the right blk_xx completion functions
> relative to the setup flow, and to ensure that the "goose"
> when I jump out while the rport is blocked is correct.
>
> Comments welcome
>
> -- james s
>
>
>
> Signed-off-by: James Smart <james.smart@emulex.com>
>
> ---
>
> Documentation/scsi/scsi_fc_transport.txt |   11
> Documentation/scsi/scsi_mid_low_api.txt  |    5
> drivers/scsi/scsi_transport_fc.c         |  581 +++++++++++++++++++++ 
> +++++++++-
> include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
> include/scsi/scsi_host.h                 |    9
> include/scsi/scsi_transport_fc.h         |   53 ++
> 6 files changed, 946 insertions(+), 4 deletions(-)
>
>
> diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/ 
> Documentation/scsi/scsi_fc_transport.txt
> --- a/Documentation/scsi/scsi_fc_transport.txt	2008-09-23  
> 15:11:57.000000000 -0400
> +++ b/Documentation/scsi/scsi_fc_transport.txt	2008-11-18  
> 15:54:25.000000000 -0500
> @@ -1,10 +1,11 @@
>                              SCSI FC Tansport
>                  =============================================
>
> -Date:  4/12/2007
> +Date:  11/18/2008
> Kernel Revisions for features:
>   rports : <<TBS>>
>   vports : 2.6.22 (? TBD)
> +  bsg support : 2.6.29 (?TBD)
>
>
> Introduction
> @@ -15,6 +16,7 @@ The FC transport can be found at:
>   drivers/scsi/scsi_transport_fc.c
>   include/scsi/scsi_transport_fc.h
>   include/scsi/scsi_netlink_fc.h
> +  include/scsi/scsi_bsg_fc.h
>
> This file is found at Documentation/scsi/scsi_fc_transport.txt
>
> @@ -472,6 +474,13 @@ int
> fc_vport_terminate(struct fc_vport *vport)
>
>
> +FC BSG support (CT & ELS passthru, and more)
> + 
> = 
> = 
> ======================================================================
> +<< To Be Supplied >>
> +
> +
> +
> +
> Credits
> =======
> The following people have contributed to this document:
> diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/ 
> scsi/scsi_mid_low_api.txt
> --- a/Documentation/scsi/scsi_mid_low_api.txt	2008-09-23  
> 15:11:57.000000000 -0400
> +++ b/Documentation/scsi/scsi_mid_low_api.txt	2008-11-18  
> 15:54:25.000000000 -0500
> @@ -1271,6 +1271,11 @@ of interest:
>     hostdata[0]  - area reserved for LLD at end of struct Scsi_Host.  
> Size
>                    is set by the second argument (named 'xtr_bytes')  
> to
>                    scsi_host_alloc() or scsi_register().
> +    vendor_id    - a unique value that identifies the vendor  
> supplying
> +                   the LLD for the Scsi_Host.  Used most often in  
> validating
> +                   vendor-specific message requests.  Value  
> consists of an
> +                   identifier type and a vendor-specific value.
> +                   See scsi_netlink.h for a description of valid  
> formats.
>
> The scsi_host structure is defined in include/scsi/scsi_host.h
>
> diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/ 
> scsi_transport_fc.c
> --- a/drivers/scsi/scsi_transport_fc.c	2008-10-18 10:32:52.000000000  
> -0400
> +++ b/drivers/scsi/scsi_transport_fc.c	2008-11-18 15:58:38.000000000  
> -0500
> @@ -35,6 +35,7 @@
> #include <linux/netlink.h>
> #include <net/netlink.h>
> #include <scsi/scsi_netlink_fc.h>
> +#include <scsi/scsi_bsg_fc.h>
> #include "scsi_priv.h"
> #include "scsi_transport_fc_internal.h"
>
> @@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
> static int fc_vport_setup(struct Scsi_Host *shost, int channel,
> 	struct device *pdev, struct fc_vport_identifiers  *ids,
> 	struct fc_vport **vport);
> +static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs  
> *);
> +static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
> +static void fc_bsg_remove(struct request_queue *);
> +static void fc_bsg_goose_queue(struct fc_rport *);
>
> /*
>  * Redefine so that we can have same named attributes in the
> @@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
> 		return -ENOMEM;
> 	}
>
> +	fc_bsg_hostadd(shost, fc_host);
> +	/* ignore any bsg add error - we just can't do sgio */
> +
> +	return 0;
> +}
> +
> +static int fc_host_remove(struct transport_container *tc, struct  
> device *dev,
> +			 struct device *cdev)
> +{
> +	struct Scsi_Host *shost = dev_to_shost(dev);
> +	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
> +
> +	fc_bsg_remove(fc_host->rqst_q);
> 	return 0;
> }
>
> static DECLARE_TRANSPORT_CLASS(fc_host_class,
> 			       "fc_host",
> 			       fc_host_setup,
> -			       NULL,
> +			       fc_host_remove,
> 			       NULL);
>
> /*
> @@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
> 		scsi_flush_work(shost);
>
> 	fc_terminate_rport_io(rport);
> +
> 	/*
> 	 * Cancel any outstanding timers. These should really exist
> 	 * only when rmmod'ing the LLDD and we're asking for
> @@ -2411,6 +2430,8 @@ fc_rport_final_delete(struct work_struct
> 	if (i->f->dev_loss_tmo_callbk)
> 		i->f->dev_loss_tmo_callbk(rport);
>
> +	fc_bsg_remove(rport->rqst_q);
> +
> 	transport_remove_device(dev);
> 	device_del(dev);
> 	transport_destroy_device(dev);
> @@ -2498,6 +2519,9 @@ fc_rport_create(struct Scsi_Host *shost,
> 	transport_add_device(dev);
> 	transport_configure_device(dev);
>
> +	fc_bsg_rportadd(shost, rport);
> +	/* ignore any bsg add error - we just can't do sgio */
> +
> 	if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
> 		/* initiate a scan of the target */
> 		rport->flags |= FC_RPORT_SCAN_PENDING;
> @@ -2661,6 +2685,8 @@ fc_remote_port_add(struct Scsi_Host *sho
> 					spin_unlock_irqrestore(shost->host_lock,
> 							flags);
>
> +				fc_bsg_goose_queue(rport);
> +
> 				return rport;
> 			}
> 		}
> @@ -3327,6 +3353,559 @@ fc_vport_sched_delete(struct work_struct
> }
>
>
> +/*
> + * BSG support
> + */
> +
> +
> +/**
> + * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
> + * @job:	fc_bsg_job that is to be torn down
> + */
> +static void
> +fc_destroy_bsgjob(struct fc_bsg_job *job)
> +{
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	if (job->ref_cnt) {
> +		spin_unlock_irqrestore(&job->job_lock, flags);
> +		return;
> +	}
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	put_device(job->dev);	/* release reference for the request */
> +
> +	kfree(job->request_payload.sg_list);
> +	kfree(job->reply_payload.sg_list);
> +	kfree(job);
> +}
> +
> +
> +/**
> + * fc_bsg_jobdone - completion routine for bsg requests that the  
> LLD has
> + *                  completed
> + * @job:	fc_bsg_job that is complete
> + */
> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +	struct request *req = job->req->next_rq;
> +	struct request *rsp = req->next_rq;
> +	unsigned long flags;
> +	int err;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	job->state_flags |= FC_RQST_STATE_DONE;
> +	job->ref_cnt--;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	err = job->req->errors = job->reply->result;
> +	if (err < 0)
> +		/* we're only returning the result field in the reply */
> +		job->req->sense_len = sizeof(uint32_t);
> +	else
> +		job->req->sense_len = job->reply_len;
> +
> +	/*
> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
> +	 * but try to be honest about the amount of rcv data received
> +	 */
> +	if (rsp)
> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> +	    			     job->reply->reply_payload_rcv_len);
> +	else
> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
> +
> +	fc_destroy_bsgjob(job);
> +}
> +
> +
> +/**
> + * fc_bsg_job_timeout - handler for when a bsg request timesout
> + * @req:	request that timed out
> + */
> +static enum blk_eh_timer_return
> +fc_bsg_job_timeout(struct request *req)
> +{
> +	struct fc_bsg_job *job = (void *) req->special;
> +	struct Scsi_Host *shost = job->shost;
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	unsigned long flags;
> +	int err = 0, done = 0;
> +
> +	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
> +		return BLK_EH_RESET_TIMER;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	if (job->state_flags & FC_RQST_STATE_DONE)
> +		done = 1;
> +	else
> +		job->ref_cnt++;
Any reason to increase ref_cnt here?
I think that the ref_cnt is already 1 when it came here so that we don't
want to change it to destroy the job successfully.

>
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	if (!done && i->f->bsg_timeout) {
> +		/* call LLDD to abort the i/o as it has timed out */
> +		err = i->f->bsg_timeout(job);
> +		if (err)
> +			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
> +				"abort failed with status %d\n", err);
> +	}
> +
> +	if (!done) {
> +		spin_lock_irqsave(&job->job_lock, flags);
> +		job->ref_cnt--;
> +		spin_unlock_irqrestore(&job->job_lock, flags);
> +		fc_destroy_bsgjob(job);
> +	}
> +
> +	/* the blk_end_sync_io() doesn't check the error */
> +	return BLK_EH_HANDLED;
> +}
> +
> +
> +
> +static void
> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
> +{
> +	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
> +
> +	BUG_ON(!req->nr_phys_segments);
> +
> +	buf->sg_list = kzalloc(sz, GFP_KERNEL);
> +	sg_init_table(buf->sg_list, req->nr_phys_segments);
> +	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
> +	buf->payload_len = req->data_len;
> +}
> +
> +
> +/**
> + * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for  
> the
> + *                   bsg request
> + * @shost:	SCSI Host corresponding to the bsg object
> + * @rport:	(optional) FC Remote Port corresponding to the bsg object
> + * @req:	BSG request that needs a job structure
> + */
> +static int
> +fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
> +	struct request *req)
> +{
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	struct request *rsp = req->next_rq;
> +	struct fc_bsg_job *job;
> +
> +	BUG_ON(req->special);
> +
> +	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
> +			GFP_KERNEL);
> +	if (!job)
> +		return -ENOMEM;
> +
> +	/*
> +	 * Note: this is a bit silly.
> +	 * The request gets formatted as a SGIO v4 ioctl request, which
> +	 * then gets reformatted as a blk request, which then gets
> +	 * reformatted as a fc bsg request. And on completion, we have
> +	 * to wrap return results such that SGIO v4 thinks it was a scsi
> +	 * status.  I hope this was all worth it.
> +	 */
> +
> +	req->special = job;
> +	job->shost = shost;
> +	job->rport = rport;
> +	job->req = req;
> +	if (i->f->dd_bsg_size)
> +		job->dd_data = (void *)&job[1];
> +	spin_lock_init(&job->job_lock);
> +	job->request = (struct fc_bsg_request *)req->cmd;
> +	job->request_len = req->cmd_len;
> +	job->reply = req->sense;
> +	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
> +						 * allocated */
> +	if (req->bio)
> +		fc_bsg_map_buffer(&job->request_payload, req);
> +	if (rsp && rsp->bio)
> +		fc_bsg_map_buffer(&job->reply_payload, rsp);
> +	job->job_done = fc_bsg_jobdone;
> +	if (rport)
> +		job->dev = &rport->dev;
> +	else
> +		job->dev = &shost->shost_gendev;
> +	get_device(job->dev);		/* take a reference for the request */
> +
> +	job->ref_cnt = 1;
> +
> +	return 0;
> +}
> +
> +
> +enum fc_dispatch_result {
> +	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
> +	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
> +	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
> +};
> +
> +
> +/**
> + * fc_bsg_host_dispatch - process fc host bsg requests and dispatch  
> to LLDD
> + * @shost:	scsi host rport attached to
> + * @job:	bsg job to be processed
> + */
> +static enum fc_dispatch_result
> +fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host  
> *shost,
> +			 struct fc_bsg_job *job)
> +{
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
> +	int ret;
> +
> +	/* Validate the host command */
> +	switch (job->request->msgcode) {
> +	case FC_BSG_HST_ADD_RPORT:
> +		cmdlen += sizeof(struct fc_bsg_host_add_rport);
> +		break;
> +
> +	case FC_BSG_HST_DEL_RPORT:
> +		cmdlen += sizeof(struct fc_bsg_host_del_rport);
> +		break;
> +
> +	case FC_BSG_HST_ELS_NOLOGIN:
> +		cmdlen += sizeof(struct fc_bsg_host_els);
> +		/* there better be a xmt and rcv payloads */
> +		if ((!job->request_payload.payload_len) ||
> +		    (!job->reply_payload.payload_len)) {
> +			ret = -EINVAL;
> +			goto fail_host_msg;
> +		}
> +		break;
> +
> +	case FC_BSG_HST_VENDOR:
> +		cmdlen += sizeof(struct fc_bsg_host_vendor);
> +		if ((shost->hostt->vendor_id == 0L) ||
> +		    (job->request->rqst_data.h_vendor.vendor_id !=
> +			shost->hostt->vendor_id)) {
> +			ret = -ESRCH;
> +			goto fail_host_msg;
> +		}
> +		break;
> +
> +	default:
> +		ret = -EBADR;
> +		goto fail_host_msg;
> +	}
> +
> +	/* check if we really have all the request data needed */
> +	if (job->request_len < cmdlen) {
> +		ret = -ENOMSG;
> +		goto fail_host_msg;
> +	}
> +
> +	ret = i->f->bsg_request(job);
> +	if (ret) {
> +fail_host_msg:
> +		/* return the errno failure code as the only status */
> +		BUG_ON(job->reply_len < sizeof(uint32_t));
> +		job->reply->result = ret;
> +		job->reply_len = sizeof(uint32_t);
> +		fc_bsg_jobdone(job);
> +		/* fall thru */
> +	}
> +
> +	return FC_DISPATCH_UNLOCKED;
> +}
> +
> +
> +/*
> + * fc_bsg_goose_queue - restart rport queue in case it was stopped
> + * @rport:	rport to be restarted
> + */
> +static void
> +fc_bsg_goose_queue(struct fc_rport *rport)
> +{
> +	int flagset;
> +
> +	if (!rport->rqst_q)
> +		return;
> +
> +	get_device(&rport->dev);
> +	
> +	spin_lock(rport->rqst_q->queue_lock);
> +	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
> >queue_flags) &&
> +		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
> +	if (flagset)
> +		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
> +	__blk_run_queue(rport->rqst_q);
> +	if (flagset)
> +		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
> +	spin_unlock(rport->rqst_q->queue_lock);
> +
> +	put_device(&rport->dev);
> +}
> +
> +
> +/**
> + * fc_bsg_rport_dispatch - process rport bsg requests and dispatch  
> to LLDD
> + * @shost:	scsi host rport attached to
> + * @rport:	rport request destined to
> + * @job:	bsg job to be processed
> + */
> +static enum fc_dispatch_result
> +fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host  
> *shost,
> +			 struct fc_rport *rport, struct fc_bsg_job *job)
> +{
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
> +	int ret;
> +
> +	/* Validate the rport command */
> +	switch (job->request->msgcode) {
> +	case FC_BSG_RPT_ELS:
> +	case FC_BSG_RPT_CT:
> +		cmdlen += sizeof(struct fc_bsg_rport_els);
> +		/* there better be a xmt and rcv payloads */
> +		if ((!job->request_payload.payload_len) ||
> +		    (!job->reply_payload.payload_len)) {
> +			ret = -EINVAL;
> +			goto fail_rport_msg;
> +		}
> +		break;
> +	default:
> +		ret = -EBADR;
> +		goto fail_rport_msg;
> +	}
> +
> +	/* check if we really have all the request data needed */
> +	if (job->request_len < cmdlen) {
> +		ret = -ENOMSG;
> +		goto fail_rport_msg;
> +	}
> +
> +	ret = i->f->bsg_request(job);
> +	if (ret) {
> +fail_rport_msg:
> +		/* return the errno failure code as the only status */
> +		BUG_ON(job->reply_len < sizeof(uint32_t));
> +		job->reply->result = ret;
> +		job->reply_len = sizeof(uint32_t);
> +		fc_bsg_jobdone(job);
> +		/* fall thru */
> +	}
> +
> +	return FC_DISPATCH_UNLOCKED;
> +}
> +
> +
> +/**
> + * fc_bsg_request_handler - generic handler for bsg requests
> + * @q:		request queue to manage
> + * @shost:	Scsi_Host related to the bsg object
> + * @rport:	FC remote port related to the bsg object (optional)
> + * @dev:	device structure for bsg object
> + */
> +static void
> +fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host  
> *shost,
> +		       struct fc_rport *rport, struct device *dev)
> +{
> +	struct request *req;
> +	struct fc_bsg_job *job;
> +	enum fc_dispatch_result ret;
> +
> +	if (!get_device(dev))
> +		return;
> +
> +	while (!blk_queue_plugged(q)) {
> +		req = elv_next_request(q);
> +		if (!req)
> +			break;
> +
> +		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
> +				break;
Shouldn't this be moved to into fc_bsg_rport_dispatch()?
>
> +
> +		blkdev_dequeue_request(req);
> +
> +		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
> +			req->errors = -ENXIO;
> +			blk_complete_request(req);
> +			continue;
> +		}
This one, too?
>
> +
> +		spin_unlock_irq(q->queue_lock);
> +
> +		ret = fc_req_to_bsgjob(shost, rport, req);
> +		if (ret) {
> +			req->errors = ret;
> +			spin_lock_irq(q->queue_lock);
> +			blk_complete_request(req);
Shouldn't this be blk_end_bidi_request()/blk_end_request()?

>
> +			continue;
> +		}
> +
> +		job = req->special;
> +
> +		/* check if we have the msgcode value at least */
> +		if (job->request_len < sizeof(uint32_t)) {
> +			BUG_ON(job->reply_len < sizeof(uint32_t));
> +			job->reply->result = -ENOMSG;
> +			job->reply_len = sizeof(uint32_t);
> +			fc_bsg_jobdone(job);
> +			spin_lock_irq(q->queue_lock);
> +			continue;
> +		}
> +		
> +		/* the dispatch routines will unlock the queue_lock */
> +		if (rport)
> +			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
> +		else
> +			ret = fc_bsg_host_dispatch(q, shost, job);
> +
> +		/* did dispatcher hit state that can't process any more */
> +		if (ret == FC_DISPATCH_BREAK)
> +			break;
> +
> +		/* did dispatcher had released the lock */
> +		if (ret == FC_DISPATCH_UNLOCKED)
> +			spin_lock_irq(q->queue_lock);
> +	}
> +
> +	spin_unlock_irq(q->queue_lock);
> +	put_device(dev);
> +	spin_lock_irq(q->queue_lock);
> +}
> +
> +
> +/**
> + * fc_bsg_host_handler - handler for bsg requests for a fc host
> + * @q:		fc host request queue
> + */
> +static void
> +fc_bsg_host_handler(struct request_queue *q)
> +{
> +	struct Scsi_Host *shost = q->queuedata;
> +
> +	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
> +}
> +
> +
> +/**
> + * fc_bsg_rport_handler - handler for bsg requests for a fc rport
> + * @q:		rport request queue
> + */
> +static void
> +fc_bsg_rport_handler(struct request_queue *q)
> +{
> +	struct fc_rport *rport = q->queuedata;
> +	struct Scsi_Host *shost = rport_to_shost(rport);
> +
> +	fc_bsg_request_handler(q, shost, rport, &rport->dev);
> +}
> +
> +
> +/**
> + * fc_bsg_hostadd - Create and add the bsg hooks so we can receive  
> requests
> + * @shost:	shost for fc_host
> + * @fc_host:	fc_host adding the structures to
> + */
> +static int
> +fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs  
> *fc_host)
> +{
> +	struct device *dev = &shost->shost_gendev;
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	struct request_queue *q;
> +	int err;
> +
> +	fc_host->rqst_q = NULL;
> +
> +	if (!i->f->bsg_request)
> +		return -ENOTSUPP;
> +
> +	snprintf(fc_host->bsg_name, sizeof(fc_host->bsg_name),
> +		 "fc_host%d", shost->host_no);
> +
> +	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
> +	if (!q) {
> +		printk(KERN_ERR "fc_host%d: bsg interface failed to "
> +				"initialize - no request queue\n",
> +				 shost->host_no);
> +		return -ENOMEM;
> +	}
> +
> +	q->queuedata = shost;
> +	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
> +	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
> +	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
> +
> +	err = bsg_register_queue(q, dev, fc_host->bsg_name, NULL);
> +	if (err) {
> +		printk(KERN_ERR "fc_host%d: bsg interface failed to "
> +				"initialize - register queue\n",
> +			 	shost->host_no);
> +		blk_cleanup_queue(q);
> +		return err;
> +	}
> +
> +	fc_host->rqst_q = q;
> +	return 0;
> +}
> +
> +
> +/**
> + * fc_bsg_rportadd - Create and add the bsg hooks so we can receive  
> requests
> + * @shost:	shost that rport is attached to
> + * @rport:	rport that the bsg hooks are being attached to
> + */
> +static int
> +fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
> +{
> +	struct device *dev = &rport->dev;
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	struct request_queue *q;
> +	int err;
> +
> +	rport->rqst_q = NULL;
> +
> +	if (!i->f->bsg_request)
> +		return -ENOTSUPP;
> +
> +	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
> +	if (!q) {
> +		printk(KERN_ERR "%s: bsg interface failed to "
> +				"initialize - no request queue\n",
> +				 dev->bus_id);
> +		return -ENOMEM;
> +	}
> +
> +	q->queuedata = rport;
> +	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
> +	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
> +	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
> +
> +	err = bsg_register_queue(q, dev, dev->bus_id, NULL);
> +	if (err) {
> +		printk(KERN_ERR "%s: bsg interface failed to "
> +				"initialize - register queue\n",
> +				 dev->bus_id);
> +		blk_cleanup_queue(q);
> +		return err;
> +	}
> +
> +	rport->rqst_q = q;
> +	return 0;
> +}
> +
> +
> +/**
> + * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
> + * @q:	the request_queue that is to be torn down.
> + */
> +static void
> +fc_bsg_remove(struct request_queue *q)
> +{
> +	if (q) {
> +		bsg_unregister_queue(q);
> +		blk_cleanup_queue(q);
> +	}
> +}
> +
> +
> /* Original Author:  Martin Hicks */
> MODULE_AUTHOR("James Smart");
> MODULE_DESCRIPTION("FC Transport Attributes");
> diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
> --- a/include/scsi/scsi_bsg_fc.h	1969-12-31 19:00:00.000000000 -0500
> +++ b/include/scsi/scsi_bsg_fc.h	2008-11-18 15:54:25.000000000 -0500
> @@ -0,0 +1,291 @@
> +/*
> + *  FC Transport BSG Interface
> + *
> + *  Copyright (C) 2008   James Smart, Emulex Corporation
> + *
> + *  This program is free software; you can redistribute it and/or  
> modify
> + *  it under the terms of the GNU General Public License as  
> published by
> + *  the Free Software Foundation; either version 2 of the License, or
> + *  (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with this program; if not, write to the Free Software
> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA   
> 02111-1307  USA
> + *
> + */
> +
> +#ifndef SCSI_BSG_FC_H
> +#define SCSI_BSG_FC_H
> +
> +/*
> + * This file intended to be included by both kernel and user space
> + */
> +
> +#include <scsi/scsi.h>
> +#include <scsi/sg.h>
> +
> +/*
> + * FC Transport SGIO v4 BSG Message Support
> + */
> +
> +/* Default BSG request timeout (in seconds) */
> +#define FC_DEFAULT_BSG_TIMEOUT		10
> +
> +
> +/*
> + * Request Message Codes supported by the FC Transport
> + */
> +
> +/* define the class masks for the message codes */
> +#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
> +#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
> +#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
> +
> +	/* fc_host Message Codes */
> +#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
> +#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
> +#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
> +#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
> +
> +	/* fc_rport Message Codes */
> +#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
> +#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
> +
> +
> +
> +/*
> + * FC Address Identifiers in Message Structures :
> + *
> + *   Whenever a command payload contains a FC Address Identifier
> + *   (aka port_id), the value is effectively in big-endian
> + *   order, thus the array elements are decoded as follows:
> + *     element [0] is bits 23:16 of the FC Address Identifier
> + *     element [1] is bits 15:8 of the FC Address Identifier
> + *     element [2] is bits 7:0 of the FC Address Identifier
> + */
> +
> +
> +/*
> + * FC Host Messages
> + */
> +
> +/* FC_BSG_HST_ADDR_PORT : */
> +
> +/* Request:
> + * This message requests the FC host to login to the remote port
> + * at the specified N_Port_Id.  The remote port is to be enumerated
> + * with the transport upon completion of the login.
> + */
> +struct fc_bsg_host_add_rport {
> +	uint8_t		reserved;
> +
> +	/* FC Address Identier of the remote port to login to */
> +	uint8_t		port_id[3];
> +};
> +
> +/* Response:
> + * There is no additional response data - fc_bsg_reply->result is  
> sufficient
> + */
> +
> +
> +/* FC_BSG_HST_DEL_RPORT : */
> +
> +/* Request:
> + * This message requests the FC host to remove an enumerated
> + * remote port and to terminate the login to it.
> + *
> + * Note: The driver is free to reject this request if it desires to
> + * remain logged in with the remote port.
> + */
> +struct fc_bsg_host_del_rport {
> +	uint8_t		reserved;
> +
> +	/* FC Address Identier of the remote port to logout of */
> +	uint8_t		port_id[3];
> +};
> +
> +/* Response:
> + * There is no additional response data - fc_bsg_reply->result is  
> sufficient
> + */
> +
> +
> +/* FC_BSG_HST_ELS_NOLOGIN : */
> +
> +/* Request:
> + * This message requests the FC_Host to send an ELS to a specific
> + * N_Port_ID. The host does not need to log into the remote port,
> + * nor does it need to enumerate the rport for further traffic
> + * (although, the FC host is free to do so if it desires).
> + */
> +struct fc_bsg_host_els {
> +	/*
> +	 * ELS Command Code being sent (must be the same as byte 0
> +	 * of the payload)
> +	 */
> +	uint8_t 	command_code;
> +
> +	/* FC Address Identier of the remote port to send the ELS to */
> +	uint8_t		port_id[3];
> +};
> +
> +/* Response:
> + */
> +/* fc_bsg_ctels_reply->status values */
> +#define FC_CTELS_STATUS_OK	0x00000000
> +#define FC_CTELS_STATUS_REJECT	0x00000001
> +#define FC_CTELS_STATUS_P_RJT	0x00000002
> +#define FC_CTELS_STATUS_F_RJT	0x00000003
> +#define FC_CTELS_STATUS_P_BSY	0x00000004
> +#define FC_CTELS_STATUS_F_BSY	0x00000006
> +struct fc_bsg_ctels_reply {
> +	/*
> +	 * Note: An ELS LS_RJT may be reported in 2 ways:
> +	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
> +	 *     is to look into the ELS receive payload to determine
> +	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
> +	 *     data will be in word 1.
> +	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
> +	 *     rjt_data field will contain valid data.
> +	 *
> +	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
> +	 *   the receive payload word 0 indicates LS_ACC
> +	 *   (e.g. value is 0x02xxxxxx).
> +	 *
> +	 * Note: Similarly, a CT Reject may be reported in 2 ways:
> +	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
> +	 *     is to look into the CT receive payload to determine
> +	 *     Accept or Reject (by contents of word 2). The reject
> +	 *     data will be in word 3.
> +	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
> +	 *     rjt_data field will contain valid data.
> +	 *
> +	 * Note: x_RJT/BSY status will indicae that the rjt_data field
> +	 *   is valid and contains the reason/explanation values.
> +	 */
> +	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
> +
> +	/* valid if status is not FC_CTELS_STATUS_OK */
> +	struct	{
> +		uint8_t	action;		/* fragment_id for CT REJECT */
> +		uint8_t	reason_code;
> +		uint8_t	reason_explanation;
> +		uint8_t	vendor_unique;
> +	} rjt_data;
> +};
> +
> +
> +/* FC_BSG_HST_VENDOR : */
> +
> +/* Request:
> + * Note: When specifying vendor_id, be sure to read the Vendor Type  
> and ID
> + *   formatting requirements specified in scsi_netlink.h
> + */
> +struct fc_bsg_host_vendor {
> +	/*
> +	 * Identifies the vendor that the message is formatted for. This
> +	 * should be the recipient of the message.
> +	 */
> +	uint64_t vendor_id;
> +
> +	/* start of vendor command area */
> +	uint32_t vendor_cmd[0];
> +};
> +
> +/* Response:
> + */
> +struct fc_bsg_host_vendor_reply {
> +	/* start of vendor response area */
> +	uint32_t vendor_rsp[0];
> +};
> +
> +
> +
> +/*
> + * FC Remote Port Messages
> + */
> +
> +/* FC_BSG_RPT_ELS : */
> +
> +/* Request:
> + * This message requests that an ELS be performed with the rport.
> + */
> +struct fc_bsg_rport_els {
> +	/*
> +	 * ELS Command Code being sent (must be the same as
> +	 * byte 0 of the payload)
> +	 */
> +	uint8_t els_code;
> +};
> +
> +/* Response:
> + *
> + * The reply structure is an fc_bsg_ctels_reply structure
> + */
> +
> +
> +/* FC_BSG_RPT_CT : */
> +
> +/* Request:
> + * This message requests that a CT Request be performed with the  
> rport.
> + */
> +struct fc_bsg_rport_ct {
> +	/*
> +	 * We need words 0-2 of the generic preamble for the LLD's
> +	 */
> +	uint32_t	preamble_word0;	/* revision & IN_ID */
> +	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
> +	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
> +};
> +/* Response:
> + *
> + * The reply structure is an fc_bsg_ctels_reply structure
> + */
> +
> +
> +
> +
> +/* request (CDB) structure of the sg_io_v4 */
> +struct fc_bsg_request {
> +	uint32_t msgcode;
> +	union {
> +		struct fc_bsg_host_add_rport	h_addrport;
> +		struct fc_bsg_host_del_rport	h_delrport;
> +		struct fc_bsg_host_els	h_els;
> +		struct fc_bsg_host_vendor	h_vendor;
> +
> +		struct fc_bsg_rport_els		r_els;
> +		struct fc_bsg_rport_ct		r_ct;
> +	} rqst_data;
> +};
> +
> +
> +/* response (request sense data) structure of the sg_io_v4 */
> +struct fc_bsg_reply {
> +	/*
> +	 * The completion result. Result exists in two forms:
> +	 *  if negative, it is an -Exxx system errno value. There will
> +	 *    be no further reply information supplied.
> +	 *  else, it's the 4-byte scsi error result, with driver, host,
> +	 *    msg and status fields. The per-msgcode reply structure
> +	 *    will contain valid data.
> +	 */
> +	uint32_t result;
> +
> +	/* If there was reply_payload, how much was recevied ? */
> +	uint32_t reply_payload_rcv_len;
> +
> +	union {
> +		struct fc_bsg_host_vendor_reply		vendor_reply;
> +
> +		struct fc_bsg_ctels_reply		ctels_reply;
> +	} reply_data;
> +};
> +
> +
> +#endif /* SCSI_BSG_FC_H */
> +
> diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
> --- a/include/scsi/scsi_host.h	2008-10-18 10:32:54.000000000 -0400
> +++ b/include/scsi/scsi_host.h	2008-11-18 15:54:25.000000000 -0500
> @@ -478,6 +478,15 @@ struct scsi_host_template {
> 	 * module_init/module_exit.
> 	 */
> 	struct list_head legacy_hosts;
> +
> +	/*
> +	 * Vendor Identifier associated with the host
> +	 *
> +	 * Note: When specifying vendor_id, be sure to read the
> +	 *   Vendor Type and ID formatting requirements specified in
> +	 *   scsi_netlink.h
> +	 */
> +	u64 vendor_id;
> };
>
> /*
> diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/ 
> scsi_transport_fc.h
> --- a/include/scsi/scsi_transport_fc.h	2008-10-18 10:32:54.000000000  
> -0400
> +++ b/include/scsi/scsi_transport_fc.h	2008-11-18 15:54:25.000000000  
> -0500
> @@ -33,7 +33,6 @@
>
> struct scsi_transport_template;
>
> -
> /*
>  * FC Port definitions - Following FC HBAAPI guidelines
>  *
> @@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
>  	struct delayed_work fail_io_work;
>  	struct work_struct stgt_delete_work;
> 	struct work_struct rport_delete_work;
> +	struct request_queue *rqst_q;	/* bsg support */
> } __attribute__((aligned(sizeof(unsigned long))));
>
> /* bit field values for struct fc_rport "flags" field: */
> @@ -513,6 +513,10 @@ struct fc_host_attrs {
> 	struct workqueue_struct *work_q;
> 	char devloss_work_q_name[20];
> 	struct workqueue_struct *devloss_work_q;
> +
> +	/* bsg support */
> +	char bsg_name[20];
> +	struct request_queue *rqst_q;
> };
>
> #define shost_to_fc_host(x) \
> @@ -578,6 +582,47 @@ struct fc_host_attrs {
> 	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
>
>
> +struct fc_bsg_buffer {
> +	unsigned int payload_len;
> +	int sg_cnt;
> +	struct scatterlist *sg_list;
> +};
> +
> +/* Values for fc_bsg_job->state_flags (bitflags) */
> +#define FC_RQST_STATE_INPROGRESS	0
> +#define FC_RQST_STATE_DONE		1
> +
> +struct fc_bsg_job {
> +	struct Scsi_Host *shost;
> +	struct fc_rport *rport;
> +	struct device *dev;
> +	struct request *req;
> +	spinlock_t job_lock;
> +	unsigned int state_flags;
> +	unsigned int ref_cnt;
> +	void (*job_done)(struct fc_bsg_job *);
> +
> +	struct fc_bsg_request *request;
> +	struct fc_bsg_reply *reply;
> +	unsigned int request_len;
> +	unsigned int reply_len;
> +	/*
> +	 * On entry : reply_len indicates the buffer size allocated for
> +	 * the reply.
> +	 *
> +	 * Upon completion : the message handler must set reply_len
> +	 *  to indicates the size of the reply to be returned to the
> +	 *  caller.
> +	 */
> +
> +	/* DMA payloads for the request/response */
> +	struct fc_bsg_buffer request_payload;
> +	struct fc_bsg_buffer reply_payload;
> +
> +	void *dd_data;			/* Used for driver-specific storage */
> +};
> +
> +
> /* The functions by which the transport class and the driver  
> communicate */
> struct fc_function_template {
> 	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
> @@ -613,9 +658,14 @@ struct fc_function_template {
> 	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
> 	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
>
> +	/* bsg support */
> +	int	(* bsg_request)(struct fc_bsg_job *);
> +	int	(* bsg_timeout)(struct fc_bsg_job *);
> +
> 	/* allocation lengths for host-specific data */
> 	u32	 			dd_fcrport_size;
> 	u32	 			dd_fcvport_size;
> +	u32				dd_bsg_size;
>
> 	/*
> 	 * The driver sets these to tell the transport class it
> @@ -736,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
> 	vport->vport_state = new_state;
> }
>
> -
> struct scsi_transport_template *fc_attach_transport(
> 			struct fc_function_template *);
> void fc_release_transport(struct scsi_transport_template *);
>
>
>

Re: [RFC] FC pass thru - Rev IV

[James Smart]

Seokmann Ju wrote:
> A couple of questions.
> 1. Should 'loopback' considered at this time?

:)

This is a topic for the transport generically, independent of any 
pass-thru thing.

> 2. Would  it be valuable to consider sending requests with
>     WWNN:WWPN?

I considered this, especially for the fchost ELS request. But it means 
the LLDD has to do a nameserver query to find the n_port_id
corresponding to the names before it sent the ELS. It started to feel
like too much ancillary stuff for the LLDD to do to support the
request - so I punted.  However, the app, where we typically don't
care if it has to do more work, is free to do this and resolve it to
the basic primitives. :)

>> +/**
>> + * fc_bsg_job_timeout - handler for when a bsg request timesout
>> + * @req:     request that timed out
>> + */
>> +static enum blk_eh_timer_return
>> +fc_bsg_job_timeout(struct request *req)
>> +{
>> +     struct fc_bsg_job *job = (void *) req->special;
>> +     struct Scsi_Host *shost = job->shost;
>> +     struct fc_internal *i = to_fc_internal(shost->transportt);
>> +     unsigned long flags;
>> +     int err = 0, done = 0;
>> +
>> +     if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
>> +             return BLK_EH_RESET_TIMER;
>> +
>> +     spin_lock_irqsave(&job->job_lock, flags);
>> +     if (job->state_flags & FC_RQST_STATE_DONE)
>> +             done = 1;
>> +     else
>> +             job->ref_cnt++;
> Any reason to increase ref_cnt here?
> I think that the ref_cnt is already 1 when it came here so that we don't
> want to change it to destroy the job successfully.

Yes - I'm taking an additional reference on the job, so that if the
completion routine is completing in parallel to the timeout code path,
it won't free the job out from under the timeout call that calls back
into the LLDD.


>> +/**
>> + * fc_bsg_request_handler - generic handler for bsg requests
>> + * @q:               request queue to manage
>> + * @shost:   Scsi_Host related to the bsg object
>> + * @rport:   FC remote port related to the bsg object (optional)
>> + * @dev:     device structure for bsg object
>> + */
>> +static void
>> +fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host
>> *shost,
>> +                    struct fc_rport *rport, struct device *dev)
>> +{
>> +     struct request *req;
>> +     struct fc_bsg_job *job;
>> +     enum fc_dispatch_result ret;
>> +
>> +     if (!get_device(dev))
>> +             return;
>> +
>> +     while (!blk_queue_plugged(q)) {
>> +             req = elv_next_request(q);
>> +             if (!req)
>> +                     break;
>> +
>> +             if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
>> +                             break;
> Shouldn't this be moved to into fc_bsg_rport_dispatch()?

I had originally done this, but moved it back to this location.

Why ?  if I moved this out, I also had to move out the code between this
and the call to fc_bsg_rport_dispatch() too.  This was just a code
organization choice so I could keep the calls to allocate the job, etc
within this one routine. It also created a nice division on which
code-path-on-error had to be dealt with when it fails.  Now, this
generic routine is the only one that would call blk_complete_request()
or leave the request queued, while the rport/fchost dispatch functions
*always* call the fc_bsg_jobdone() function - on error or completion.

>> +
>> +             spin_unlock_irq(q->queue_lock);
>> +
>> +             ret = fc_req_to_bsgjob(shost, rport, req);
>> +             if (ret) {
>> +                     req->errors = ret;
>> +                     spin_lock_irq(q->queue_lock);
>> +                     blk_complete_request(req);
> Shouldn't this be blk_end_bidi_request()/blk_end_request()?

A good question, that I was hoping to have cleared up. If this was true, 
there were a lot of errors in the last passthru code.

If I remember right - I looked at what was also happening within scsi,
and it sure appeared as if blk_complete_request() will handle the dual
condition fine.

-- james s

Re: [RFC] FC pass thru - Rev IV

[Seokmann Ju]

On Nov 24, 2008, at 1:03 PM, James Smart wrote:

>
>
> Seokmann Ju wrote:
>> A couple of questions.
>> 1. Should 'loopback' considered at this time?
>
> :)
>
> This is a topic for the transport generically, independent of any  
> pass-thru thing.
>
>> 2. Would  it be valuable to consider sending requests with
>>    WWNN:WWPN?
>
> I considered this, especially for the fchost ELS request. But it  
> means the LLDD has to do a nameserver query to find the n_port_id
> corresponding to the names before it sent the ELS. It started to feel
> like too much ancillary stuff for the LLDD to do to support the
> request - so I punted.  However, the app, where we typically don't
> care if it has to do more work, is free to do this and resolve it to
> the basic primitives. :)
Thanks, I've got it.

>
>
>>> +/**
>>> + * fc_bsg_job_timeout - handler for when a bsg request timesout
>>> + * @req:     request that timed out
>>> + */
>>> +static enum blk_eh_timer_return
>>> +fc_bsg_job_timeout(struct request *req)
>>> +{
>>> +     struct fc_bsg_job *job = (void *) req->special;
>>> +     struct Scsi_Host *shost = job->shost;
>>> +     struct fc_internal *i = to_fc_internal(shost->transportt);
>>> +     unsigned long flags;
>>> +     int err = 0, done = 0;
>>> +
>>> +     if (job->rport && job->rport->port_state ==  
>>> FC_PORTSTATE_BLOCKED)
>>> +             return BLK_EH_RESET_TIMER;
>>> +
>>> +     spin_lock_irqsave(&job->job_lock, flags);
>>> +     if (job->state_flags & FC_RQST_STATE_DONE)
>>> +             done = 1;
>>> +     else
>>> +             job->ref_cnt++;
>> Any reason to increase ref_cnt here?
>> I think that the ref_cnt is already 1 when it came here so that we  
>> don't
>> want to change it to destroy the job successfully.
>
> Yes - I'm taking an additional reference on the job, so that if the
> completion routine is completing in parallel to the timeout code path,
> it won't free the job out from under the timeout call that calls back
> into the LLDD.
I see.. I've got it. Thanks.

>
>
>
>>> +/**
>>> + * fc_bsg_request_handler - generic handler for bsg requests
>>> + * @q:               request queue to manage
>>> + * @shost:   Scsi_Host related to the bsg object
>>> + * @rport:   FC remote port related to the bsg object (optional)
>>> + * @dev:     device structure for bsg object
>>> + */
>>> +static void
>>> +fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host
>>> *shost,
>>> +                    struct fc_rport *rport, struct device *dev)
>>> +{
>>> +     struct request *req;
>>> +     struct fc_bsg_job *job;
>>> +     enum fc_dispatch_result ret;
>>> +
>>> +     if (!get_device(dev))
>>> +             return;
>>> +
>>> +     while (!blk_queue_plugged(q)) {
>>> +             req = elv_next_request(q);
>>> +             if (!req)
>>> +                     break;
>>> +
>>> +             if (rport && (rport->port_state ==  
>>> FC_PORTSTATE_BLOCKED))
>>> +                             break;
>> Shouldn't this be moved to into fc_bsg_rport_dispatch()?
>
> I had originally done this, but moved it back to this location.
>
> Why ?  if I moved this out, I also had to move out the code between  
> this
> and the call to fc_bsg_rport_dispatch() too.  This was just a code
> organization choice so I could keep the calls to allocate the job, etc
> within this one routine. It also created a nice division on which
> code-path-on-error had to be dealt with when it fails.  Now, this
> generic routine is the only one that would call blk_complete_request()
> or leave the request queued, while the rport/fchost dispatch functions
> *always* call the fc_bsg_jobdone() function - on error or completion.
OK. I see your point and I agree on that "...error had to be dealt  
with when it fails.".

>
>
>>> +
>>> +             spin_unlock_irq(q->queue_lock);
>>> +
>>> +             ret = fc_req_to_bsgjob(shost, rport, req);
>>> +             if (ret) {
>>> +                     req->errors = ret;
>>> +                     spin_lock_irq(q->queue_lock);
>>> +                     blk_complete_request(req);
>> Shouldn't this be blk_end_bidi_request()/blk_end_request()?
>
> A good question, that I was hoping to have cleared up. If this was  
> true, there were a lot of errors in the last passthru code.
I'm not sure whether the previous code had the  
'blk_complete_request()' somewhere in the code, but my main concern  
was to make sure that the code completes the requests in such a way so  
that it remove the requests from the timeout_list by hitting  
blk_delete_timer().

Seokmann

Re: [RFC] FC pass thru - Rev IV

[Sven Schuetz]

James Smart wrote:
> Sven Schuetz wrote:
>> Shouldn't we add FC_BSG_HST_CT_NOLOGIN as well?
> 
> I didn't add this as FC-GS-6 r9.21, which defines the CT protocol, says:
>  4.5.2.1 Fabric login and N_Port login
>   An Nx_Port shall perform Fabric Login, and shall perform N_Port Login
>   with the WKA or N_Port Identifier where the Service is offered, in the
>   manner that is specified in FC-FS, before making any requests
>   of a Server provided by the Service. An Nx_Port that has completed its
>   registration with a Server shall perform explicit N_Port Logout with
>   the Service. An Nx_Port that has completed any other requests with a
>   Server should also perform explicit N_Port Logout with the Service.
> 
> So - a login for CT requests is a must (at least as I read it).
> 
> That's not the case for ELS's, which can usually survive on an implicit
> login - especially things like ADISC/PDISC.   Login is also the
> distinction in my opinion for whether something should or should not be
> exported via an rport in the transport.

Hi James,

I agree.

>> As far as I can see from looking over the patch (correct me when I am
>> wrong),
>> the way to issue a CT request now is to send an FC_BSG_HST_ADD_RPORT
>> for the wka
>> port in question (if not enumerated by the driver, which zfcp does not
>> do), then
>> send the FC_BSG_RPT_CT followed by a FC_BSG_HST_DEL_RPORT. I was under
>> the
>> impression the CT stuff was one of the main reasons to have devices
>> for the
>> fc_host. What do you think?
> 
> Your flow is exactly what I intended...

OK, let me be a bit more precise. Maybe I shouldn't have called it
FC_BSG_HST_CT_NOLOGIN but something like FC_BSG_HST_CT_NOT_ENUMERATE.

As the FS-GS dictates, we do a port login and logout for CT requests, but
completely in the driver. We do that for some internal CT stuff and I have
implemented it for userspace CT requests with Seokmanns patch as well. We just
do not advertise the ports to fc_transport (we are only logged in for the
duration of the request). Having to react to FC_BSG_HST_ADD_RPORT would mean to
rewrite our wka port handling, plus some of it is done in our firmware and not
in zfcp.

So my suggestion would be to add something like FC_BSG_HST_CT_NOT_ENUMERATE for
drivers that do the port logins on their own. Other drivers can respond with an
error code, making it necessary for the app to do the port login on their own
(FC_BSG_HST_ADD_RPORT), that means following the procedure I outlined in my
previous mail.

In addition, userspace apps that deal with complete CT frames (like
implementations of HBA API) would not have to rip the frame apapart to find out
the generic service/wka port in question and then do the port login. They can
just send the frame to the driver and rely on the driver to do the port
login/logout.

Sven

Re: [RFC] FC pass thru - Rev IV

[James Smart]

Seokmann Ju wrote:
>>>> +
>>>> +             spin_unlock_irq(q->queue_lock);
>>>> +
>>>> +             ret = fc_req_to_bsgjob(shost, rport, req);
>>>> +             if (ret) {
>>>> +                     req->errors = ret;
>>>> +                     spin_lock_irq(q->queue_lock);
>>>> +                     blk_complete_request(req);
>>> Shouldn't this be blk_end_bidi_request()/blk_end_request()?
>> A good question, that I was hoping to have cleared up. If this was
>> true, there were a lot of errors in the last passthru code.
> I'm not sure whether the previous code had the
> 'blk_complete_request()' somewhere in the code, but my main concern
> was to make sure that the code completes the requests in such a way so
> that it remove the requests from the timeout_list by hitting
> blk_delete_timer().

Hopefully, Jens, Fujita, or Boaz can give us some guidance...

I'd really like to ensure I'm calling the right callback :
a) if it differs based on being a bidi request or not
b) if it differs based on based on whether I've called blk_rq_map_sg()
c) if I don't dequeue it, whether my "goose queue" is handled right.

I found the blk api to be rather confusing on what when.

-- james

Re: [RFC] FC pass thru - Rev IV

[James Smart]

You are making many of the same arguments I tried to enumerate with 
James B when we first started discussing whether we had to enumerate 
everything before talking to it. Your points are a bit more eloquent 
than mine.  I put the ELS_NOLOGIN on the host in to test the argument.

I'm not married to requiring everything to be enumerated to talk to it 
via CT/ELS and figured in many cases, simply making the request to the 
host really was the right thing to do.  Being able to do so via the 
host, as well as via anything enumerated, plus the request to enumerate 
something, are enough primitives to cover everything.

So, following your proposal, let's add a FC_BSG_HST_CT request, that 
looks similar in form to the FC_BSG_HST_ELS_NOLOGIN request and with 
response semantics similar to the rport version.  We'll let the LLDD 
determine whether it needs to do logins/logouts, and whether it wants to 
enumerate anything in the process.

-- james s

Sven Schuetz wrote:
> OK, let me be a bit more precise. Maybe I shouldn't have called it
> FC_BSG_HST_CT_NOLOGIN but something like FC_BSG_HST_CT_NOT_ENUMERATE.
> 
> As the FS-GS dictates, we do a port login and logout for CT requests, but
> completely in the driver. We do that for some internal CT stuff and I have
> implemented it for userspace CT requests with Seokmanns patch as well. We just
> do not advertise the ports to fc_transport (we are only logged in for the
> duration of the request). Having to react to FC_BSG_HST_ADD_RPORT would mean to
> rewrite our wka port handling, plus some of it is done in our firmware and not
> in zfcp.
> 
> So my suggestion would be to add something like FC_BSG_HST_CT_NOT_ENUMERATE for
> drivers that do the port logins on their own. Other drivers can respond with an
> error code, making it necessary for the app to do the port login on their own
> (FC_BSG_HST_ADD_RPORT), that means following the procedure I outlined in my
> previous mail.
> 
> In addition, userspace apps that deal with complete CT frames (like
> implementations of HBA API) would not have to rip the frame apapart to find out
> the generic service/wka port in question and then do the port login. They can
> just send the frame to the driver and rely on the driver to do the port
> login/logout.
> 
> Sven
> 

Re: [RFC] FC pass thru - Rev IV

[Sven Schuetz]

James Smart wrote:

>- A new file was created, scsi_bsg_fc.h, which contains the
>  request/response data structures that should be shared
>  between the application and the kernel entities.

So we need to add this to include/scsi/Kbuild, right?

> +/**
> + * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
> + *                  completed
> + * @job:	fc_bsg_job that is complete
> + */
> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +	struct request *req = job->req->next_rq;

Shouldn't this be just job->req?

> +	struct request *rsp = req->next_rq;
> +	unsigned long flags;
> +	int err;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	job->state_flags |= FC_RQST_STATE_DONE;
> +	job->ref_cnt--;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	err = job->req->errors = job->reply->result;
> +	if (err < 0)
> +		/* we're only returning the result field in the reply */
> +		job->req->sense_len = sizeof(uint32_t);
> +	else
> +		job->req->sense_len = job->reply_len;
> +
> +	/*
> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
> +	 * but try to be honest about the amount of rcv data received
> +	 */
> +	if (rsp)
> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> +	    			     job->reply->reply_payload_rcv_len);

Here is a problem when a LLD does not set job->reply->reply_payload_rcv_len, but
has created a response.
blk_end_bidi_request will be called with a length of 0 for the last parameter
and will thus return "1: still buffers pending for this request", which we do
not evaluate. A few lines later the job is destroyed. But a few seconds later
the timeout kicks in, leading to a crash (I suppose because the job has been
destroyed).
I see two options:
1. check if we have a response and see if a length has been set - but what do
for cases which have a response and no length set? throw the request away?
2. always use the blk_rq_bytes(job->req->next_rq) from Seokmanns initial patch
(which I prefer)

> +	else
> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
> +
> +	fc_destroy_bsgjob(job);
> +}
> +

<snip>

> +/**
> + * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
> + * @shost:	scsi host rport attached to
> + * @rport:	rport request destined to
> + * @job:	bsg job to be processed
> + */
> +static enum fc_dispatch_result
> +fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
> +			 struct fc_rport *rport, struct fc_bsg_job *job)
> +{
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
> +	int ret;
> +
> +	/* Validate the rport command */
> +	switch (job->request->msgcode) {
> +	case FC_BSG_RPT_ELS:
> +	case FC_BSG_RPT_CT:
> +		cmdlen += sizeof(struct fc_bsg_rport_els);

Why do we do the same here for ELS and CT? The struct fc_bsg_rport_ct is some
bytes larger than for the ELS and should get its own case.

> +		/* there better be a xmt and rcv payloads */
> +		if ((!job->request_payload.payload_len) ||
> +		    (!job->reply_payload.payload_len)) {
> +			ret = -EINVAL;
> +			goto fail_rport_msg;
> +		}
> +		break;
> +	default:
> +		ret = -EBADR;
> +		goto fail_rport_msg;
> +	}
> +
> +	/* check if we really have all the request data needed */
> +	if (job->request_len < cmdlen) {
> +		ret = -ENOMSG;
> +		goto fail_rport_msg;
> +	}
> +
> +	ret = i->f->bsg_request(job);
> +	if (ret) {
> +fail_rport_msg:
> +		/* return the errno failure code as the only status */
> +		BUG_ON(job->reply_len < sizeof(uint32_t));
> +		job->reply->result = ret;
> +		job->reply_len = sizeof(uint32_t);
> +		fc_bsg_jobdone(job);
> +		/* fall thru */
> +	}
> +
> +	return FC_DISPATCH_UNLOCKED;
> +}
> +
> +

<snip>
> +struct fc_bsg_job {
> +	struct Scsi_Host *shost;
> +	struct fc_rport *rport;
> +	struct device *dev;
> +	struct request *req;
> +	spinlock_t job_lock;
> +	unsigned int state_flags;
> +	unsigned int ref_cnt;
> +	void (*job_done)(struct fc_bsg_job *);
> +
> +	struct fc_bsg_request *request;
> +	struct fc_bsg_reply *reply;
> +	unsigned int request_len;
> +	unsigned int reply_len;
> +	/*
> +	 * On entry : reply_len indicates the buffer size allocated for
> +	 * the reply.
> +	 *
> +	 * Upon completion : the message handler must set reply_len
> +	 *  to indicates the size of the reply to be returned to the
> +	 *  caller.
> +	 */

Do we need this field (reply_len)? Both the fc_bsg_buffer struct (for the vector
i/o) as well as the fc_bsg_reply struct (for the sense data) contain fields for
the payload length.

> +
> +	/* DMA payloads for the request/response */
> +	struct fc_bsg_buffer request_payload;
> +	struct fc_bsg_buffer reply_payload;
> +
> +	void *dd_data;			/* Used for driver-specific storage */
> +};


Sven

Re: [RFC] FC pass thru - Rev IV

[James Smart]

Sven Schuetz wrote:
> James Smart wrote:
> 
>> - A new file was created, scsi_bsg_fc.h, which contains the
>>  request/response data structures that should be shared
>>  between the application and the kernel entities.
> 
> So we need to add this to include/scsi/Kbuild, right?

yes, whatever file(s) control the inventory

>> +/**
>> + * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
>> + *                  completed
>> + * @job:     fc_bsg_job that is complete
>> + */
>> +static void
>> +fc_bsg_jobdone(struct fc_bsg_job *job)
>> +{
>> +     struct request *req = job->req->next_rq;
> 
> Shouldn't this be just job->req?

yes

>> +     struct request *rsp = req->next_rq;
>> +     unsigned long flags;
>> +     int err;
>> +
>> +     spin_lock_irqsave(&job->job_lock, flags);
>> +     job->state_flags |= FC_RQST_STATE_DONE;
>> +     job->ref_cnt--;
>> +     spin_unlock_irqrestore(&job->job_lock, flags);
>> +
>> +     err = job->req->errors = job->reply->result;
>> +     if (err < 0)
>> +             /* we're only returning the result field in the reply */
>> +             job->req->sense_len = sizeof(uint32_t);
>> +     else
>> +             job->req->sense_len = job->reply_len;
>> +
>> +     /*
>> +      * we'll cheat: tell blk layer all of the xmt data was sent.
>> +      * but try to be honest about the amount of rcv data received
>> +      */
>> +     if (rsp)
>> +             blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
>> +                                  job->reply->reply_payload_rcv_len);
> 
> Here is a problem when a LLD does not set job->reply->reply_payload_rcv_len, but
> has created a response.
> blk_end_bidi_request will be called with a length of 0 for the last parameter
> and will thus return "1: still buffers pending for this request", which we do
> not evaluate. A few lines later the job is destroyed. But a few seconds later
> the timeout kicks in, leading to a crash (I suppose because the job has been
> destroyed).
> I see two options:
> 1. check if we have a response and see if a length has been set - but what do
> for cases which have a response and no length set? throw the request away?
> 2. always use the blk_rq_bytes(job->req->next_rq) from Seokmanns initial patch
> (which I prefer)

Good Questions..
- If the driver isn't setting reply_payload_rcv_len, isn't it a driver
   bug ? (which we don't work around).

- But, I can see a 0 length receive being a valid value as well, which
   implies the blk infrastructure can't deal with it - this should be
   fixed.  How is this working with normal BIDI commands ?

- I tried to use the real length for the response, so that it tracked
   back through the calling sequences to bsg.c, so that it could return
   the length to the app. If we don't have the correct value being
   passed back - how does the app know how much data it actually
   received ?  This is the flaw I see in choosing #2.


>> +/**
>> + * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
>> + * @shost:   scsi host rport attached to
>> + * @rport:   rport request destined to
>> + * @job:     bsg job to be processed
>> + */
>> +static enum fc_dispatch_result
>> +fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
>> +                      struct fc_rport *rport, struct fc_bsg_job *job)
>> +{
>> +     struct fc_internal *i = to_fc_internal(shost->transportt);
>> +     int cmdlen = sizeof(uint32_t);  /* start with length of msgcode */
>> +     int ret;
>> +
>> +     /* Validate the rport command */
>> +     switch (job->request->msgcode) {
>> +     case FC_BSG_RPT_ELS:
>> +     case FC_BSG_RPT_CT:
>> +             cmdlen += sizeof(struct fc_bsg_rport_els);
> 
> Why do we do the same here for ELS and CT? The struct fc_bsg_rport_ct is some
> bytes larger than for the ELS and should get its own case.

A bug...

>> +struct fc_bsg_job {
>> +     struct Scsi_Host *shost;
>> +     struct fc_rport *rport;
>> +     struct device *dev;
>> +     struct request *req;
>> +     spinlock_t job_lock;
>> +     unsigned int state_flags;
>> +     unsigned int ref_cnt;
>> +     void (*job_done)(struct fc_bsg_job *);
>> +
>> +     struct fc_bsg_request *request;
>> +     struct fc_bsg_reply *reply;
>> +     unsigned int request_len;
>> +     unsigned int reply_len;
>> +     /*
>> +      * On entry : reply_len indicates the buffer size allocated for
>> +      * the reply.
>> +      *
>> +      * Upon completion : the message handler must set reply_len
>> +      *  to indicates the size of the reply to be returned to the
>> +      *  caller.
>> +      */
> 
> Do we need this field (reply_len)? Both the fc_bsg_buffer struct (for the vector
> i/o) as well as the fc_bsg_reply struct (for the sense data) contain fields for
> the payload length.

Yes. This isn't the "dma reply" length, but rather the amount of the 
sense buffer that is returned with "reply" or status information for the 
the actual request. E.g. upon finishing a job, the driver should always 
set this to sizeof(struct fc_bsg_ctels_reply), or whatever the reply 
structure is to be.

-- james

Re: [RFC] FC pass thru - Rev IV

[FUJITA Tomonori]

On Tue, 18 Nov 2008 16:24:58 -0500
James Smart <James.Smart@Emulex.Com> wrote:

> All,
> 
> I've reworked Seokmann's patch for the following items:
> - Add an fchost interface for bsg requests
> 
> - Formalized the request/response structures that I expect
>   to have us stuff into the bsg cmd/sense data areas. These
>   are now genericized so we can essentially pass any kind of
>   transaction. It can be a request that has no transmit or
>   receive payload, and simply returns a response.
> 
> - A new file was created, scsi_bsg_fc.h, which contains the
>   request/response data structures that should be shared
>   between the application and the kernel entities. 
> 
> - I stripped out some things that were in the request
>   structure that were actually LLD fields. Instead, I added
>   a dd_bsgsize structure to the template, so the transport
>   will allocate LLD work space along with the job structure.
>   I expect the missing fields to move to this area.
> 
> - I've made a strong attempt at ensuring that the request
>   has all the information necessary for the LLD, so that
>   there is no need to have the LLD remap the transmit payload
>   to figure things out. Granted, this comes at the cost of
>   replicating some data items.
> 
>   Sven, I've added the CT information you needed as part of this.
> 
> - I've renamed things quite a bit, hoping to make it clarity
>   better. The "service" struct is now a job. I still have
>   headaches with "request" (is it the blk request, or the job
>   request, or what..)
> 
> - The CT/ELS response is a bit funky. I've noted that the
>   way Emulex returns a response, vs Qlogic is a bit different,
>   thus the 2 ways to indicate "reject".
> 
> - fixed a couple of bugs in Seokmann's code, in the teardown,
>   error flows, request que dma settings, etc.
> 
> - I added a "vendor_id" field to the scsi_host_template to
>   use when verifying that the recipient knows how to decode
>   vendor-specific message. I didn't do this with the netlink
>   things as I was prepping it to not break kabi in existing
>   and older kernels. But, I believe this is a good time to
>   add it.
> 
> - I've started the Documentation/scsi/scsi_transport_fc.txt
>   documentation, but punted finishing it in lieu of sending
>   this RFC. I'm starting from Seokman's original emails and
>   will be updating for this reformat.
> 
> I'm only starting to debug this, so user beware.
> 
> I could really use some code review from Fujita or Boaz, to
> make sure I'm calling the right blk_xx completion functions
> relative to the setup flow, and to ensure that the "goose"
> when I jump out while the rport is blocked is correct.
> 
> Comments welcome
> 
> -- james s
> 
> 
> 
>  Signed-off-by: James Smart <james.smart@emulex.com>
> 
>  ---
> 
>  Documentation/scsi/scsi_fc_transport.txt |   11 
>  Documentation/scsi/scsi_mid_low_api.txt  |    5 
>  drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
>  include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
>  include/scsi/scsi_host.h                 |    9 
>  include/scsi/scsi_transport_fc.h         |   53 ++
>  6 files changed, 946 insertions(+), 4 deletions(-)

(snip)

> +/**
> + * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
> + *                  completed
> + * @job:	fc_bsg_job that is complete
> + */
> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +	struct request *req = job->req->next_rq;
> +	struct request *rsp = req->next_rq;
> +	unsigned long flags;
> +	int err;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	job->state_flags |= FC_RQST_STATE_DONE;
> +	job->ref_cnt--;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	err = job->req->errors = job->reply->result;
> +	if (err < 0)
> +		/* we're only returning the result field in the reply */
> +		job->req->sense_len = sizeof(uint32_t);
> +	else
> +		job->req->sense_len = job->reply_len;
> +
> +	/*
> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
> +	 * but try to be honest about the amount of rcv data received
> +	 */
> +	if (rsp)
> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> +	    			     job->reply->reply_payload_rcv_len);
> +	else
> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));

I think that you can use blk_end_bidi_request() for non-bidi requests:

	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
			rsp ?
			 job->reply->reply_payload_rcv_len : 0);


I guess that it would be better to have one function to complete a
request, instead of blk_end_bidi_request and blk_end_request.


> +	fc_destroy_bsgjob(job);
> +}
> +
> +
> +/**
> + * fc_bsg_job_timeout - handler for when a bsg request timesout
> + * @req:	request that timed out
> + */
> +static enum blk_eh_timer_return
> +fc_bsg_job_timeout(struct request *req)
> +{
> +	struct fc_bsg_job *job = (void *) req->special;
> +	struct Scsi_Host *shost = job->shost;
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	unsigned long flags;
> +	int err = 0, done = 0;
> +
> +	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
> +		return BLK_EH_RESET_TIMER;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	if (job->state_flags & FC_RQST_STATE_DONE)
> +		done = 1;
> +	else
> +		job->ref_cnt++;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	if (!done && i->f->bsg_timeout) {
> +		/* call LLDD to abort the i/o as it has timed out */
> +		err = i->f->bsg_timeout(job);
> +		if (err)
> +			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
> +				"abort failed with status %d\n", err);
> +	}
> +
> +	if (!done) {
> +		spin_lock_irqsave(&job->job_lock, flags);
> +		job->ref_cnt--;
> +		spin_unlock_irqrestore(&job->job_lock, flags);
> +		fc_destroy_bsgjob(job);
> +	}
> +
> +	/* the blk_end_sync_io() doesn't check the error */
> +	return BLK_EH_HANDLED;
> +}
> +
> +
> +
> +static void
> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
> +{
> +	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
> +
> +	BUG_ON(!req->nr_phys_segments);
> +
> +	buf->sg_list = kzalloc(sz, GFP_KERNEL);

Needs to handle ENOMEM?


> +	sg_init_table(buf->sg_list, req->nr_phys_segments);
> +	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
> +	buf->payload_len = req->data_len;
> +}
> +
> +
> +/**
> + * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for the
> + *                   bsg request
> + * @shost:	SCSI Host corresponding to the bsg object
> + * @rport:	(optional) FC Remote Port corresponding to the bsg object
> + * @req:	BSG request that needs a job structure
> + */
> +static int
> +fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
> +	struct request *req)
> +{
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	struct request *rsp = req->next_rq;
> +	struct fc_bsg_job *job;
> +
> +	BUG_ON(req->special);
> +
> +	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
> +			GFP_KERNEL);
> +	if (!job)
> +		return -ENOMEM;
> +
> +	/*
> +	 * Note: this is a bit silly.
> +	 * The request gets formatted as a SGIO v4 ioctl request, which
> +	 * then gets reformatted as a blk request, which then gets
> +	 * reformatted as a fc bsg request. And on completion, we have
> +	 * to wrap return results such that SGIO v4 thinks it was a scsi
> +	 * status.  I hope this was all worth it.
> +	 */
> +
> +	req->special = job;
> +	job->shost = shost;
> +	job->rport = rport;
> +	job->req = req;
> +	if (i->f->dd_bsg_size)
> +		job->dd_data = (void *)&job[1];
> +	spin_lock_init(&job->job_lock);
> +	job->request = (struct fc_bsg_request *)req->cmd;
> +	job->request_len = req->cmd_len;
> +	job->reply = req->sense;
> +	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
> +						 * allocated */
> +	if (req->bio)
> +		fc_bsg_map_buffer(&job->request_payload, req);
> +	if (rsp && rsp->bio)
> +		fc_bsg_map_buffer(&job->reply_payload, rsp);
> +	job->job_done = fc_bsg_jobdone;
> +	if (rport)
> +		job->dev = &rport->dev;
> +	else
> +		job->dev = &shost->shost_gendev;
> +	get_device(job->dev);		/* take a reference for the request */
> +
> +	job->ref_cnt = 1;
> +
> +	return 0;
> +}
> +
> +
> +enum fc_dispatch_result {
> +	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
> +	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
> +	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
> +};
> +
> +
> +/**
> + * fc_bsg_host_dispatch - process fc host bsg requests and dispatch to LLDD
> + * @shost:	scsi host rport attached to
> + * @job:	bsg job to be processed
> + */
> +static enum fc_dispatch_result
> +fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host *shost,
> +			 struct fc_bsg_job *job)
> +{
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
> +	int ret;
> +
> +	/* Validate the host command */
> +	switch (job->request->msgcode) {
> +	case FC_BSG_HST_ADD_RPORT:
> +		cmdlen += sizeof(struct fc_bsg_host_add_rport);
> +		break;
> +
> +	case FC_BSG_HST_DEL_RPORT:
> +		cmdlen += sizeof(struct fc_bsg_host_del_rport);
> +		break;
> +
> +	case FC_BSG_HST_ELS_NOLOGIN:
> +		cmdlen += sizeof(struct fc_bsg_host_els);
> +		/* there better be a xmt and rcv payloads */
> +		if ((!job->request_payload.payload_len) ||
> +		    (!job->reply_payload.payload_len)) {
> +			ret = -EINVAL;
> +			goto fail_host_msg;
> +		}
> +		break;
> +
> +	case FC_BSG_HST_VENDOR:
> +		cmdlen += sizeof(struct fc_bsg_host_vendor);
> +		if ((shost->hostt->vendor_id == 0L) || 
> +		    (job->request->rqst_data.h_vendor.vendor_id !=
> +			shost->hostt->vendor_id)) {
> +			ret = -ESRCH;
> +			goto fail_host_msg;
> +		}
> +		break;
> +
> +	default:
> +		ret = -EBADR;
> +		goto fail_host_msg;
> +	}
> +
> +	/* check if we really have all the request data needed */
> +	if (job->request_len < cmdlen) {
> +		ret = -ENOMSG;
> +		goto fail_host_msg;
> +	}
> +
> +	ret = i->f->bsg_request(job);
> +	if (ret) {
> +fail_host_msg:
> +		/* return the errno failure code as the only status */
> +		BUG_ON(job->reply_len < sizeof(uint32_t));
> +		job->reply->result = ret;
> +		job->reply_len = sizeof(uint32_t);
> +		fc_bsg_jobdone(job);
> +		/* fall thru */
> +	}
> +
> +	return FC_DISPATCH_UNLOCKED;
> +}
> +
> +
> +/*
> + * fc_bsg_goose_queue - restart rport queue in case it was stopped
> + * @rport:	rport to be restarted
> + */
> +static void
> +fc_bsg_goose_queue(struct fc_rport *rport)
> +{
> +	int flagset;
> +
> +	if (!rport->rqst_q)
> +		return;
> +
> +	get_device(&rport->dev);
> +	
> +	spin_lock(rport->rqst_q->queue_lock);
> +	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags) &&
> +		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
> +	if (flagset)
> +		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
> +	__blk_run_queue(rport->rqst_q);
> +	if (flagset)
> +		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
> +	spin_unlock(rport->rqst_q->queue_lock);
> +
> +	put_device(&rport->dev);
> +}
> +
> +
> +/**
> + * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
> + * @shost:	scsi host rport attached to
> + * @rport:	rport request destined to
> + * @job:	bsg job to be processed
> + */
> +static enum fc_dispatch_result
> +fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
> +			 struct fc_rport *rport, struct fc_bsg_job *job)
> +{
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
> +	int ret;
> +
> +	/* Validate the rport command */
> +	switch (job->request->msgcode) {
> +	case FC_BSG_RPT_ELS:
> +	case FC_BSG_RPT_CT:
> +		cmdlen += sizeof(struct fc_bsg_rport_els);
> +		/* there better be a xmt and rcv payloads */
> +		if ((!job->request_payload.payload_len) ||
> +		    (!job->reply_payload.payload_len)) {
> +			ret = -EINVAL;
> +			goto fail_rport_msg;
> +		}
> +		break;
> +	default:
> +		ret = -EBADR;
> +		goto fail_rport_msg;
> +	}
> +
> +	/* check if we really have all the request data needed */
> +	if (job->request_len < cmdlen) {
> +		ret = -ENOMSG;
> +		goto fail_rport_msg;
> +	}
> +
> +	ret = i->f->bsg_request(job);
> +	if (ret) {
> +fail_rport_msg:
> +		/* return the errno failure code as the only status */
> +		BUG_ON(job->reply_len < sizeof(uint32_t));
> +		job->reply->result = ret;
> +		job->reply_len = sizeof(uint32_t);
> +		fc_bsg_jobdone(job);
> +		/* fall thru */
> +	}
> +
> +	return FC_DISPATCH_UNLOCKED;
> +}
> +
> +
> +/**
> + * fc_bsg_request_handler - generic handler for bsg requests
> + * @q:		request queue to manage
> + * @shost:	Scsi_Host related to the bsg object
> + * @rport:	FC remote port related to the bsg object (optional)
> + * @dev:	device structure for bsg object
> + */
> +static void
> +fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host *shost,
> +		       struct fc_rport *rport, struct device *dev)
> +{
> +	struct request *req;
> +	struct fc_bsg_job *job;
> +	enum fc_dispatch_result ret;
> +
> +	if (!get_device(dev))
> +		return;
> +
> +	while (!blk_queue_plugged(q)) {
> +		req = elv_next_request(q);
> +		if (!req)
> +			break;
> +
> +		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
> +				break;
> +
> +		blkdev_dequeue_request(req);
> +
> +		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
> +			req->errors = -ENXIO;
> +			blk_complete_request(req);

I think that calling blk_complete_request() here crashes your kernel
since you don't set q->softirq_done_fn.

Calling blk_end_bidi_request should work here. Note that you need to
call spin_unlock_irq(q->queue_lock) before it.


> +			continue;
> +		}
> +
> +		spin_unlock_irq(q->queue_lock);
> +
> +		ret = fc_req_to_bsgjob(shost, rport, req);
> +		if (ret) {
> +			req->errors = ret;
> +			spin_lock_irq(q->queue_lock);
> +			blk_complete_request(req);

Ditto.

> +			continue;
> +		}
> +
> +		job = req->special;
> +
> +		/* check if we have the msgcode value at least */
> +		if (job->request_len < sizeof(uint32_t)) {
> +			BUG_ON(job->reply_len < sizeof(uint32_t));
> +			job->reply->result = -ENOMSG;
> +			job->reply_len = sizeof(uint32_t);
> +			fc_bsg_jobdone(job);
> +			spin_lock_irq(q->queue_lock);
> +			continue;
> +		}
> +		
> +		/* the dispatch routines will unlock the queue_lock */
> +		if (rport)
> +			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
> +		else
> +			ret = fc_bsg_host_dispatch(q, shost, job);
> +
> +		/* did dispatcher hit state that can't process any more */
> +		if (ret == FC_DISPATCH_BREAK)
> +			break;
> +
> +		/* did dispatcher had released the lock */
> +		if (ret == FC_DISPATCH_UNLOCKED)
> +			spin_lock_irq(q->queue_lock);
> +	}
> +
> +	spin_unlock_irq(q->queue_lock);
> +	put_device(dev);
> +	spin_lock_irq(q->queue_lock);
> +}
> +
> +
> +/**
> + * fc_bsg_host_handler - handler for bsg requests for a fc host
> + * @q:		fc host request queue 
> + */
> +static void
> +fc_bsg_host_handler(struct request_queue *q)
> +{
> +	struct Scsi_Host *shost = q->queuedata;
> +
> +	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
> +}
> +
> +
> +/**
> + * fc_bsg_rport_handler - handler for bsg requests for a fc rport
> + * @q:		rport request queue 
> + */
> +static void
> +fc_bsg_rport_handler(struct request_queue *q)
> +{
> +	struct fc_rport *rport = q->queuedata;
> +	struct Scsi_Host *shost = rport_to_shost(rport);
> +
> +	fc_bsg_request_handler(q, shost, rport, &rport->dev);
> +}
> +
> +
> +/**
> + * fc_bsg_hostadd - Create and add the bsg hooks so we can receive requests
> + * @shost:	shost for fc_host
> + * @fc_host:	fc_host adding the structures to
> + */
> +static int
> +fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs *fc_host)
> +{
> +	struct device *dev = &shost->shost_gendev;
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	struct request_queue *q;
> +	int err;
> +
> +	fc_host->rqst_q = NULL;
> +
> +	if (!i->f->bsg_request)
> +		return -ENOTSUPP;
> +
> +	snprintf(fc_host->bsg_name, sizeof(fc_host->bsg_name),
> +		 "fc_host%d", shost->host_no);

You don't need fc_host->bsg_name. bsg_register_queue() allocates
memory for the name.


> +	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
> +	if (!q) {
> +		printk(KERN_ERR "fc_host%d: bsg interface failed to "
> +				"initialize - no request queue\n",
> +				 shost->host_no);
> +		return -ENOMEM;
> +	} 
> +
> +	q->queuedata = shost;
> +	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
> +	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
> +	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
> +
> +	err = bsg_register_queue(q, dev, fc_host->bsg_name, NULL);
> +	if (err) {
> +		printk(KERN_ERR "fc_host%d: bsg interface failed to "
> +				"initialize - register queue\n",
> +			 	shost->host_no);
> +		blk_cleanup_queue(q);
> +		return err;
> +	}
> +
> +	fc_host->rqst_q = q;
> +	return 0;
> +}
> +
> +
> +/**
> + * fc_bsg_rportadd - Create and add the bsg hooks so we can receive requests
> + * @shost:	shost that rport is attached to
> + * @rport:	rport that the bsg hooks are being attached to
> + */
> +static int
> +fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
> +{
> +	struct device *dev = &rport->dev;
> +	struct fc_internal *i = to_fc_internal(shost->transportt);
> +	struct request_queue *q;
> +	int err;
> +
> +	rport->rqst_q = NULL;
> +
> +	if (!i->f->bsg_request)
> +		return -ENOTSUPP;
> +
> +	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
> +	if (!q) {
> +		printk(KERN_ERR "%s: bsg interface failed to "
> +				"initialize - no request queue\n",
> +				 dev->bus_id);
> +		return -ENOMEM;
> +	} 
> +
> +	q->queuedata = rport;
> +	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
> +	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
> +	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
> +
> +	err = bsg_register_queue(q, dev, dev->bus_id, NULL);

You don't need to pass dev->bus_id. It is used if you pass NULL.

bsg_register_queue(q, dev, NULL, NULL);


> +	if (err) {
> +		printk(KERN_ERR "%s: bsg interface failed to "
> +				"initialize - register queue\n",
> +				 dev->bus_id);
> +		blk_cleanup_queue(q);
> +		return err;
> +	}
> +
> +	rport->rqst_q = q;
> +	return 0;
> +}
> +
> +
> +/**
> + * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
> + * @q:	the request_queue that is to be torn down.
> + */
> +static void
> +fc_bsg_remove(struct request_queue *q)
> +{
> +	if (q) {
> +		bsg_unregister_queue(q);
> +		blk_cleanup_queue(q);
> +	}
> +}
> +
> +
>  /* Original Author:  Martin Hicks */
>  MODULE_AUTHOR("James Smart");
>  MODULE_DESCRIPTION("FC Transport Attributes");
> diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
> --- a/include/scsi/scsi_bsg_fc.h	1969-12-31 19:00:00.000000000 -0500
> +++ b/include/scsi/scsi_bsg_fc.h	2008-11-18 15:54:25.000000000 -0500
> @@ -0,0 +1,291 @@
> +/*
> + *  FC Transport BSG Interface
> + *
> + *  Copyright (C) 2008   James Smart, Emulex Corporation
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License as published by
> + *  the Free Software Foundation; either version 2 of the License, or
> + *  (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with this program; if not, write to the Free Software
> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
> + *
> + */
> +
> +#ifndef SCSI_BSG_FC_H
> +#define SCSI_BSG_FC_H
> +
> +/*
> + * This file intended to be included by both kernel and user space
> + */
> +
> +#include <scsi/scsi.h>
> +#include <scsi/sg.h>

<scsi/sg.h> is necessary?


> +/*
> + * FC Transport SGIO v4 BSG Message Support
> + */
> +
> +/* Default BSG request timeout (in seconds) */
> +#define FC_DEFAULT_BSG_TIMEOUT		10

Should be (10 * HZ)?


> +
> +/*
> + * Request Message Codes supported by the FC Transport
> + */
> +
> +/* define the class masks for the message codes */
> +#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
> +#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
> +#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
> +
> +	/* fc_host Message Codes */
> +#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
> +#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
> +#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
> +#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
> +
> +	/* fc_rport Message Codes */
> +#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
> +#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
> +
> +
> +
> +/*
> + * FC Address Identifiers in Message Structures :
> + *
> + *   Whenever a command payload contains a FC Address Identifier
> + *   (aka port_id), the value is effectively in big-endian
> + *   order, thus the array elements are decoded as follows:
> + *     element [0] is bits 23:16 of the FC Address Identifier
> + *     element [1] is bits 15:8 of the FC Address Identifier
> + *     element [2] is bits 7:0 of the FC Address Identifier
> + */
> +
> +
> +/*
> + * FC Host Messages
> + */
> +
> +/* FC_BSG_HST_ADDR_PORT : */
> +
> +/* Request:
> + * This message requests the FC host to login to the remote port
> + * at the specified N_Port_Id.  The remote port is to be enumerated
> + * with the transport upon completion of the login.
> + */
> +struct fc_bsg_host_add_rport {
> +	uint8_t		reserved;
> +
> +	/* FC Address Identier of the remote port to login to */
> +	uint8_t		port_id[3];
> +};
> +
> +/* Response:
> + * There is no additional response data - fc_bsg_reply->result is sufficient
> + */
> +
> +
> +/* FC_BSG_HST_DEL_RPORT : */
> +
> +/* Request:
> + * This message requests the FC host to remove an enumerated
> + * remote port and to terminate the login to it.
> + *
> + * Note: The driver is free to reject this request if it desires to
> + * remain logged in with the remote port.
> + */
> +struct fc_bsg_host_del_rport {
> +	uint8_t		reserved;
> +
> +	/* FC Address Identier of the remote port to logout of */
> +	uint8_t		port_id[3];
> +};
> +
> +/* Response:
> + * There is no additional response data - fc_bsg_reply->result is sufficient 
> + */
> +
> +
> +/* FC_BSG_HST_ELS_NOLOGIN : */
> +
> +/* Request:
> + * This message requests the FC_Host to send an ELS to a specific
> + * N_Port_ID. The host does not need to log into the remote port,
> + * nor does it need to enumerate the rport for further traffic
> + * (although, the FC host is free to do so if it desires).
> + */
> +struct fc_bsg_host_els {
> +	/* 
> +	 * ELS Command Code being sent (must be the same as byte 0
> +	 * of the payload)
> +	 */
> +	uint8_t 	command_code;
> +
> +	/* FC Address Identier of the remote port to send the ELS to */
> +	uint8_t		port_id[3];
> +};
> +
> +/* Response:
> + */
> +/* fc_bsg_ctels_reply->status values */
> +#define FC_CTELS_STATUS_OK	0x00000000
> +#define FC_CTELS_STATUS_REJECT	0x00000001
> +#define FC_CTELS_STATUS_P_RJT	0x00000002
> +#define FC_CTELS_STATUS_F_RJT	0x00000003
> +#define FC_CTELS_STATUS_P_BSY	0x00000004
> +#define FC_CTELS_STATUS_F_BSY	0x00000006
> +struct fc_bsg_ctels_reply {
> +	/*
> +	 * Note: An ELS LS_RJT may be reported in 2 ways:
> +	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
> +	 *     is to look into the ELS receive payload to determine
> +	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
> +	 *     data will be in word 1.
> +	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
> +	 *     rjt_data field will contain valid data.
> +	 *
> +	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
> +	 *   the receive payload word 0 indicates LS_ACC
> +	 *   (e.g. value is 0x02xxxxxx).
> +	 *
> +	 * Note: Similarly, a CT Reject may be reported in 2 ways:
> +	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
> +	 *     is to look into the CT receive payload to determine
> +	 *     Accept or Reject (by contents of word 2). The reject
> +	 *     data will be in word 3.
> +	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
> +	 *     rjt_data field will contain valid data.
> +	 *
> +	 * Note: x_RJT/BSY status will indicae that the rjt_data field
> +	 *   is valid and contains the reason/explanation values.
> +	 */
> +	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
> +
> +	/* valid if status is not FC_CTELS_STATUS_OK */
> +	struct	{
> +		uint8_t	action;		/* fragment_id for CT REJECT */
> +		uint8_t	reason_code;
> +		uint8_t	reason_explanation;
> +		uint8_t	vendor_unique;
> +	} rjt_data;
> +};
> +
> +
> +/* FC_BSG_HST_VENDOR : */
> +
> +/* Request:
> + * Note: When specifying vendor_id, be sure to read the Vendor Type and ID
> + *   formatting requirements specified in scsi_netlink.h
> + */
> +struct fc_bsg_host_vendor {
> +	/*
> +	 * Identifies the vendor that the message is formatted for. This
> +	 * should be the recipient of the message.
> +	 */
> +	uint64_t vendor_id;
> +
> +	/* start of vendor command area */
> +	uint32_t vendor_cmd[0];
> +};
> +
> +/* Response:
> + */
> +struct fc_bsg_host_vendor_reply {
> +	/* start of vendor response area */
> +	uint32_t vendor_rsp[0];
> +};
> +
> +
> +
> +/*
> + * FC Remote Port Messages
> + */
> +
> +/* FC_BSG_RPT_ELS : */
> +
> +/* Request:
> + * This message requests that an ELS be performed with the rport.
> + */
> +struct fc_bsg_rport_els {
> +	/* 
> +	 * ELS Command Code being sent (must be the same as
> +	 * byte 0 of the payload)
> +	 */
> +	uint8_t els_code;
> +};
> +
> +/* Response:
> + * 
> + * The reply structure is an fc_bsg_ctels_reply structure
> + */
> +
> +
> +/* FC_BSG_RPT_CT : */
> +
> +/* Request:
> + * This message requests that a CT Request be performed with the rport.
> + */
> +struct fc_bsg_rport_ct {
> +	/* 
> +	 * We need words 0-2 of the generic preamble for the LLD's
> +	 */
> +	uint32_t	preamble_word0;	/* revision & IN_ID */
> +	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
> +	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
> +};
> +/* Response:
> + *
> + * The reply structure is an fc_bsg_ctels_reply structure
> + */
> +
> +
> +
> +
> +/* request (CDB) structure of the sg_io_v4 */
> +struct fc_bsg_request {
> +	uint32_t msgcode;
> +	union {
> +		struct fc_bsg_host_add_rport	h_addrport;
> +		struct fc_bsg_host_del_rport	h_delrport;
> +		struct fc_bsg_host_els	h_els;
> +		struct fc_bsg_host_vendor	h_vendor;
> +
> +		struct fc_bsg_rport_els		r_els;
> +		struct fc_bsg_rport_ct		r_ct;
> +	} rqst_data;
> +};
> +
> +
> +/* response (request sense data) structure of the sg_io_v4 */
> +struct fc_bsg_reply {
> +	/*
> +	 * The completion result. Result exists in two forms:
> +	 *  if negative, it is an -Exxx system errno value. There will
> +	 *    be no further reply information supplied.
> +	 *  else, it's the 4-byte scsi error result, with driver, host,
> +	 *    msg and status fields. The per-msgcode reply structure
> +	 *    will contain valid data.
> +	 */
> +	uint32_t result;
> +
> +	/* If there was reply_payload, how much was recevied ? */
> +	uint32_t reply_payload_rcv_len;
> +
> +	union {
> +		struct fc_bsg_host_vendor_reply		vendor_reply;
> +
> +		struct fc_bsg_ctels_reply		ctels_reply;
> +	} reply_data;
> +};
> +
> +
> +#endif /* SCSI_BSG_FC_H */
> +
> diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
> --- a/include/scsi/scsi_host.h	2008-10-18 10:32:54.000000000 -0400
> +++ b/include/scsi/scsi_host.h	2008-11-18 15:54:25.000000000 -0500
> @@ -478,6 +478,15 @@ struct scsi_host_template {
>  	 * module_init/module_exit.
>  	 */
>  	struct list_head legacy_hosts;
> +
> +	/*
> +	 * Vendor Identifier associated with the host
> +	 *
> +	 * Note: When specifying vendor_id, be sure to read the
> +	 *   Vendor Type and ID formatting requirements specified in
> +	 *   scsi_netlink.h
> +	 */
> +	u64 vendor_id;
>  };
>  
>  /*
> diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h
> --- a/include/scsi/scsi_transport_fc.h	2008-10-18 10:32:54.000000000 -0400
> +++ b/include/scsi/scsi_transport_fc.h	2008-11-18 15:54:25.000000000 -0500
> @@ -33,7 +33,6 @@
>  
>  struct scsi_transport_template;
>  
> -
>  /*
>   * FC Port definitions - Following FC HBAAPI guidelines
>   *
> @@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
>   	struct delayed_work fail_io_work;
>   	struct work_struct stgt_delete_work;
>  	struct work_struct rport_delete_work;
> +	struct request_queue *rqst_q;	/* bsg support */
>  } __attribute__((aligned(sizeof(unsigned long))));
>  
>  /* bit field values for struct fc_rport "flags" field: */
> @@ -513,6 +513,10 @@ struct fc_host_attrs {
>  	struct workqueue_struct *work_q;
>  	char devloss_work_q_name[20];
>  	struct workqueue_struct *devloss_work_q;
> +
> +	/* bsg support */
> +	char bsg_name[20];
> +	struct request_queue *rqst_q;
>  };
>  
>  #define shost_to_fc_host(x) \
> @@ -578,6 +582,47 @@ struct fc_host_attrs {
>  	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
>  
>  
> +struct fc_bsg_buffer {
> +	unsigned int payload_len;
> +	int sg_cnt;
> +	struct scatterlist *sg_list;
> +};
> +
> +/* Values for fc_bsg_job->state_flags (bitflags) */
> +#define FC_RQST_STATE_INPROGRESS	0
> +#define FC_RQST_STATE_DONE		1
> +
> +struct fc_bsg_job {
> +	struct Scsi_Host *shost;
> +	struct fc_rport *rport;
> +	struct device *dev;
> +	struct request *req;
> +	spinlock_t job_lock;
> +	unsigned int state_flags;
> +	unsigned int ref_cnt;
> +	void (*job_done)(struct fc_bsg_job *);
> +
> +	struct fc_bsg_request *request;
> +	struct fc_bsg_reply *reply;
> +	unsigned int request_len;
> +	unsigned int reply_len;
> +	/*
> +	 * On entry : reply_len indicates the buffer size allocated for
> +	 * the reply.
> +	 *
> +	 * Upon completion : the message handler must set reply_len
> +	 *  to indicates the size of the reply to be returned to the
> +	 *  caller.
> +	 */
> +
> +	/* DMA payloads for the request/response */
> +	struct fc_bsg_buffer request_payload;
> +	struct fc_bsg_buffer reply_payload;
> +
> +	void *dd_data;			/* Used for driver-specific storage */
> +};
> +
> +
>  /* The functions by which the transport class and the driver communicate */
>  struct fc_function_template {
>  	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
> @@ -613,9 +658,14 @@ struct fc_function_template {
>  	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
>  	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
>  
> +	/* bsg support */
> +	int	(* bsg_request)(struct fc_bsg_job *);
> +	int	(* bsg_timeout)(struct fc_bsg_job *);
> +
>  	/* allocation lengths for host-specific data */
>  	u32	 			dd_fcrport_size;
>  	u32	 			dd_fcvport_size;
> +	u32				dd_bsg_size;
>  
>  	/*
>  	 * The driver sets these to tell the transport class it
> @@ -736,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
>  	vport->vport_state = new_state;
>  }
>  
> -
>  struct scsi_transport_template *fc_attach_transport(
>  			struct fc_function_template *);
>  void fc_release_transport(struct scsi_transport_template *);
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] FC pass thru - Rev IV

[Boaz Harrosh]

FUJITA Tomonori wrote:
> On Tue, 18 Nov 2008 16:24:58 -0500
> James Smart <James.Smart@Emulex.Com> wrote:
> 
>> All,
>>
>> I've reworked Seokmann's patch for the following items:
>> - Add an fchost interface for bsg requests
>>
>> - Formalized the request/response structures that I expect
>>   to have us stuff into the bsg cmd/sense data areas. These
>>   are now genericized so we can essentially pass any kind of
>>   transaction. It can be a request that has no transmit or
>>   receive payload, and simply returns a response.
>>
>> - A new file was created, scsi_bsg_fc.h, which contains the
>>   request/response data structures that should be shared
>>   between the application and the kernel entities. 
>>
>> - I stripped out some things that were in the request
>>   structure that were actually LLD fields. Instead, I added
>>   a dd_bsgsize structure to the template, so the transport
>>   will allocate LLD work space along with the job structure.
>>   I expect the missing fields to move to this area.
>>
>> - I've made a strong attempt at ensuring that the request
>>   has all the information necessary for the LLD, so that
>>   there is no need to have the LLD remap the transmit payload
>>   to figure things out. Granted, this comes at the cost of
>>   replicating some data items.
>>
>>   Sven, I've added the CT information you needed as part of this.
>>
>> - I've renamed things quite a bit, hoping to make it clarity
>>   better. The "service" struct is now a job. I still have
>>   headaches with "request" (is it the blk request, or the job
>>   request, or what..)
>>
>> - The CT/ELS response is a bit funky. I've noted that the
>>   way Emulex returns a response, vs Qlogic is a bit different,
>>   thus the 2 ways to indicate "reject".
>>
>> - fixed a couple of bugs in Seokmann's code, in the teardown,
>>   error flows, request que dma settings, etc.
>>
>> - I added a "vendor_id" field to the scsi_host_template to
>>   use when verifying that the recipient knows how to decode
>>   vendor-specific message. I didn't do this with the netlink
>>   things as I was prepping it to not break kabi in existing
>>   and older kernels. But, I believe this is a good time to
>>   add it.
>>
>> - I've started the Documentation/scsi/scsi_transport_fc.txt
>>   documentation, but punted finishing it in lieu of sending
>>   this RFC. I'm starting from Seokman's original emails and
>>   will be updating for this reformat.
>>
>> I'm only starting to debug this, so user beware.
>>
>> I could really use some code review from Fujita or Boaz, to
>> make sure I'm calling the right blk_xx completion functions
>> relative to the setup flow, and to ensure that the "goose"
>> when I jump out while the rport is blocked is correct.
>>
>> Comments welcome
>>
>> -- james s
>>
>>
>>
>>  Signed-off-by: James Smart <james.smart@emulex.com>
>>
>>  ---
>>
>>  Documentation/scsi/scsi_fc_transport.txt |   11 
>>  Documentation/scsi/scsi_mid_low_api.txt  |    5 
>>  drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
>>  include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
>>  include/scsi/scsi_host.h                 |    9 
>>  include/scsi/scsi_transport_fc.h         |   53 ++
>>  6 files changed, 946 insertions(+), 4 deletions(-)
> 
> (snip)
> 
>> +/**
>> + * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
>> + *                  completed
>> + * @job:	fc_bsg_job that is complete
>> + */
>> +static void
>> +fc_bsg_jobdone(struct fc_bsg_job *job)
>> +{
>> +	struct request *req = job->req->next_rq;
>> +	struct request *rsp = req->next_rq;
>> +	unsigned long flags;
>> +	int err;

+	unsigned bytes_requested = 0;

>> +
>> +	spin_lock_irqsave(&job->job_lock, flags);
>> +	job->state_flags |= FC_RQST_STATE_DONE;
>> +	job->ref_cnt--;
>> +	spin_unlock_irqrestore(&job->job_lock, flags);
>> +
>> +	err = job->req->errors = job->reply->result;
>> +	if (err < 0)
>> +		/* we're only returning the result field in the reply */
>> +		job->req->sense_len = sizeof(uint32_t);
>> +	else
>> +		job->req->sense_len = job->reply_len;
>> +
>> +	/*
>> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
>> +	 * but try to be honest about the amount of rcv data received
>> +	 */
>> +	if (rsp)
>> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
>> +	    			     job->reply->reply_payload_rcv_len);
>> +	else
>> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
> 
> I think that you can use blk_end_bidi_request() for non-bidi requests:
> 
> 	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> 			rsp ?
> 			 job->reply->reply_payload_rcv_len : 0);
> 
> 
> I guess that it would be better to have one function to complete a
> request, instead of blk_end_bidi_request and blk_end_request.
> 
> 

+	/*
+	 * tell blk layer all of the xmt data was sent.
+	 * but set residual count to: requested - received
+	 */
+
+	if (rsp) {
+		bytes_requested = blk_rq_bytes(rsp);
+		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
+	}
+
+	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);

The residual count is left in req->data_len. Does bsg have a way to return the
residual to user-mode? It must, since Pete was using that for sure. Note that
you are looking for the bidi_read residual count.

As was said by people. You must complete ALL bytes on both sides. Residual information
is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

>> +	fc_destroy_bsgjob(job);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_job_timeout - handler for when a bsg request timesout
>> + * @req:	request that timed out
>> + */
>> +static enum blk_eh_timer_return
>> +fc_bsg_job_timeout(struct request *req)
>> +{
>> +	struct fc_bsg_job *job = (void *) req->special;
>> +	struct Scsi_Host *shost = job->shost;
>> +	struct fc_internal *i = to_fc_internal(shost->transportt);
>> +	unsigned long flags;
>> +	int err = 0, done = 0;
>> +
>> +	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
>> +		return BLK_EH_RESET_TIMER;
>> +
>> +	spin_lock_irqsave(&job->job_lock, flags);
>> +	if (job->state_flags & FC_RQST_STATE_DONE)
>> +		done = 1;
>> +	else
>> +		job->ref_cnt++;
>> +	spin_unlock_irqrestore(&job->job_lock, flags);
>> +
>> +	if (!done && i->f->bsg_timeout) {
>> +		/* call LLDD to abort the i/o as it has timed out */
>> +		err = i->f->bsg_timeout(job);
>> +		if (err)
>> +			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
>> +				"abort failed with status %d\n", err);
>> +	}
>> +
>> +	if (!done) {
>> +		spin_lock_irqsave(&job->job_lock, flags);
>> +		job->ref_cnt--;
>> +		spin_unlock_irqrestore(&job->job_lock, flags);
>> +		fc_destroy_bsgjob(job);
>> +	}
>> +
>> +	/* the blk_end_sync_io() doesn't check the error */
>> +	return BLK_EH_HANDLED;
>> +}
>> +
>> +
>> +
>> +static void
>> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
>> +{
>> +	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
>> +
>> +	BUG_ON(!req->nr_phys_segments);
>> +
>> +	buf->sg_list = kzalloc(sz, GFP_KERNEL);
> 
> Needs to handle ENOMEM?
> 
> 
>> +	sg_init_table(buf->sg_list, req->nr_phys_segments);
>> +	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
>> +	buf->payload_len = req->data_len;
>> +}
>> +
>> +
>> +/**
>> + * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for the
>> + *                   bsg request
>> + * @shost:	SCSI Host corresponding to the bsg object
>> + * @rport:	(optional) FC Remote Port corresponding to the bsg object
>> + * @req:	BSG request that needs a job structure
>> + */
>> +static int
>> +fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
>> +	struct request *req)
>> +{
>> +	struct fc_internal *i = to_fc_internal(shost->transportt);
>> +	struct request *rsp = req->next_rq;
>> +	struct fc_bsg_job *job;
>> +
>> +	BUG_ON(req->special);
>> +
>> +	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
>> +			GFP_KERNEL);
>> +	if (!job)
>> +		return -ENOMEM;
>> +
>> +	/*
>> +	 * Note: this is a bit silly.
>> +	 * The request gets formatted as a SGIO v4 ioctl request, which
>> +	 * then gets reformatted as a blk request, which then gets
>> +	 * reformatted as a fc bsg request. And on completion, we have
>> +	 * to wrap return results such that SGIO v4 thinks it was a scsi
>> +	 * status.  I hope this was all worth it.
>> +	 */
>> +
>> +	req->special = job;
>> +	job->shost = shost;
>> +	job->rport = rport;
>> +	job->req = req;
>> +	if (i->f->dd_bsg_size)
>> +		job->dd_data = (void *)&job[1];
>> +	spin_lock_init(&job->job_lock);
>> +	job->request = (struct fc_bsg_request *)req->cmd;
>> +	job->request_len = req->cmd_len;
>> +	job->reply = req->sense;
>> +	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
>> +						 * allocated */
>> +	if (req->bio)
>> +		fc_bsg_map_buffer(&job->request_payload, req);
>> +	if (rsp && rsp->bio)
>> +		fc_bsg_map_buffer(&job->reply_payload, rsp);
>> +	job->job_done = fc_bsg_jobdone;
>> +	if (rport)
>> +		job->dev = &rport->dev;
>> +	else
>> +		job->dev = &shost->shost_gendev;
>> +	get_device(job->dev);		/* take a reference for the request */
>> +
>> +	job->ref_cnt = 1;
>> +
>> +	return 0;
>> +}
>> +
>> +
>> +enum fc_dispatch_result {
>> +	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
>> +	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
>> +	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
>> +};
>> +
>> +
>> +/**
>> + * fc_bsg_host_dispatch - process fc host bsg requests and dispatch to LLDD
>> + * @shost:	scsi host rport attached to
>> + * @job:	bsg job to be processed
>> + */
>> +static enum fc_dispatch_result
>> +fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host *shost,
>> +			 struct fc_bsg_job *job)
>> +{
>> +	struct fc_internal *i = to_fc_internal(shost->transportt);
>> +	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
>> +	int ret;
>> +
>> +	/* Validate the host command */
>> +	switch (job->request->msgcode) {
>> +	case FC_BSG_HST_ADD_RPORT:
>> +		cmdlen += sizeof(struct fc_bsg_host_add_rport);
>> +		break;
>> +
>> +	case FC_BSG_HST_DEL_RPORT:
>> +		cmdlen += sizeof(struct fc_bsg_host_del_rport);
>> +		break;
>> +
>> +	case FC_BSG_HST_ELS_NOLOGIN:
>> +		cmdlen += sizeof(struct fc_bsg_host_els);
>> +		/* there better be a xmt and rcv payloads */
>> +		if ((!job->request_payload.payload_len) ||
>> +		    (!job->reply_payload.payload_len)) {
>> +			ret = -EINVAL;
>> +			goto fail_host_msg;
>> +		}
>> +		break;
>> +
>> +	case FC_BSG_HST_VENDOR:
>> +		cmdlen += sizeof(struct fc_bsg_host_vendor);
>> +		if ((shost->hostt->vendor_id == 0L) || 
>> +		    (job->request->rqst_data.h_vendor.vendor_id !=
>> +			shost->hostt->vendor_id)) {
>> +			ret = -ESRCH;
>> +			goto fail_host_msg;
>> +		}
>> +		break;
>> +
>> +	default:
>> +		ret = -EBADR;
>> +		goto fail_host_msg;
>> +	}
>> +
>> +	/* check if we really have all the request data needed */
>> +	if (job->request_len < cmdlen) {
>> +		ret = -ENOMSG;
>> +		goto fail_host_msg;
>> +	}
>> +
>> +	ret = i->f->bsg_request(job);
>> +	if (ret) {
>> +fail_host_msg:
>> +		/* return the errno failure code as the only status */
>> +		BUG_ON(job->reply_len < sizeof(uint32_t));
>> +		job->reply->result = ret;
>> +		job->reply_len = sizeof(uint32_t);
>> +		fc_bsg_jobdone(job);
>> +		/* fall thru */
>> +	}
>> +
>> +	return FC_DISPATCH_UNLOCKED;
>> +}
>> +
>> +
>> +/*
>> + * fc_bsg_goose_queue - restart rport queue in case it was stopped
>> + * @rport:	rport to be restarted
>> + */
>> +static void
>> +fc_bsg_goose_queue(struct fc_rport *rport)
>> +{
>> +	int flagset;
>> +
>> +	if (!rport->rqst_q)
>> +		return;
>> +
>> +	get_device(&rport->dev);
>> +	
>> +	spin_lock(rport->rqst_q->queue_lock);
>> +	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags) &&
>> +		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
>> +	if (flagset)
>> +		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
>> +	__blk_run_queue(rport->rqst_q);
>> +	if (flagset)
>> +		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
>> +	spin_unlock(rport->rqst_q->queue_lock);
>> +
>> +	put_device(&rport->dev);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
>> + * @shost:	scsi host rport attached to
>> + * @rport:	rport request destined to
>> + * @job:	bsg job to be processed
>> + */
>> +static enum fc_dispatch_result
>> +fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
>> +			 struct fc_rport *rport, struct fc_bsg_job *job)
>> +{
>> +	struct fc_internal *i = to_fc_internal(shost->transportt);
>> +	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
>> +	int ret;
>> +
>> +	/* Validate the rport command */
>> +	switch (job->request->msgcode) {
>> +	case FC_BSG_RPT_ELS:
>> +	case FC_BSG_RPT_CT:
>> +		cmdlen += sizeof(struct fc_bsg_rport_els);
>> +		/* there better be a xmt and rcv payloads */
>> +		if ((!job->request_payload.payload_len) ||
>> +		    (!job->reply_payload.payload_len)) {
>> +			ret = -EINVAL;
>> +			goto fail_rport_msg;
>> +		}
>> +		break;
>> +	default:
>> +		ret = -EBADR;
>> +		goto fail_rport_msg;
>> +	}
>> +
>> +	/* check if we really have all the request data needed */
>> +	if (job->request_len < cmdlen) {
>> +		ret = -ENOMSG;
>> +		goto fail_rport_msg;
>> +	}
>> +
>> +	ret = i->f->bsg_request(job);
>> +	if (ret) {
>> +fail_rport_msg:
>> +		/* return the errno failure code as the only status */
>> +		BUG_ON(job->reply_len < sizeof(uint32_t));
>> +		job->reply->result = ret;
>> +		job->reply_len = sizeof(uint32_t);
>> +		fc_bsg_jobdone(job);
>> +		/* fall thru */
>> +	}
>> +
>> +	return FC_DISPATCH_UNLOCKED;
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_request_handler - generic handler for bsg requests
>> + * @q:		request queue to manage
>> + * @shost:	Scsi_Host related to the bsg object
>> + * @rport:	FC remote port related to the bsg object (optional)
>> + * @dev:	device structure for bsg object
>> + */
>> +static void
>> +fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host *shost,
>> +		       struct fc_rport *rport, struct device *dev)
>> +{
>> +	struct request *req;
>> +	struct fc_bsg_job *job;
>> +	enum fc_dispatch_result ret;
>> +
>> +	if (!get_device(dev))
>> +		return;
>> +
>> +	while (!blk_queue_plugged(q)) {
>> +		req = elv_next_request(q);
>> +		if (!req)
>> +			break;
>> +
>> +		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
>> +				break;
>> +
>> +		blkdev_dequeue_request(req);
>> +
>> +		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
>> +			req->errors = -ENXIO;
>> +			blk_complete_request(req);
> 
> I think that calling blk_complete_request() here crashes your kernel
> since you don't set q->softirq_done_fn.
> 
> Calling blk_end_bidi_request should work here. Note that you need to
> call spin_unlock_irq(q->queue_lock) before it.
> 
> 
>> +			continue;
>> +		}
>> +
>> +		spin_unlock_irq(q->queue_lock);
>> +
>> +		ret = fc_req_to_bsgjob(shost, rport, req);
>> +		if (ret) {
>> +			req->errors = ret;
>> +			spin_lock_irq(q->queue_lock);
>> +			blk_complete_request(req);
> 
> Ditto.
> 

I would suggest an fc_complete_request() defined that would do that.

>> +			continue;
>> +		}
>> +
>> +		job = req->special;
>> +
>> +		/* check if we have the msgcode value at least */
>> +		if (job->request_len < sizeof(uint32_t)) {
>> +			BUG_ON(job->reply_len < sizeof(uint32_t));
>> +			job->reply->result = -ENOMSG;
>> +			job->reply_len = sizeof(uint32_t);
>> +			fc_bsg_jobdone(job);
>> +			spin_lock_irq(q->queue_lock);
>> +			continue;
>> +		}
>> +		
>> +		/* the dispatch routines will unlock the queue_lock */
>> +		if (rport)
>> +			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
>> +		else
>> +			ret = fc_bsg_host_dispatch(q, shost, job);
>> +
>> +		/* did dispatcher hit state that can't process any more */
>> +		if (ret == FC_DISPATCH_BREAK)
>> +			break;
>> +
>> +		/* did dispatcher had released the lock */
>> +		if (ret == FC_DISPATCH_UNLOCKED)
>> +			spin_lock_irq(q->queue_lock);
>> +	}
>> +
>> +	spin_unlock_irq(q->queue_lock);
>> +	put_device(dev);
>> +	spin_lock_irq(q->queue_lock);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_host_handler - handler for bsg requests for a fc host
>> + * @q:		fc host request queue 
>> + */
>> +static void
>> +fc_bsg_host_handler(struct request_queue *q)
>> +{
>> +	struct Scsi_Host *shost = q->queuedata;
>> +
>> +	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_rport_handler - handler for bsg requests for a fc rport
>> + * @q:		rport request queue 
>> + */
>> +static void
>> +fc_bsg_rport_handler(struct request_queue *q)
>> +{
>> +	struct fc_rport *rport = q->queuedata;
>> +	struct Scsi_Host *shost = rport_to_shost(rport);
>> +
>> +	fc_bsg_request_handler(q, shost, rport, &rport->dev);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_hostadd - Create and add the bsg hooks so we can receive requests
>> + * @shost:	shost for fc_host
>> + * @fc_host:	fc_host adding the structures to
>> + */
>> +static int
>> +fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs *fc_host)
>> +{
>> +	struct device *dev = &shost->shost_gendev;
>> +	struct fc_internal *i = to_fc_internal(shost->transportt);
>> +	struct request_queue *q;
>> +	int err;
>> +
>> +	fc_host->rqst_q = NULL;
>> +
>> +	if (!i->f->bsg_request)
>> +		return -ENOTSUPP;
>> +
>> +	snprintf(fc_host->bsg_name, sizeof(fc_host->bsg_name),
>> +		 "fc_host%d", shost->host_no);
> 
> You don't need fc_host->bsg_name. bsg_register_queue() allocates
> memory for the name.
> 
> 
>> +	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
>> +	if (!q) {
>> +		printk(KERN_ERR "fc_host%d: bsg interface failed to "
>> +				"initialize - no request queue\n",
>> +				 shost->host_no);
>> +		return -ENOMEM;
>> +	} 
>> +
>> +	q->queuedata = shost;
>> +	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
>> +	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
>> +	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
>> +
>> +	err = bsg_register_queue(q, dev, fc_host->bsg_name, NULL);
>> +	if (err) {
>> +		printk(KERN_ERR "fc_host%d: bsg interface failed to "
>> +				"initialize - register queue\n",
>> +			 	shost->host_no);
>> +		blk_cleanup_queue(q);
>> +		return err;
>> +	}
>> +
>> +	fc_host->rqst_q = q;
>> +	return 0;
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_rportadd - Create and add the bsg hooks so we can receive requests
>> + * @shost:	shost that rport is attached to
>> + * @rport:	rport that the bsg hooks are being attached to
>> + */
>> +static int
>> +fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
>> +{
>> +	struct device *dev = &rport->dev;
>> +	struct fc_internal *i = to_fc_internal(shost->transportt);
>> +	struct request_queue *q;
>> +	int err;
>> +
>> +	rport->rqst_q = NULL;
>> +
>> +	if (!i->f->bsg_request)
>> +		return -ENOTSUPP;
>> +
>> +	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
>> +	if (!q) {
>> +		printk(KERN_ERR "%s: bsg interface failed to "
>> +				"initialize - no request queue\n",
>> +				 dev->bus_id);
>> +		return -ENOMEM;
>> +	} 
>> +
>> +	q->queuedata = rport;
>> +	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
>> +	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
>> +	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
>> +
>> +	err = bsg_register_queue(q, dev, dev->bus_id, NULL);
> 
> You don't need to pass dev->bus_id. It is used if you pass NULL.
> 
> bsg_register_queue(q, dev, NULL, NULL);
> 
> 
>> +	if (err) {
>> +		printk(KERN_ERR "%s: bsg interface failed to "
>> +				"initialize - register queue\n",
>> +				 dev->bus_id);
>> +		blk_cleanup_queue(q);
>> +		return err;
>> +	}
>> +
>> +	rport->rqst_q = q;
>> +	return 0;
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
>> + * @q:	the request_queue that is to be torn down.
>> + */
>> +static void
>> +fc_bsg_remove(struct request_queue *q)
>> +{
>> +	if (q) {
>> +		bsg_unregister_queue(q);
>> +		blk_cleanup_queue(q);
>> +	}
>> +}
>> +
>> +
>>  /* Original Author:  Martin Hicks */
>>  MODULE_AUTHOR("James Smart");
>>  MODULE_DESCRIPTION("FC Transport Attributes");
>> diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
>> --- a/include/scsi/scsi_bsg_fc.h	1969-12-31 19:00:00.000000000 -0500
>> +++ b/include/scsi/scsi_bsg_fc.h	2008-11-18 15:54:25.000000000 -0500
>> @@ -0,0 +1,291 @@
>> +/*
>> + *  FC Transport BSG Interface
>> + *
>> + *  Copyright (C) 2008   James Smart, Emulex Corporation
>> + *
>> + *  This program is free software; you can redistribute it and/or modify
>> + *  it under the terms of the GNU General Public License as published by
>> + *  the Free Software Foundation; either version 2 of the License, or
>> + *  (at your option) any later version.
>> + *
>> + *  This program is distributed in the hope that it will be useful,
>> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + *  GNU General Public License for more details.
>> + *
>> + *  You should have received a copy of the GNU General Public License
>> + *  along with this program; if not, write to the Free Software
>> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
>> + *
>> + */
>> +
>> +#ifndef SCSI_BSG_FC_H
>> +#define SCSI_BSG_FC_H
>> +
>> +/*
>> + * This file intended to be included by both kernel and user space
>> + */
>> +
>> +#include <scsi/scsi.h>
>> +#include <scsi/sg.h>
> 
> <scsi/sg.h> is necessary?
> 
> 
>> +/*
>> + * FC Transport SGIO v4 BSG Message Support
>> + */
>> +
>> +/* Default BSG request timeout (in seconds) */
>> +#define FC_DEFAULT_BSG_TIMEOUT		10
> 
> Should be (10 * HZ)?
> 
> 
>> +
>> +/*
>> + * Request Message Codes supported by the FC Transport
>> + */
>> +
>> +/* define the class masks for the message codes */
>> +#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
>> +#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
>> +#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
>> +
>> +	/* fc_host Message Codes */
>> +#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
>> +#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
>> +#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
>> +#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
>> +
>> +	/* fc_rport Message Codes */
>> +#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
>> +#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
>> +
>> +
>> +
>> +/*
>> + * FC Address Identifiers in Message Structures :
>> + *
>> + *   Whenever a command payload contains a FC Address Identifier
>> + *   (aka port_id), the value is effectively in big-endian
>> + *   order, thus the array elements are decoded as follows:
>> + *     element [0] is bits 23:16 of the FC Address Identifier
>> + *     element [1] is bits 15:8 of the FC Address Identifier
>> + *     element [2] is bits 7:0 of the FC Address Identifier
>> + */
>> +
>> +
>> +/*
>> + * FC Host Messages
>> + */
>> +
>> +/* FC_BSG_HST_ADDR_PORT : */
>> +
>> +/* Request:
>> + * This message requests the FC host to login to the remote port
>> + * at the specified N_Port_Id.  The remote port is to be enumerated
>> + * with the transport upon completion of the login.
>> + */
>> +struct fc_bsg_host_add_rport {
>> +	uint8_t		reserved;
>> +
>> +	/* FC Address Identier of the remote port to login to */
>> +	uint8_t		port_id[3];
>> +};
>> +
>> +/* Response:
>> + * There is no additional response data - fc_bsg_reply->result is sufficient
>> + */
>> +
>> +
>> +/* FC_BSG_HST_DEL_RPORT : */
>> +
>> +/* Request:
>> + * This message requests the FC host to remove an enumerated
>> + * remote port and to terminate the login to it.
>> + *
>> + * Note: The driver is free to reject this request if it desires to
>> + * remain logged in with the remote port.
>> + */
>> +struct fc_bsg_host_del_rport {
>> +	uint8_t		reserved;
>> +
>> +	/* FC Address Identier of the remote port to logout of */
>> +	uint8_t		port_id[3];
>> +};
>> +
>> +/* Response:
>> + * There is no additional response data - fc_bsg_reply->result is sufficient 
>> + */
>> +
>> +
>> +/* FC_BSG_HST_ELS_NOLOGIN : */
>> +
>> +/* Request:
>> + * This message requests the FC_Host to send an ELS to a specific
>> + * N_Port_ID. The host does not need to log into the remote port,
>> + * nor does it need to enumerate the rport for further traffic
>> + * (although, the FC host is free to do so if it desires).
>> + */
>> +struct fc_bsg_host_els {
>> +	/* 
>> +	 * ELS Command Code being sent (must be the same as byte 0
>> +	 * of the payload)
>> +	 */
>> +	uint8_t 	command_code;
>> +
>> +	/* FC Address Identier of the remote port to send the ELS to */
>> +	uint8_t		port_id[3];
>> +};
>> +
>> +/* Response:
>> + */
>> +/* fc_bsg_ctels_reply->status values */
>> +#define FC_CTELS_STATUS_OK	0x00000000
>> +#define FC_CTELS_STATUS_REJECT	0x00000001
>> +#define FC_CTELS_STATUS_P_RJT	0x00000002
>> +#define FC_CTELS_STATUS_F_RJT	0x00000003
>> +#define FC_CTELS_STATUS_P_BSY	0x00000004
>> +#define FC_CTELS_STATUS_F_BSY	0x00000006
>> +struct fc_bsg_ctels_reply {
>> +	/*
>> +	 * Note: An ELS LS_RJT may be reported in 2 ways:
>> +	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
>> +	 *     is to look into the ELS receive payload to determine
>> +	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
>> +	 *     data will be in word 1.
>> +	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
>> +	 *     rjt_data field will contain valid data.
>> +	 *
>> +	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
>> +	 *   the receive payload word 0 indicates LS_ACC
>> +	 *   (e.g. value is 0x02xxxxxx).
>> +	 *
>> +	 * Note: Similarly, a CT Reject may be reported in 2 ways:
>> +	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
>> +	 *     is to look into the CT receive payload to determine
>> +	 *     Accept or Reject (by contents of word 2). The reject
>> +	 *     data will be in word 3.
>> +	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
>> +	 *     rjt_data field will contain valid data.
>> +	 *
>> +	 * Note: x_RJT/BSY status will indicae that the rjt_data field
>> +	 *   is valid and contains the reason/explanation values.
>> +	 */
>> +	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
>> +
>> +	/* valid if status is not FC_CTELS_STATUS_OK */
>> +	struct	{
>> +		uint8_t	action;		/* fragment_id for CT REJECT */
>> +		uint8_t	reason_code;
>> +		uint8_t	reason_explanation;
>> +		uint8_t	vendor_unique;
>> +	} rjt_data;
>> +};
>> +
>> +
>> +/* FC_BSG_HST_VENDOR : */
>> +
>> +/* Request:
>> + * Note: When specifying vendor_id, be sure to read the Vendor Type and ID
>> + *   formatting requirements specified in scsi_netlink.h
>> + */
>> +struct fc_bsg_host_vendor {
>> +	/*
>> +	 * Identifies the vendor that the message is formatted for. This
>> +	 * should be the recipient of the message.
>> +	 */
>> +	uint64_t vendor_id;
>> +
>> +	/* start of vendor command area */
>> +	uint32_t vendor_cmd[0];
>> +};
>> +
>> +/* Response:
>> + */
>> +struct fc_bsg_host_vendor_reply {
>> +	/* start of vendor response area */
>> +	uint32_t vendor_rsp[0];
>> +};
>> +
>> +
>> +
>> +/*
>> + * FC Remote Port Messages
>> + */
>> +
>> +/* FC_BSG_RPT_ELS : */
>> +
>> +/* Request:
>> + * This message requests that an ELS be performed with the rport.
>> + */
>> +struct fc_bsg_rport_els {
>> +	/* 
>> +	 * ELS Command Code being sent (must be the same as
>> +	 * byte 0 of the payload)
>> +	 */
>> +	uint8_t els_code;
>> +};
>> +
>> +/* Response:
>> + * 
>> + * The reply structure is an fc_bsg_ctels_reply structure
>> + */
>> +
>> +
>> +/* FC_BSG_RPT_CT : */
>> +
>> +/* Request:
>> + * This message requests that a CT Request be performed with the rport.
>> + */
>> +struct fc_bsg_rport_ct {
>> +	/* 
>> +	 * We need words 0-2 of the generic preamble for the LLD's
>> +	 */
>> +	uint32_t	preamble_word0;	/* revision & IN_ID */
>> +	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
>> +	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
>> +};
>> +/* Response:
>> + *
>> + * The reply structure is an fc_bsg_ctels_reply structure
>> + */
>> +
>> +
>> +
>> +
>> +/* request (CDB) structure of the sg_io_v4 */
>> +struct fc_bsg_request {
>> +	uint32_t msgcode;
>> +	union {
>> +		struct fc_bsg_host_add_rport	h_addrport;
>> +		struct fc_bsg_host_del_rport	h_delrport;
>> +		struct fc_bsg_host_els	h_els;
>> +		struct fc_bsg_host_vendor	h_vendor;
>> +
>> +		struct fc_bsg_rport_els		r_els;
>> +		struct fc_bsg_rport_ct		r_ct;
>> +	} rqst_data;
>> +};
>> +
>> +
>> +/* response (request sense data) structure of the sg_io_v4 */
>> +struct fc_bsg_reply {
>> +	/*
>> +	 * The completion result. Result exists in two forms:
>> +	 *  if negative, it is an -Exxx system errno value. There will
>> +	 *    be no further reply information supplied.
>> +	 *  else, it's the 4-byte scsi error result, with driver, host,
>> +	 *    msg and status fields. The per-msgcode reply structure
>> +	 *    will contain valid data.
>> +	 */
>> +	uint32_t result;
>> +
>> +	/* If there was reply_payload, how much was recevied ? */
>> +	uint32_t reply_payload_rcv_len;
>> +
>> +	union {
>> +		struct fc_bsg_host_vendor_reply		vendor_reply;
>> +
>> +		struct fc_bsg_ctels_reply		ctels_reply;
>> +	} reply_data;
>> +};
>> +
>> +
>> +#endif /* SCSI_BSG_FC_H */
>> +
>> diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
>> --- a/include/scsi/scsi_host.h	2008-10-18 10:32:54.000000000 -0400
>> +++ b/include/scsi/scsi_host.h	2008-11-18 15:54:25.000000000 -0500
>> @@ -478,6 +478,15 @@ struct scsi_host_template {
>>  	 * module_init/module_exit.
>>  	 */
>>  	struct list_head legacy_hosts;
>> +
>> +	/*
>> +	 * Vendor Identifier associated with the host
>> +	 *
>> +	 * Note: When specifying vendor_id, be sure to read the
>> +	 *   Vendor Type and ID formatting requirements specified in
>> +	 *   scsi_netlink.h
>> +	 */
>> +	u64 vendor_id;
>>  };
>>  
>>  /*
>> diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h
>> --- a/include/scsi/scsi_transport_fc.h	2008-10-18 10:32:54.000000000 -0400
>> +++ b/include/scsi/scsi_transport_fc.h	2008-11-18 15:54:25.000000000 -0500
>> @@ -33,7 +33,6 @@
>>  
>>  struct scsi_transport_template;
>>  
>> -
>>  /*
>>   * FC Port definitions - Following FC HBAAPI guidelines
>>   *
>> @@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
>>   	struct delayed_work fail_io_work;
>>   	struct work_struct stgt_delete_work;
>>  	struct work_struct rport_delete_work;
>> +	struct request_queue *rqst_q;	/* bsg support */
>>  } __attribute__((aligned(sizeof(unsigned long))));
>>  
>>  /* bit field values for struct fc_rport "flags" field: */
>> @@ -513,6 +513,10 @@ struct fc_host_attrs {
>>  	struct workqueue_struct *work_q;
>>  	char devloss_work_q_name[20];
>>  	struct workqueue_struct *devloss_work_q;
>> +
>> +	/* bsg support */
>> +	char bsg_name[20];
>> +	struct request_queue *rqst_q;
>>  };
>>  
>>  #define shost_to_fc_host(x) \
>> @@ -578,6 +582,47 @@ struct fc_host_attrs {
>>  	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
>>  
>>  
>> +struct fc_bsg_buffer {
>> +	unsigned int payload_len;
>> +	int sg_cnt;
>> +	struct scatterlist *sg_list;
>> +};
>> +
>> +/* Values for fc_bsg_job->state_flags (bitflags) */
>> +#define FC_RQST_STATE_INPROGRESS	0
>> +#define FC_RQST_STATE_DONE		1
>> +
>> +struct fc_bsg_job {
>> +	struct Scsi_Host *shost;
>> +	struct fc_rport *rport;
>> +	struct device *dev;
>> +	struct request *req;
>> +	spinlock_t job_lock;
>> +	unsigned int state_flags;
>> +	unsigned int ref_cnt;
>> +	void (*job_done)(struct fc_bsg_job *);
>> +
>> +	struct fc_bsg_request *request;
>> +	struct fc_bsg_reply *reply;
>> +	unsigned int request_len;
>> +	unsigned int reply_len;
>> +	/*
>> +	 * On entry : reply_len indicates the buffer size allocated for
>> +	 * the reply.
>> +	 *
>> +	 * Upon completion : the message handler must set reply_len
>> +	 *  to indicates the size of the reply to be returned to the
>> +	 *  caller.
>> +	 */
>> +
>> +	/* DMA payloads for the request/response */
>> +	struct fc_bsg_buffer request_payload;
>> +	struct fc_bsg_buffer reply_payload;
>> +
>> +	void *dd_data;			/* Used for driver-specific storage */
>> +};
>> +
>> +
>>  /* The functions by which the transport class and the driver communicate */
>>  struct fc_function_template {
>>  	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
>> @@ -613,9 +658,14 @@ struct fc_function_template {
>>  	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
>>  	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
>>  
>> +	/* bsg support */
>> +	int	(* bsg_request)(struct fc_bsg_job *);
>> +	int	(* bsg_timeout)(struct fc_bsg_job *);
>> +
>>  	/* allocation lengths for host-specific data */
>>  	u32	 			dd_fcrport_size;
>>  	u32	 			dd_fcvport_size;
>> +	u32				dd_bsg_size;
>>  
>>  	/*
>>  	 * The driver sets these to tell the transport class it
>> @@ -736,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
>>  	vport->vport_state = new_state;
>>  }
>>  
>> -
>>  struct scsi_transport_template *fc_attach_transport(
>>  			struct fc_function_template *);
>>  void fc_release_transport(struct scsi_transport_template *);
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] FC pass thru - Rev IV

[FUJITA Tomonori]

On Thu, 27 Nov 2008 10:58:43 +0200
Boaz Harrosh <bharrosh@panasas.com> wrote:

> FUJITA Tomonori wrote:
> > On Tue, 18 Nov 2008 16:24:58 -0500
> > James Smart <James.Smart@Emulex.Com> wrote:
> > 
> >> All,
> >>
> >> I've reworked Seokmann's patch for the following items:
> >> - Add an fchost interface for bsg requests
> >>
> >> - Formalized the request/response structures that I expect
> >>   to have us stuff into the bsg cmd/sense data areas. These
> >>   are now genericized so we can essentially pass any kind of
> >>   transaction. It can be a request that has no transmit or
> >>   receive payload, and simply returns a response.
> >>
> >> - A new file was created, scsi_bsg_fc.h, which contains the
> >>   request/response data structures that should be shared
> >>   between the application and the kernel entities. 
> >>
> >> - I stripped out some things that were in the request
> >>   structure that were actually LLD fields. Instead, I added
> >>   a dd_bsgsize structure to the template, so the transport
> >>   will allocate LLD work space along with the job structure.
> >>   I expect the missing fields to move to this area.
> >>
> >> - I've made a strong attempt at ensuring that the request
> >>   has all the information necessary for the LLD, so that
> >>   there is no need to have the LLD remap the transmit payload
> >>   to figure things out. Granted, this comes at the cost of
> >>   replicating some data items.
> >>
> >>   Sven, I've added the CT information you needed as part of this.
> >>
> >> - I've renamed things quite a bit, hoping to make it clarity
> >>   better. The "service" struct is now a job. I still have
> >>   headaches with "request" (is it the blk request, or the job
> >>   request, or what..)
> >>
> >> - The CT/ELS response is a bit funky. I've noted that the
> >>   way Emulex returns a response, vs Qlogic is a bit different,
> >>   thus the 2 ways to indicate "reject".
> >>
> >> - fixed a couple of bugs in Seokmann's code, in the teardown,
> >>   error flows, request que dma settings, etc.
> >>
> >> - I added a "vendor_id" field to the scsi_host_template to
> >>   use when verifying that the recipient knows how to decode
> >>   vendor-specific message. I didn't do this with the netlink
> >>   things as I was prepping it to not break kabi in existing
> >>   and older kernels. But, I believe this is a good time to
> >>   add it.
> >>
> >> - I've started the Documentation/scsi/scsi_transport_fc.txt
> >>   documentation, but punted finishing it in lieu of sending
> >>   this RFC. I'm starting from Seokman's original emails and
> >>   will be updating for this reformat.
> >>
> >> I'm only starting to debug this, so user beware.
> >>
> >> I could really use some code review from Fujita or Boaz, to
> >> make sure I'm calling the right blk_xx completion functions
> >> relative to the setup flow, and to ensure that the "goose"
> >> when I jump out while the rport is blocked is correct.
> >>
> >> Comments welcome
> >>
> >> -- james s
> >>
> >>
> >>
> >>  Signed-off-by: James Smart <james.smart@emulex.com>
> >>
> >>  ---
> >>
> >>  Documentation/scsi/scsi_fc_transport.txt |   11 
> >>  Documentation/scsi/scsi_mid_low_api.txt  |    5 
> >>  drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
> >>  include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
> >>  include/scsi/scsi_host.h                 |    9 
> >>  include/scsi/scsi_transport_fc.h         |   53 ++
> >>  6 files changed, 946 insertions(+), 4 deletions(-)
> > 
> > (snip)
> > 
> >> +/**
> >> + * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
> >> + *                  completed
> >> + * @job:	fc_bsg_job that is complete
> >> + */
> >> +static void
> >> +fc_bsg_jobdone(struct fc_bsg_job *job)
> >> +{
> >> +	struct request *req = job->req->next_rq;
> >> +	struct request *rsp = req->next_rq;
> >> +	unsigned long flags;
> >> +	int err;
> 
> +	unsigned bytes_requested = 0;
> 
> >> +
> >> +	spin_lock_irqsave(&job->job_lock, flags);
> >> +	job->state_flags |= FC_RQST_STATE_DONE;
> >> +	job->ref_cnt--;
> >> +	spin_unlock_irqrestore(&job->job_lock, flags);
> >> +
> >> +	err = job->req->errors = job->reply->result;
> >> +	if (err < 0)
> >> +		/* we're only returning the result field in the reply */
> >> +		job->req->sense_len = sizeof(uint32_t);
> >> +	else
> >> +		job->req->sense_len = job->reply_len;
> >> +
> >> +	/*
> >> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
> >> +	 * but try to be honest about the amount of rcv data received
> >> +	 */
> >> +	if (rsp)
> >> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> >> +	    			     job->reply->reply_payload_rcv_len);
> >> +	else
> >> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
> > 
> > I think that you can use blk_end_bidi_request() for non-bidi requests:
> > 
> > 	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> > 			rsp ?
> > 			 job->reply->reply_payload_rcv_len : 0);
> > 
> > 
> > I guess that it would be better to have one function to complete a
> > request, instead of blk_end_bidi_request and blk_end_request.
> > 
> > 
> 
> +	/*
> +	 * tell blk layer all of the xmt data was sent.
> +	 * but set residual count to: requested - received
> +	 */
> +
> +	if (rsp) {
> +		bytes_requested = blk_rq_bytes(rsp);
> +		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
> +	}
> +
> +	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
> 
> The residual count is left in req->data_len. Does bsg have a way to return the
> residual to user-mode? It must, since Pete was using that for sure. Note that
> you are looking for the bidi_read residual count.

Yeah, bsg has. struct sg_io_v4 has:

__s32 din_resid;	/* [o] din_xfer_len - actual_din_xfer_len */
__s32 dout_resid;	/* [o] dout_xfer_len - actual_dout_xfer_len */


> As was said by people. You must complete ALL bytes on both sides. Residual information
> is passed through req->data_len. Other wise the request is still active.
> 
> (And yes blk_end_request uses blk_end_bidi_request internally)

We always complete all bytes on both sides. So why we do something
like:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
{
	unsigned int bidi_bytes	= 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = req->next_rq->data_len;

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}

The callers can do something like:

blk_end_request(rq, err, rq->data_len);
rq-->next_rq->data_len = resid;

Re: [RFC] FC pass thru - Rev IV

[Boaz Harrosh]

FUJITA Tomonori wrote:
> On Thu, 27 Nov 2008 10:58:43 +0200
> Boaz Harrosh <bharrosh@panasas.com> wrote:
> 
>> FUJITA Tomonori wrote:
>>> On Tue, 18 Nov 2008 16:24:58 -0500
>>> James Smart <James.Smart@Emulex.Com> wrote:
>>>
>>>> All,
>>>>
>>>> I've reworked Seokmann's patch for the following items:
>>>> - Add an fchost interface for bsg requests
>>>>
>>>> - Formalized the request/response structures that I expect
>>>>   to have us stuff into the bsg cmd/sense data areas. These
>>>>   are now genericized so we can essentially pass any kind of
>>>>   transaction. It can be a request that has no transmit or
>>>>   receive payload, and simply returns a response.
>>>>
>>>> - A new file was created, scsi_bsg_fc.h, which contains the
>>>>   request/response data structures that should be shared
>>>>   between the application and the kernel entities. 
>>>>
>>>> - I stripped out some things that were in the request
>>>>   structure that were actually LLD fields. Instead, I added
>>>>   a dd_bsgsize structure to the template, so the transport
>>>>   will allocate LLD work space along with the job structure.
>>>>   I expect the missing fields to move to this area.
>>>>
>>>> - I've made a strong attempt at ensuring that the request
>>>>   has all the information necessary for the LLD, so that
>>>>   there is no need to have the LLD remap the transmit payload
>>>>   to figure things out. Granted, this comes at the cost of
>>>>   replicating some data items.
>>>>
>>>>   Sven, I've added the CT information you needed as part of this.
>>>>
>>>> - I've renamed things quite a bit, hoping to make it clarity
>>>>   better. The "service" struct is now a job. I still have
>>>>   headaches with "request" (is it the blk request, or the job
>>>>   request, or what..)
>>>>
>>>> - The CT/ELS response is a bit funky. I've noted that the
>>>>   way Emulex returns a response, vs Qlogic is a bit different,
>>>>   thus the 2 ways to indicate "reject".
>>>>
>>>> - fixed a couple of bugs in Seokmann's code, in the teardown,
>>>>   error flows, request que dma settings, etc.
>>>>
>>>> - I added a "vendor_id" field to the scsi_host_template to
>>>>   use when verifying that the recipient knows how to decode
>>>>   vendor-specific message. I didn't do this with the netlink
>>>>   things as I was prepping it to not break kabi in existing
>>>>   and older kernels. But, I believe this is a good time to
>>>>   add it.
>>>>
>>>> - I've started the Documentation/scsi/scsi_transport_fc.txt
>>>>   documentation, but punted finishing it in lieu of sending
>>>>   this RFC. I'm starting from Seokman's original emails and
>>>>   will be updating for this reformat.
>>>>
>>>> I'm only starting to debug this, so user beware.
>>>>
>>>> I could really use some code review from Fujita or Boaz, to
>>>> make sure I'm calling the right blk_xx completion functions
>>>> relative to the setup flow, and to ensure that the "goose"
>>>> when I jump out while the rport is blocked is correct.
>>>>
>>>> Comments welcome
>>>>
>>>> -- james s
>>>>
>>>>
>>>>
>>>>  Signed-off-by: James Smart <james.smart@emulex.com>
>>>>
>>>>  ---
>>>>
>>>>  Documentation/scsi/scsi_fc_transport.txt |   11 
>>>>  Documentation/scsi/scsi_mid_low_api.txt  |    5 
>>>>  drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
>>>>  include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
>>>>  include/scsi/scsi_host.h                 |    9 
>>>>  include/scsi/scsi_transport_fc.h         |   53 ++
>>>>  6 files changed, 946 insertions(+), 4 deletions(-)
>>> (snip)
>>>
>>>> +/**
>>>> + * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
>>>> + *                  completed
>>>> + * @job:	fc_bsg_job that is complete
>>>> + */
>>>> +static void
>>>> +fc_bsg_jobdone(struct fc_bsg_job *job)
>>>> +{
>>>> +	struct request *req = job->req->next_rq;
>>>> +	struct request *rsp = req->next_rq;
>>>> +	unsigned long flags;
>>>> +	int err;
>> +	unsigned bytes_requested = 0;
>>
>>>> +
>>>> +	spin_lock_irqsave(&job->job_lock, flags);
>>>> +	job->state_flags |= FC_RQST_STATE_DONE;
>>>> +	job->ref_cnt--;
>>>> +	spin_unlock_irqrestore(&job->job_lock, flags);
>>>> +
>>>> +	err = job->req->errors = job->reply->result;
>>>> +	if (err < 0)
>>>> +		/* we're only returning the result field in the reply */
>>>> +		job->req->sense_len = sizeof(uint32_t);
>>>> +	else
>>>> +		job->req->sense_len = job->reply_len;
>>>> +
>>>> +	/*
>>>> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
>>>> +	 * but try to be honest about the amount of rcv data received
>>>> +	 */
>>>> +	if (rsp)
>>>> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
>>>> +	    			     job->reply->reply_payload_rcv_len);
>>>> +	else
>>>> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
>>> I think that you can use blk_end_bidi_request() for non-bidi requests:
>>>
>>> 	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
>>> 			rsp ?
>>> 			 job->reply->reply_payload_rcv_len : 0);
>>>
>>>
>>> I guess that it would be better to have one function to complete a
>>> request, instead of blk_end_bidi_request and blk_end_request.
>>>
>>>
>> +	/*
>> +	 * tell blk layer all of the xmt data was sent.
>> +	 * but set residual count to: requested - received
>> +	 */
>> +
>> +	if (rsp) {
>> +		bytes_requested = blk_rq_bytes(rsp);
>> +		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
>> +	}
>> +
>> +	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
>>
>> The residual count is left in req->data_len. Does bsg have a way to return the
>> residual to user-mode? It must, since Pete was using that for sure. Note that
>> you are looking for the bidi_read residual count.
> 
> Yeah, bsg has. struct sg_io_v4 has:
> 
> __s32 din_resid;	/* [o] din_xfer_len - actual_din_xfer_len */
> __s32 dout_resid;	/* [o] dout_xfer_len - actual_dout_xfer_len */
> 
> 
>> As was said by people. You must complete ALL bytes on both sides. Residual information
>> is passed through req->data_len. Other wise the request is still active.
>>
>> (And yes blk_end_request uses blk_end_bidi_request internally)
> 
> We always complete all bytes on both sides. So why we do something
> like:
> 
> int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
> {
> 	unsigned int bidi_bytes	= 0;
> 
> 	if (blk_bidi_rq(rq))
> 		bidi_bytes = req->next_rq->data_len;
> 
> 	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
> }
> 
> The callers can do something like:
> 
> blk_end_request(rq, err, rq->data_len);
> rq-->next_rq->data_len = resid;

Sorry TOMO, I do not understand what you mean. Do you say that we should
change blk_end_request() in blk-core.c ?

In anyway, the code you suggest has a bug you can not use rq-> after call to blk_end_io()
because it might not exist at this point. You must set residual before. And also
you should use  blk_rq_bytes(rq). To see how a request is fully competed see
scsi_end_bidi_request().

Boaz

Re: [RFC] FC pass thru - Rev IV

[FUJITA Tomonori]

On Thu, 27 Nov 2008 13:51:51 +0200
Boaz Harrosh <bharrosh@panasas.com> wrote:

> >>>> +	spin_lock_irqsave(&job->job_lock, flags);
> >>>> +	job->state_flags |= FC_RQST_STATE_DONE;
> >>>> +	job->ref_cnt--;
> >>>> +	spin_unlock_irqrestore(&job->job_lock, flags);
> >>>> +
> >>>> +	err = job->req->errors = job->reply->result;
> >>>> +	if (err < 0)
> >>>> +		/* we're only returning the result field in the reply */
> >>>> +		job->req->sense_len = sizeof(uint32_t);
> >>>> +	else
> >>>> +		job->req->sense_len = job->reply_len;
> >>>> +
> >>>> +	/*
> >>>> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
> >>>> +	 * but try to be honest about the amount of rcv data received
> >>>> +	 */
> >>>> +	if (rsp)
> >>>> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> >>>> +	    			     job->reply->reply_payload_rcv_len);
> >>>> +	else
> >>>> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
> >>> I think that you can use blk_end_bidi_request() for non-bidi requests:
> >>>
> >>> 	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> >>> 			rsp ?
> >>> 			 job->reply->reply_payload_rcv_len : 0);
> >>>
> >>>
> >>> I guess that it would be better to have one function to complete a
> >>> request, instead of blk_end_bidi_request and blk_end_request.
> >>>
> >>>
> >> +	/*
> >> +	 * tell blk layer all of the xmt data was sent.
> >> +	 * but set residual count to: requested - received
> >> +	 */
> >> +
> >> +	if (rsp) {
> >> +		bytes_requested = blk_rq_bytes(rsp);
> >> +		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
> >> +	}
> >> +
> >> +	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
> >>
> >> The residual count is left in req->data_len. Does bsg have a way to return the
> >> residual to user-mode? It must, since Pete was using that for sure. Note that
> >> you are looking for the bidi_read residual count.
> > 
> > Yeah, bsg has. struct sg_io_v4 has:
> > 
> > __s32 din_resid;	/* [o] din_xfer_len - actual_din_xfer_len */
> > __s32 dout_resid;	/* [o] dout_xfer_len - actual_dout_xfer_len */
> > 
> > 
> >> As was said by people. You must complete ALL bytes on both sides. Residual information
> >> is passed through req->data_len. Other wise the request is still active.
> >>
> >> (And yes blk_end_request uses blk_end_bidi_request internally)
> > 
> > We always complete all bytes on both sides. So why we do something
> > like:
> > 
> > int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
> > {
> > 	unsigned int bidi_bytes	= 0;
> > 
> > 	if (blk_bidi_rq(rq))
> > 		bidi_bytes = req->next_rq->data_len;
> > 
> > 	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
> > }
> > 
> > The callers can do something like:
> > 
> > blk_end_request(rq, err, rq->data_len);
> > rq-->next_rq->data_len = resid;
> 
> Sorry TOMO, I do not understand what you mean. Do you say that we should
> change blk_end_request() in blk-core.c ?

Having two kinds of functions (blk_end_request and
blk_end_bidi_request) to complete requests confuse people. As we saw,
developers tend to do something like this:

+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));


The callers don't care about whether a request is bidi or not. It's be
simpler to have a single function to complete a request (whether a
request is bidi or not) rather than having two different functions.

We must complete all bytes on both sides with a bidi request. So why
can't we modify blk_end_request to handle both bidi and non-bidi
requests:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
{
 	unsigned int bidi_bytes	= 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = blk_rq_bytes(rq->next_rq);

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}


> In anyway, the code you suggest has a bug you can not use rq-> after call to blk_end_io()
> because it might not exist at this point. You must set residual before. And also

What is 'rq->' exactly?

We must set residual before calling blk_end_request? Really?

Note that scsi-ml and bsg (blk_execute_rq) work differently. For
scsi-ml, blk_end_io frees request structure (end_that_request_last)
but for blk_execute_rq, it doesn't.


Anyway, it's fine to set bidi_resid before blk_end_request, I
guess. FC pass thru code could do something like this if we modify
blk_end_request in the above way:

/* we calculate bidi_resid here */

if (blk_bidi_rq(req))
	req->next_rq->data_len = bidi_resid;

blk_end_request(req, 0, blk_rq_bytes(req));

Re: [RFC] FC pass thru - Rev IV

[James Bottomley]

On Thu, 2008-11-27 at 18:53 +0900, FUJITA Tomonori wrote:
> > +	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
> > 
> > The residual count is left in req->data_len. Does bsg have a way to return the
> > residual to user-mode? It must, since Pete was using that for sure. Note that
> > you are looking for the bidi_read residual count.
> 
> Yeah, bsg has. struct sg_io_v4 has:
> 
> __s32 din_resid;	/* [o] din_xfer_len - actual_din_xfer_len */
> __s32 dout_resid;	/* [o] dout_xfer_len - actual_dout_xfer_len */
> 
> 
> > As was said by people. You must complete ALL bytes on both sides. Residual information
> > is passed through req->data_len. Other wise the request is still active.
> > 
> > (And yes blk_end_request uses blk_end_bidi_request internally)
> 
> We always complete all bytes on both sides. So why we do something
> like:
> 
> int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
> {
> 	unsigned int bidi_bytes	= 0;
> 
> 	if (blk_bidi_rq(rq))
> 		bidi_bytes = req->next_rq->data_len;
> 
> 	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
> }

That looks good ... care to patch it up (including conversion of all the
users) and send it to Jens for consideration?

> The callers can do something like:
> 
> blk_end_request(rq, err, rq->data_len);
> rq-->next_rq->data_len = resid;

Um ... I see what you're trying to do, but I don't think it works like
this.  The residual could be collected in the end_io call back which is
inline in the blk_end_request().

James

Re: [RFC] FC pass thru - Rev IV

[FUJITA Tomonori]

On Thu, 27 Nov 2008 20:01:41 -0600
James Bottomley <James.Bottomley@HansenPartnership.com> wrote:

> On Thu, 2008-11-27 at 18:53 +0900, FUJITA Tomonori wrote:
> > > +	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
> > > 
> > > The residual count is left in req->data_len. Does bsg have a way to return the
> > > residual to user-mode? It must, since Pete was using that for sure. Note that
> > > you are looking for the bidi_read residual count.
> > 
> > Yeah, bsg has. struct sg_io_v4 has:
> > 
> > __s32 din_resid;	/* [o] din_xfer_len - actual_din_xfer_len */
> > __s32 dout_resid;	/* [o] dout_xfer_len - actual_dout_xfer_len */
> > 
> > 
> > > As was said by people. You must complete ALL bytes on both sides. Residual information
> > > is passed through req->data_len. Other wise the request is still active.
> > > 
> > > (And yes blk_end_request uses blk_end_bidi_request internally)
> > 
> > We always complete all bytes on both sides. So why we do something
> > like:
> > 
> > int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
> > {
> > 	unsigned int bidi_bytes	= 0;
> > 
> > 	if (blk_bidi_rq(rq))
> > 		bidi_bytes = req->next_rq->data_len;
> > 
> > 	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
> > }
> 
> That looks good ... care to patch it up (including conversion of all the
> users) and send it to Jens for consideration?

Sure, I'll do.


> > The callers can do something like:
> > 
> > blk_end_request(rq, err, rq->data_len);
> > rq-->next_rq->data_len = resid;
> 
> Um ... I see what you're trying to do, but I don't think it works like
> this.  The residual could be collected in the end_io call back which is
> inline in the blk_end_request().

Yeah, you are right. bsg doesn't though. But after calling
blk_end_request, bsg can touch (or free) rq any time. So the callers
can't touch rq in the above way after calling blk_end_request (it's
racy).

As Boaz said, we must (and can) set resid before blk_end_request.

Re: [RFC] FC pass thru - Rev IV

[Boaz Harrosh]

FUJITA Tomonori wrote:
> On Thu, 27 Nov 2008 13:51:51 +0200
> Boaz Harrosh <bharrosh@panasas.com> wrote:
> 
>>>>>> +	spin_lock_irqsave(&job->job_lock, flags);
>>>>>> +	job->state_flags |= FC_RQST_STATE_DONE;
>>>>>> +	job->ref_cnt--;
>>>>>> +	spin_unlock_irqrestore(&job->job_lock, flags);
>>>>>> +
>>>>>> +	err = job->req->errors = job->reply->result;
>>>>>> +	if (err < 0)
>>>>>> +		/* we're only returning the result field in the reply */
>>>>>> +		job->req->sense_len = sizeof(uint32_t);
>>>>>> +	else
>>>>>> +		job->req->sense_len = job->reply_len;
>>>>>> +
>>>>>> +	/*
>>>>>> +	 * we'll cheat: tell blk layer all of the xmt data was sent.
>>>>>> +	 * but try to be honest about the amount of rcv data received
>>>>>> +	 */
>>>>>> +	if (rsp)
>>>>>> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
>>>>>> +	    			     job->reply->reply_payload_rcv_len);
>>>>>> +	else
>>>>>> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
>>>>> I think that you can use blk_end_bidi_request() for non-bidi requests:
>>>>>
>>>>> 	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
>>>>> 			rsp ?
>>>>> 			 job->reply->reply_payload_rcv_len : 0);
>>>>>
>>>>>
>>>>> I guess that it would be better to have one function to complete a
>>>>> request, instead of blk_end_bidi_request and blk_end_request.
>>>>>
>>>>>
>>>> +	/*
>>>> +	 * tell blk layer all of the xmt data was sent.
>>>> +	 * but set residual count to: requested - received
>>>> +	 */
>>>> +
>>>> +	if (rsp) {
>>>> +		bytes_requested = blk_rq_bytes(rsp);
>>>> +		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
>>>> +	}
>>>> +
>>>> +	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
>>>>
>>>> The residual count is left in req->data_len. Does bsg have a way to return the
>>>> residual to user-mode? It must, since Pete was using that for sure. Note that
>>>> you are looking for the bidi_read residual count.
>>> Yeah, bsg has. struct sg_io_v4 has:
>>>
>>> __s32 din_resid;	/* [o] din_xfer_len - actual_din_xfer_len */
>>> __s32 dout_resid;	/* [o] dout_xfer_len - actual_dout_xfer_len */
>>>
>>>
>>>> As was said by people. You must complete ALL bytes on both sides. Residual information
>>>> is passed through req->data_len. Other wise the request is still active.
>>>>
>>>> (And yes blk_end_request uses blk_end_bidi_request internally)
>>> We always complete all bytes on both sides. So why we do something
>>> like:
>>>
>>> int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
>>> {
>>> 	unsigned int bidi_bytes	= 0;
>>>
>>> 	if (blk_bidi_rq(rq))
>>> 		bidi_bytes = req->next_rq->data_len;
>>>
>>> 	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
>>> }
>>>
>>> The callers can do something like:
>>>
>>> blk_end_request(rq, err, rq->data_len);
>>> rq-->next_rq->data_len = resid;
>> Sorry TOMO, I do not understand what you mean. Do you say that we should
>> change blk_end_request() in blk-core.c ?
> 
> Having two kinds of functions (blk_end_request and
> blk_end_bidi_request) to complete requests confuse people. As we saw,
> developers tend to do something like this:
> 
> +	if (rsp)
> +		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
> +	    			     job->reply->reply_payload_rcv_len);
> +	else
> +		blk_end_request(job->req, err, blk_rq_bytes(job->req));
> 
> 
> The callers don't care about whether a request is bidi or not. It's be
> simpler to have a single function to complete a request (whether a
> request is bidi or not) rather than having two different functions.
> 
> We must complete all bytes on both sides with a bidi request. So why
> can't we modify blk_end_request to handle both bidi and non-bidi
> requests:
> 
> int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)
> {
>  	unsigned int bidi_bytes	= 0;
> 
> 	if (blk_bidi_rq(rq))
> 		bidi_bytes = blk_rq_bytes(rq->next_rq);
> 
> 	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
> }
> 
> 
>> In anyway, the code you suggest has a bug you can not use rq-> after call to blk_end_io()
>> because it might not exist at this point. You must set residual before. And also
> 
> What is 'rq->' exactly?
> 
> We must set residual before calling blk_end_request? Really?
> 
> Note that scsi-ml and bsg (blk_execute_rq) work differently. For
> scsi-ml, blk_end_io frees request structure (end_that_request_last)
> but for blk_execute_rq, it doesn't.
> 

It does not matter if bsg or any other user has an extra reference
on the request (so end_that_request_last does not deallocate the
request). The end_io function is called from within the end_that_request_last
so setting the residual into req->data_len will be to late.

> 
> Anyway, it's fine to set bidi_resid before blk_end_request, I
> guess. FC pass thru code could do something like this if we modify
> blk_end_request in the above way:
> 
> /* we calculate bidi_resid here */
> 
> if (blk_bidi_rq(req))
> 	req->next_rq->data_len = bidi_resid;
> 

No that will not work. You lost the req->next_rq->data_len byte-count
and blk_end_request() will not be able to complete all bytes of req->next_rq.

Please see scsi_end_bidi_request() for the only way to complete a bidi
request with returned residual count on both sides.

> blk_end_request(req, 0, blk_rq_bytes(req));

The way blk_end_request() is now it cannot complete bidi requests.
because of residual count missing. I have shown above the only way
that you can complete both bidi or uni request with a single call to
blk_end_bidi_request(). If you want people not to get confused it is
blk_end_request() that should be dropped.

Boaz

Re: [RFC] FC pass thru - Rev IV

[Seokmann Ju]

Hi James,

On Nov 25, 2008, at 6:38 AM, Seokmann Ju wrote:
>
> On Nov 24, 2008, at 1:03 PM, James Smart wrote:
[snip]
>>
>>> 2. Would  it be valuable to consider sending requests with
>>>   WWNN:WWPN?
>>
>> I considered this, especially for the fchost ELS request. But it  
>> means the LLDD has to do a nameserver query to find the n_port_id
>> corresponding to the names before it sent the ELS. It started to feel
>> like too much ancillary stuff for the LLDD to do to support the
>> request - so I punted.  However, the app, where we typically don't
>> care if it has to do more work, is free to do this and resolve it to
>> the basic primitives. :)
> Thanks, I've got it.
One minor questions/clarification.
If I understood correctly, ELS only supports single sg. If this is  
true, shouldn't it checked somewhere before the transport layer calls  
LLD?

Thank you,
Seokmann

Re: [RFC] FC pass thru - Rev IV

[James Smart]

Seokmann Ju wrote:
> One minor questions/clarification.
> If I understood correctly, ELS only supports single sg. If this is
> true, shouldn't it checked somewhere before the transport layer calls
> LLD?

I didn't think we needed to mandate this. So, I've left it in the 
position where the transport allows it to be more than one, but the 
driver is free to reject it if it desires.  That said, I wouldn't expect 
an ELS to be more than 1 buffer.  If everyone feels strongly enough 
about it - we can make this restriction.

However, we can't do this on CT operations.

-- james

[RFC] FC pass thru - Rev V

[James Smart]

Trying to kick-start this again...  
I've updated the prior RFC with the comments from Seokmann,
SvenFujita, and Boaz. I would still like review on the
blk_xxx completion calls in the std and error paths.

It currently expects that blk_end_request() has been updated
by Fujita's patch to incorporate blk_end_bidi_request()
functionality :
http://marc.info/?l-linux-scsi&m=122785157116659&w=2

-- james s

======

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect
  to have us stuff into the bsg cmd/sense data areas. These
  are now genericized so we can essentially pass any kind of
  transaction. It can be a request that has no transmit or
  receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the
  request/response data structures that should be shared
  between the application and the kernel entities. 

- I stripped out some things that were in the request
  structure that were actually LLD fields. Instead, I added
  a dd_bsgsize structure to the template, so the transport
  will allocate LLD work space along with the job structure.
  I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request
  has all the information necessary for the LLD, so that
  there is no need to have the LLD remap the transmit payload
  to figure things out. Granted, this comes at the cost of
  replicating some data items.

  Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity
  better. The "service" struct is now a job. I still have
  headaches with "request" (is it the blk request, or the job
  request, or what..)

- The CT/ELS response is a bit funky. I've noted that the
  way Emulex returns a response, vs Qlogic is a bit different,
  thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown,
  error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to
  use when verifying that the recipient knows how to decode
  vendor-specific message. I didn't do this with the netlink
  things as I was prepping it to not break kabi in existing
  and older kernels. But, I believe this is a good time to
  add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt
  documentation, but punted finishing it in lieu of sending
  this RFC. I'm starting from Seokman's original emails and
  will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to
make sure I'm calling the right blk_xx completion functions
relative to the setup flow, and to ensure that the "goose"
when I jump out while the rport is blocked is correct.

Comments welcome

-- james s





 Signed-off-by: James Smart <james.smart@emulex.com>

 ---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  614 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   52 ++
 6 files changed, 1009 insertions(+), 4 deletions(-)


diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt
--- a/Documentation/scsi/scsi_fc_transport.txt	2009-01-27 09:44:22.000000000 -0500
+++ b/Documentation/scsi/scsi_fc_transport.txt	2009-02-10 10:34:42.000000000 -0500
@@ -1,10 +1,11 @@
                              SCSI FC Tansport
                  =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008
 Kernel Revisions for features:
   rports : <<TBS>>
   vports : 2.6.22 (? TBD)
+  bsg support : 2.6.29 (?TBD)
 
 
 Introduction
@@ -15,6 +16,7 @@ The FC transport can be found at:
   drivers/scsi/scsi_transport_fc.c
   include/scsi/scsi_transport_fc.h
   include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 
 This file is found at Documentation/scsi/scsi_fc_transport.txt
 
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+========================================================================
+<< To Be Supplied >>
+
+
+
+
 Credits
 =======
 The following people have contributed to this document:
diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
--- a/Documentation/scsi/scsi_mid_low_api.txt	2008-09-23 15:11:57.000000000 -0400
+++ b/Documentation/scsi/scsi_mid_low_api.txt	2009-02-10 10:34:42.000000000 -0500
@@ -1271,6 +1271,11 @@ of interest:
     hostdata[0]  - area reserved for LLD at end of struct Scsi_Host. Size
                    is set by the second argument (named 'xtr_bytes') to
                    scsi_host_alloc() or scsi_register().
+    vendor_id    - a unique value that identifies the vendor supplying
+                   the LLD for the Scsi_Host.  Used most often in validating
+                   vendor-specific message requests.  Value consists of an
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid formats.
 
 The scsi_host structure is defined in include/scsi/scsi_host.h
 
diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c
--- a/drivers/scsi/scsi_transport_fc.c	2009-01-27 09:44:31.000000000 -0500
+++ b/drivers/scsi/scsi_transport_fc.c	2009-02-10 14:58:39.000000000 -0500
@@ -35,6 +35,7 @@
 #include <linux/netlink.h>
 #include <net/netlink.h>
 #include <scsi/scsi_netlink_fc.h>
+#include <scsi/scsi_bsg_fc.h>
 #include "scsi_priv.h"
 #include "scsi_transport_fc_internal.h"
 
@@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
 static int fc_vport_setup(struct Scsi_Host *shost, int channel,
 	struct device *pdev, struct fc_vport_identifiers  *ids,
 	struct fc_vport **vport);
+static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs *);
+static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
+static void fc_bsg_remove(struct request_queue *);
+static void fc_bsg_goose_queue(struct fc_rport *);
 
 /*
  * Redefine so that we can have same named attributes in the
@@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
 		return -ENOMEM;
 	}
 
+	fc_bsg_hostadd(shost, fc_host);
+	/* ignore any bsg add error - we just can't do sgio */
+
+	return 0;
+}
+
+static int fc_host_remove(struct transport_container *tc, struct device *dev,
+			 struct device *cdev)
+{
+	struct Scsi_Host *shost = dev_to_shost(dev);
+	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
+
+	fc_bsg_remove(fc_host->rqst_q);
 	return 0;
 }
 
 static DECLARE_TRANSPORT_CLASS(fc_host_class,
 			       "fc_host",
 			       fc_host_setup,
-			       NULL,
+			       fc_host_remove,
 			       NULL);
 
 /*
@@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
 		scsi_flush_work(shost);
 
 	fc_terminate_rport_io(rport);
+
 	/*
 	 * Cancel any outstanding timers. These should really exist
 	 * only when rmmod'ing the LLDD and we're asking for
@@ -2415,6 +2434,8 @@ fc_rport_final_delete(struct work_struct
 	    (i->f->dev_loss_tmo_callbk))
 		i->f->dev_loss_tmo_callbk(rport);
 
+	fc_bsg_remove(rport->rqst_q);
+
 	transport_remove_device(dev);
 	device_del(dev);
 	transport_destroy_device(dev);
@@ -2502,6 +2523,9 @@ fc_rport_create(struct Scsi_Host *shost,
 	transport_add_device(dev);
 	transport_configure_device(dev);
 
+	fc_bsg_rportadd(shost, rport);
+	/* ignore any bsg add error - we just can't do sgio */
+
 	if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
 		/* initiate a scan of the target */
 		rport->flags |= FC_RPORT_SCAN_PENDING;
@@ -2666,6 +2690,8 @@ fc_remote_port_add(struct Scsi_Host *sho
 					spin_unlock_irqrestore(shost->host_lock,
 							flags);
 
+				fc_bsg_goose_queue(rport);
+
 				return rport;
 			}
 		}
@@ -3351,6 +3377,592 @@ fc_vport_sched_delete(struct work_struct
 }
 
 
+/*
+ * BSG support
+ */
+
+
+/**
+ * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
+ * @job:	fc_bsg_job that is to be torn down
+ */
+static void
+fc_destroy_bsgjob(struct fc_bsg_job *job)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->ref_cnt) {
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		return;
+	}
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	put_device(job->dev);	/* release reference for the request */
+
+	kfree(job->request_payload.sg_list);
+	kfree(job->reply_payload.sg_list);
+	kfree(job);
+}
+
+
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	unsigned rsp_len;
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;
+
+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	/* we assume all request payload was transferred */
+	blk_end_request(req, err, blk_rq_bytes(req));
+
+	fc_destroy_bsgjob(job);
+}
+
+
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:	request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+	struct fc_bsg_job *job = (void *) req->special;
+	struct Scsi_Host *shost = job->shost;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	unsigned long flags;
+	int err = 0, done = 0;
+
+	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
+		return BLK_EH_RESET_TIMER;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->state_flags & FC_RQST_STATE_DONE)
+		done = 1;
+	else
+		job->ref_cnt++;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	if (!done && i->f->bsg_timeout) {
+		/* call LLDD to abort the i/o as it has timed out */
+		err = i->f->bsg_timeout(job);
+		if (err)
+			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
+				"abort failed with status %d\n", err);
+	}
+
+	if (!done) {
+		spin_lock_irqsave(&job->job_lock, flags);
+		job->ref_cnt--;
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		fc_destroy_bsgjob(job);
+	}
+
+	/* the blk_end_sync_io() doesn't check the error */
+	return BLK_EH_HANDLED;
+}
+
+
+
+static int
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);
+	if (!buf->sg_list)
+		return -ENOMEM;
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+	return 0;
+}
+
+
+/**
+ * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for the
+ *                   bsg request
+ * @shost:	SCSI Host corresponding to the bsg object
+ * @rport:	(optional) FC Remote Port corresponding to the bsg object
+ * @req:	BSG request that needs a job structure
+ */
+static int
+fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
+	struct request *req)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request *rsp = req->next_rq;
+	struct fc_bsg_job *job;
+	int ret;
+
+	BUG_ON(req->special);
+
+	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
+			GFP_KERNEL);
+	if (!job)
+		return -ENOMEM;
+
+	/*
+	 * Note: this is a bit silly.
+	 * The request gets formatted as a SGIO v4 ioctl request, which
+	 * then gets reformatted as a blk request, which then gets
+	 * reformatted as a fc bsg request. And on completion, we have
+	 * to wrap return results such that SGIO v4 thinks it was a scsi
+	 * status.  I hope this was all worth it.
+	 */
+
+	req->special = job;
+	job->shost = shost;
+	job->rport = rport;
+	job->req = req;
+	if (i->f->dd_bsg_size)
+		job->dd_data = (void *)&job[1];
+	spin_lock_init(&job->job_lock);
+	job->request = (struct fc_bsg_request *)req->cmd;
+	job->request_len = req->cmd_len;
+	job->reply = req->sense;
+	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
+						 * allocated */
+	if (req->bio) {
+		ret = fc_bsg_map_buffer(&job->request_payload, req);
+		if (ret)
+			goto failjob_rls_job;
+	}
+	if (rsp && rsp->bio) {
+		ret = fc_bsg_map_buffer(&job->reply_payload, rsp);
+		if (ret)
+			goto failjob_rls_rqst_payload;
+	}
+	job->job_done = fc_bsg_jobdone;
+	if (rport)
+		job->dev = &rport->dev;
+	else
+		job->dev = &shost->shost_gendev;
+	get_device(job->dev);		/* take a reference for the request */
+
+	job->ref_cnt = 1;
+
+	return 0;
+
+
+failjob_rls_rqst_payload:
+	kfree(job->request_payload.sg_list);
+failjob_rls_job:
+	kfree(job);
+	return -ENOMEM;
+}
+
+
+enum fc_dispatch_result {
+	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
+	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
+	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
+};
+
+
+/**
+ * fc_bsg_host_dispatch - process fc host bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the host command */
+	switch (job->request->msgcode) {
+	case FC_BSG_HST_ADD_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_add_rport);
+		break;
+
+	case FC_BSG_HST_DEL_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_del_rport);
+		break;
+
+	case FC_BSG_HST_ELS_NOLOGIN:
+		cmdlen += sizeof(struct fc_bsg_host_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_host_msg;
+		}
+		break;
+
+	case FC_BSG_HST_CT:
+		cmdlen += sizeof(struct fc_bsg_host_ct);
+		/* there better be xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_host_msg;
+		}
+		break;
+
+	case FC_BSG_HST_VENDOR:
+		cmdlen += sizeof(struct fc_bsg_host_vendor);
+		if ((shost->hostt->vendor_id == 0L) ||
+		    (job->request->rqst_data.h_vendor.vendor_id !=
+			shost->hostt->vendor_id)) {
+			ret = -ESRCH;
+			goto fail_host_msg;
+		}
+		break;
+
+	default:
+		ret = -EBADR;
+		goto fail_host_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_host_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (!ret)
+		return FC_DISPATCH_UNLOCKED;
+
+fail_host_msg:
+	/* return the errno failure code as the only status */
+	BUG_ON(job->reply_len < sizeof(uint32_t));
+	job->reply->result = ret;
+	job->reply_len = sizeof(uint32_t);
+	fc_bsg_jobdone(job);
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/*
+ * fc_bsg_goose_queue - restart rport queue in case it was stopped
+ * @rport:	rport to be restarted
+ */
+static void
+fc_bsg_goose_queue(struct fc_rport *rport)
+{
+	int flagset;
+
+	if (!rport->rqst_q)
+		return;
+
+	get_device(&rport->dev);
+
+	spin_lock(rport->rqst_q->queue_lock);
+	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags) &&
+		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
+	if (flagset)
+		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
+	__blk_run_queue(rport->rqst_q);
+	if (flagset)
+		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
+	spin_unlock(rport->rqst_q->queue_lock);
+
+	put_device(&rport->dev);
+}
+
+
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @rport:	rport request destined to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_rport *rport, struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the rport command */
+	switch (job->request->msgcode) {
+	case FC_BSG_RPT_ELS:
+		cmdlen += sizeof(struct fc_bsg_rport_els);
+		goto check_bidi;
+
+	case FC_BSG_RPT_CT:
+		cmdlen += sizeof(struct fc_bsg_rport_ct);
+check_bidi:
+		/* there better be xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_rport_msg;
+		}
+		break;
+	default:
+		ret = -EBADR;
+		goto fail_rport_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_rport_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (!ret)
+		return FC_DISPATCH_UNLOCKED;
+
+fail_rport_msg:
+	/* return the errno failure code as the only status */
+	BUG_ON(job->reply_len < sizeof(uint32_t));
+	job->reply->result = ret;
+	job->reply_len = sizeof(uint32_t);
+	fc_bsg_jobdone(job);
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:		request queue to manage
+ * @shost:	Scsi_Host related to the bsg object
+ * @rport:	FC remote port related to the bsg object (optional)
+ * @dev:	device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host *shost,
+		       struct fc_rport *rport, struct device *dev)
+{
+	struct request *req;
+	struct fc_bsg_job *job;
+	enum fc_dispatch_result ret;
+
+	if (!get_device(dev))
+		return;
+
+	while (!blk_queue_plugged(q)) {
+		req = elv_next_request(q);
+		if (!req)
+			break;
+
+		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
+				break;
+
+		blkdev_dequeue_request(req);
+
+		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
+			req->errors = -ENXIO;
+			spin_unlock_irq(q->queue_lock);
+			blk_end_request(req, -ENXIO, blk_rq_bytes(req));
+			spin_lock_irq(q->queue_lock);
+			continue;
+		}
+
+		spin_unlock_irq(q->queue_lock);
+
+		ret = fc_req_to_bsgjob(shost, rport, req);
+		if (ret) {
+			req->errors = ret;
+			blk_end_request(req, ret, blk_rq_bytes(req));
+			spin_lock_irq(q->queue_lock);
+			continue;
+		}
+
+		job = req->special;
+
+		/* check if we have the msgcode value at least */
+		if (job->request_len < sizeof(uint32_t)) {
+			BUG_ON(job->reply_len < sizeof(uint32_t));
+			job->reply->result = -ENOMSG;
+			job->reply_len = sizeof(uint32_t);
+			fc_bsg_jobdone(job);
+			spin_lock_irq(q->queue_lock);
+			continue;
+		}
+
+		/* the dispatch routines will unlock the queue_lock */
+		if (rport)
+			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
+		else
+			ret = fc_bsg_host_dispatch(q, shost, job);
+
+		/* did dispatcher hit state that can't process any more */
+		if (ret == FC_DISPATCH_BREAK)
+			break;
+
+		/* did dispatcher had released the lock */
+		if (ret == FC_DISPATCH_UNLOCKED)
+			spin_lock_irq(q->queue_lock);
+	}
+
+	spin_unlock_irq(q->queue_lock);
+	put_device(dev);
+	spin_lock_irq(q->queue_lock);
+}
+
+
+/**
+ * fc_bsg_host_handler - handler for bsg requests for a fc host
+ * @q:		fc host request queue
+ */
+static void
+fc_bsg_host_handler(struct request_queue *q)
+{
+	struct Scsi_Host *shost = q->queuedata;
+
+	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
+}
+
+
+/**
+ * fc_bsg_rport_handler - handler for bsg requests for a fc rport
+ * @q:		rport request queue
+ */
+static void
+fc_bsg_rport_handler(struct request_queue *q)
+{
+	struct fc_rport *rport = q->queuedata;
+	struct Scsi_Host *shost = rport_to_shost(rport);
+
+	fc_bsg_request_handler(q, shost, rport, &rport->dev);
+}
+
+
+/**
+ * fc_bsg_hostadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost for fc_host
+ * @fc_host:	fc_host adding the structures to
+ */
+static int
+fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs *fc_host)
+{
+	struct device *dev = &shost->shost_gendev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+	char bsg_name[BUS_ID_SIZE]; /*20*/
+
+	fc_host->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	snprintf(bsg_name, sizeof(bsg_name),
+		 "fc_host%d", shost->host_no);
+
+	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
+	if (!q) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - no request queue\n",
+				 shost->host_no);
+		return -ENOMEM;
+	}
+
+	q->queuedata = shost;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, bsg_name, NULL);
+	if (err) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - register queue\n",
+				shost->host_no);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	fc_host->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_rportadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost that rport is attached to
+ * @rport:	rport that the bsg hooks are being attached to
+ */
+static int
+fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
+{
+	struct device *dev = &rport->dev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+
+	rport->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
+	if (!q) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - no request queue\n",
+				 dev->bus_id);
+		return -ENOMEM;
+	}
+
+	q->queuedata = rport;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, NULL, NULL);
+	if (err) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - register queue\n",
+				 dev->bus_id);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	rport->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
+ * @q:	the request_queue that is to be torn down.
+ */
+static void
+fc_bsg_remove(struct request_queue *q)
+{
+	if (q) {
+		bsg_unregister_queue(q);
+		blk_cleanup_queue(q);
+	}
+}
+
+
 /* Original Author:  Martin Hicks */
 MODULE_AUTHOR("James Smart");
 MODULE_DESCRIPTION("FC Transport Attributes");
diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
--- a/include/scsi/scsi_bsg_fc.h	1969-12-31 19:00:00.000000000 -0500
+++ b/include/scsi/scsi_bsg_fc.h	2009-02-10 13:26:17.000000000 -0500
@@ -0,0 +1,322 @@
+/*
+ *  FC Transport BSG Interface
+ *
+ *  Copyright (C) 2008   James Smart, Emulex Corporation
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ *
+ */
+
+#ifndef SCSI_BSG_FC_H
+#define SCSI_BSG_FC_H
+
+/*
+ * This file intended to be included by both kernel and user space
+ */
+
+#include <scsi/scsi.h>
+
+/*
+ * FC Transport SGIO v4 BSG Message Support
+ */
+
+/* Default BSG request timeout (in seconds) */
+#define FC_DEFAULT_BSG_TIMEOUT		(10 * HZ)
+
+
+/*
+ * Request Message Codes supported by the FC Transport
+ */
+
+/* define the class masks for the message codes */
+#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
+#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
+#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
+
+	/* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
+#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
+#define FC_BSG_HST_CT			(FC_BSG_HST_MASK | 0x00000004)
+#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
+
+	/* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
+#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
+
+
+
+/*
+ * FC Address Identifiers in Message Structures :
+ *
+ *   Whenever a command payload contains a FC Address Identifier
+ *   (aka port_id), the value is effectively in big-endian
+ *   order, thus the array elements are decoded as follows:
+ *     element [0] is bits 23:16 of the FC Address Identifier
+ *     element [1] is bits 15:8 of the FC Address Identifier
+ *     element [2] is bits 7:0 of the FC Address Identifier
+ */
+
+
+/*
+ * FC Host Messages
+ */
+
+/* FC_BSG_HST_ADDR_PORT : */
+
+/* Request:
+ * This message requests the FC host to login to the remote port
+ * at the specified N_Port_Id.  The remote port is to be enumerated
+ * with the transport upon completion of the login.
+ */
+struct fc_bsg_host_add_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to login to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient
+ */
+
+
+/* FC_BSG_HST_DEL_RPORT : */
+
+/* Request:
+ * This message requests the FC host to remove an enumerated
+ * remote port and to terminate the login to it.
+ *
+ * Note: The driver is free to reject this request if it desires to
+ * remain logged in with the remote port.
+ */
+struct fc_bsg_host_del_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to logout of */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient
+ */
+
+
+/* FC_BSG_HST_ELS_NOLOGIN : */
+
+/* Request:
+ * This message requests the FC_Host to send an ELS to a specific
+ * N_Port_ID. The host does not need to log into the remote port,
+ * nor does it need to enumerate the rport for further traffic
+ * (although, the FC host is free to do so if it desires).
+ */
+struct fc_bsg_host_els {
+	/*
+	 * ELS Command Code being sent (must be the same as byte 0
+	 * of the payload)
+	 */
+	uint8_t 	command_code;
+
+	/* FC Address Identier of the remote port to send the ELS to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ */
+/* fc_bsg_ctels_reply->status values */
+#define FC_CTELS_STATUS_OK	0x00000000
+#define FC_CTELS_STATUS_REJECT	0x00000001
+#define FC_CTELS_STATUS_P_RJT	0x00000002
+#define FC_CTELS_STATUS_F_RJT	0x00000003
+#define FC_CTELS_STATUS_P_BSY	0x00000004
+#define FC_CTELS_STATUS_F_BSY	0x00000006
+struct fc_bsg_ctels_reply {
+	/*
+	 * Note: An ELS LS_RJT may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the ELS receive payload to determine
+	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
+	 *     data will be in word 1.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
+	 *   the receive payload word 0 indicates LS_ACC
+	 *   (e.g. value is 0x02xxxxxx).
+	 *
+	 * Note: Similarly, a CT Reject may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the CT receive payload to determine
+	 *     Accept or Reject (by contents of word 2). The reject
+	 *     data will be in word 3.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: x_RJT/BSY status will indicae that the rjt_data field
+	 *   is valid and contains the reason/explanation values.
+	 */
+	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
+
+	/* valid if status is not FC_CTELS_STATUS_OK */
+	struct	{
+		uint8_t	action;		/* fragment_id for CT REJECT */
+		uint8_t	reason_code;
+		uint8_t	reason_explanation;
+		uint8_t	vendor_unique;
+	} rjt_data;
+};
+
+
+/* FC_BSG_HST_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the
+ * indicated N_Port_ID. The driver is responsible for logging in with
+ * the fabric and/or N_Port_ID, etc as per FC rules. This request does
+ * not mandate that the driver must enumerate the destination in the
+ * transport. The driver is allowed to decide whether to enumerate it,
+ * and whether to tear it down after the request.
+ */
+struct fc_bsg_host_ct {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to send the ELS to */
+	uint8_t		port_id[3];
+
+	/*
+	 * We need words 0-2 of the generic preamble for the LLD's
+	 */
+	uint32_t	preamble_word0;	/* revision & IN_ID */
+	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
+	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
+
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_HST_VENDOR : */
+
+/* Request:
+ * Note: When specifying vendor_id, be sure to read the Vendor Type and ID
+ *   formatting requirements specified in scsi_netlink.h
+ */
+struct fc_bsg_host_vendor {
+	/*
+	 * Identifies the vendor that the message is formatted for. This
+	 * should be the recipient of the message.
+	 */
+	uint64_t vendor_id;
+
+	/* start of vendor command area */
+	uint32_t vendor_cmd[0];
+};
+
+/* Response:
+ */
+struct fc_bsg_host_vendor_reply {
+	/* start of vendor response area */
+	uint32_t vendor_rsp[0];
+};
+
+
+
+/*
+ * FC Remote Port Messages
+ */
+
+/* FC_BSG_RPT_ELS : */
+
+/* Request:
+ * This message requests that an ELS be performed with the rport.
+ */
+struct fc_bsg_rport_els {
+	/*
+	 * ELS Command Code being sent (must be the same as
+	 * byte 0 of the payload)
+	 */
+	uint8_t els_code;
+};
+
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_RPT_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the rport.
+ */
+struct fc_bsg_rport_ct {
+	/*
+	 * We need words 0-2 of the generic preamble for the LLD's
+	 */
+	uint32_t	preamble_word0;	/* revision & IN_ID */
+	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
+	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+
+
+/* request (CDB) structure of the sg_io_v4 */
+struct fc_bsg_request {
+	uint32_t msgcode;
+	union {
+		struct fc_bsg_host_add_rport	h_addrport;
+		struct fc_bsg_host_del_rport	h_delrport;
+		struct fc_bsg_host_els		h_els;
+		struct fc_bsg_host_ct		h_ct;
+		struct fc_bsg_host_vendor	h_vendor;
+
+		struct fc_bsg_rport_els		r_els;
+		struct fc_bsg_rport_ct		r_ct;
+	} rqst_data;
+};
+
+
+/* response (request sense data) structure of the sg_io_v4 */
+struct fc_bsg_reply {
+	/*
+	 * The completion result. Result exists in two forms:
+	 *  if negative, it is an -Exxx system errno value. There will
+	 *    be no further reply information supplied.
+	 *  else, it's the 4-byte scsi error result, with driver, host,
+	 *    msg and status fields. The per-msgcode reply structure
+	 *    will contain valid data.
+	 */
+	uint32_t result;
+
+	/* If there was reply_payload, how much was recevied ? */
+	uint32_t reply_payload_rcv_len;
+
+	union {
+		struct fc_bsg_host_vendor_reply		vendor_reply;
+
+		struct fc_bsg_ctels_reply		ctels_reply;
+	} reply_data;
+};
+
+
+#endif /* SCSI_BSG_FC_H */
+
diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
--- a/include/scsi/scsi_host.h	2008-10-18 10:32:54.000000000 -0400
+++ b/include/scsi/scsi_host.h	2009-02-10 10:34:42.000000000 -0500
@@ -478,6 +478,15 @@ struct scsi_host_template {
 	 * module_init/module_exit.
 	 */
 	struct list_head legacy_hosts;
+
+	/*
+	 * Vendor Identifier associated with the host
+	 *
+	 * Note: When specifying vendor_id, be sure to read the
+	 *   Vendor Type and ID formatting requirements specified in
+	 *   scsi_netlink.h
+	 */
+	u64 vendor_id;
 };
 
 /*
diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h
--- a/include/scsi/scsi_transport_fc.h	2009-01-27 09:44:40.000000000 -0500
+++ b/include/scsi/scsi_transport_fc.h	2009-02-10 13:12:20.000000000 -0500
@@ -33,7 +33,6 @@
 
 struct scsi_transport_template;
 
-
 /*
  * FC Port definitions - Following FC HBAAPI guidelines
  *
@@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
  	struct delayed_work fail_io_work;
  	struct work_struct stgt_delete_work;
 	struct work_struct rport_delete_work;
+	struct request_queue *rqst_q;	/* bsg support */
 } __attribute__((aligned(sizeof(unsigned long))));
 
 /* bit field values for struct fc_rport "flags" field: */
@@ -514,6 +514,9 @@ struct fc_host_attrs {
 	struct workqueue_struct *work_q;
 	char devloss_work_q_name[20];
 	struct workqueue_struct *devloss_work_q;
+
+	/* bsg support */
+	struct request_queue *rqst_q;
 };
 
 #define shost_to_fc_host(x) \
@@ -579,6 +582,47 @@ struct fc_host_attrs {
 	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
 
 
+struct fc_bsg_buffer {
+	unsigned int payload_len;
+	int sg_cnt;
+	struct scatterlist *sg_list;
+};
+
+/* Values for fc_bsg_job->state_flags (bitflags) */
+#define FC_RQST_STATE_INPROGRESS	0
+#define FC_RQST_STATE_DONE		1
+
+struct fc_bsg_job {
+	struct Scsi_Host *shost;
+	struct fc_rport *rport;
+	struct device *dev;
+	struct request *req;
+	spinlock_t job_lock;
+	unsigned int state_flags;
+	unsigned int ref_cnt;
+	void (*job_done)(struct fc_bsg_job *);
+
+	struct fc_bsg_request *request;
+	struct fc_bsg_reply *reply;
+	unsigned int request_len;
+	unsigned int reply_len;
+	/*
+	 * On entry : reply_len indicates the buffer size allocated for
+	 * the reply.
+	 *
+	 * Upon completion : the message handler must set reply_len
+	 *  to indicates the size of the reply to be returned to the
+	 *  caller.
+	 */
+
+	/* DMA payloads for the request/response */
+	struct fc_bsg_buffer request_payload;
+	struct fc_bsg_buffer reply_payload;
+
+	void *dd_data;			/* Used for driver-specific storage */
+};
+
+
 /* The functions by which the transport class and the driver communicate */
 struct fc_function_template {
 	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
@@ -614,9 +658,14 @@ struct fc_function_template {
 	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
 	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
 
+	/* bsg support */
+	int	(*bsg_request)(struct fc_bsg_job *);
+	int	(*bsg_timeout)(struct fc_bsg_job *);
+
 	/* allocation lengths for host-specific data */
 	u32	 			dd_fcrport_size;
 	u32	 			dd_fcvport_size;
+	u32				dd_bsg_size;
 
 	/*
 	 * The driver sets these to tell the transport class it
@@ -737,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
 	vport->vport_state = new_state;
 }
 
-
 struct scsi_transport_template *fc_attach_transport(
 			struct fc_function_template *);
 void fc_release_transport(struct scsi_transport_template *);

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Feb 11, 2009, at 7:13 AM, James Smart wrote:

> Trying to kick-start this again...
> I've updated the prior RFC with the comments from Seokmann,
> SvenFujita, and Boaz. I would still like review on the
> blk_xxx completion calls in the std and error paths.
I'm looking the driver changes to match it with latest updates in the  
transport layer.
The driver with the changes should be available within a couple of  
weeks.

Thank you,
Seokmann
>
>
> It currently expects that blk_end_request() has been updated
> by Fujita's patch to incorporate blk_end_bidi_request()
> functionality :
> http://marc.info/?l-linux-scsi&m=122785157116659&w=2

>
>
> -- james s
>
> ======
>
> All,
>
> I've reworked Seokmann's patch for the following items:
> - Add an fchost interface for bsg requests
>
> - Formalized the request/response structures that I expect
>  to have us stuff into the bsg cmd/sense data areas. These
>  are now genericized so we can essentially pass any kind of
>  transaction. It can be a request that has no transmit or
>  receive payload, and simply returns a response.
>
> - A new file was created, scsi_bsg_fc.h, which contains the
>  request/response data structures that should be shared
>  between the application and the kernel entities.
>
> - I stripped out some things that were in the request
>  structure that were actually LLD fields. Instead, I added
>  a dd_bsgsize structure to the template, so the transport
>  will allocate LLD work space along with the job structure.
>  I expect the missing fields to move to this area.
>
> - I've made a strong attempt at ensuring that the request
>  has all the information necessary for the LLD, so that
>  there is no need to have the LLD remap the transmit payload
>  to figure things out. Granted, this comes at the cost of
>  replicating some data items.
>
>  Sven, I've added the CT information you needed as part of this.
>
> - I've renamed things quite a bit, hoping to make it clarity
>  better. The "service" struct is now a job. I still have
>  headaches with "request" (is it the blk request, or the job
>  request, or what..)
>
> - The CT/ELS response is a bit funky. I've noted that the
>  way Emulex returns a response, vs Qlogic is a bit different,
>  thus the 2 ways to indicate "reject".
>
> - fixed a couple of bugs in Seokmann's code, in the teardown,
>  error flows, request que dma settings, etc.
>
> - I added a "vendor_id" field to the scsi_host_template to
>  use when verifying that the recipient knows how to decode
>  vendor-specific message. I didn't do this with the netlink
>  things as I was prepping it to not break kabi in existing
>  and older kernels. But, I believe this is a good time to
>  add it.
>
> - I've started the Documentation/scsi/scsi_transport_fc.txt
>  documentation, but punted finishing it in lieu of sending
>  this RFC. I'm starting from Seokman's original emails and
>  will be updating for this reformat.
>
> I'm only starting to debug this, so user beware.
>
> I could really use some code review from Fujita or Boaz, to
> make sure I'm calling the right blk_xx completion functions
> relative to the setup flow, and to ensure that the "goose"
> when I jump out while the rport is blocked is correct.
>
> Comments welcome
>
> -- james s
>
>
>
>
>
> Signed-off-by: James Smart <james.smart@emulex.com>
>
> ---
>
> Documentation/scsi/scsi_fc_transport.txt |   11
> Documentation/scsi/scsi_mid_low_api.txt  |    5
> drivers/scsi/scsi_transport_fc.c         |  614 +++++++++++++++++++++ 
> +++++++++-
> include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
> include/scsi/scsi_host.h                 |    9
> include/scsi/scsi_transport_fc.h         |   52 ++
> 6 files changed, 1009 insertions(+), 4 deletions(-)
>
>
> diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/ 
> Documentation/scsi/scsi_fc_transport.txt
> --- a/Documentation/scsi/scsi_fc_transport.txt  2009-01-27  
> 09:44:22.000000000 -0500
> +++ b/Documentation/scsi/scsi_fc_transport.txt  2009-02-10  
> 10:34:42.000000000 -0500
> @@ -1,10 +1,11 @@
>                              SCSI FC Tansport
>                  =============================================
>
> -Date:  4/12/2007
> +Date:  11/18/2008
> Kernel Revisions for features:
>   rports : <<TBS>>
>   vports : 2.6.22 (? TBD)
> +  bsg support : 2.6.29 (?TBD)
>
>
> Introduction
> @@ -15,6 +16,7 @@ The FC transport can be found at:
>   drivers/scsi/scsi_transport_fc.c
>   include/scsi/scsi_transport_fc.h
>   include/scsi/scsi_netlink_fc.h
> +  include/scsi/scsi_bsg_fc.h
>
> This file is found at Documentation/scsi/scsi_fc_transport.txt
>
> @@ -472,6 +474,13 @@ int
> fc_vport_terminate(struct fc_vport *vport)
>
>
> +FC BSG support (CT & ELS passthru, and more)
> + 
> = 
> = 
> ======================================================================
> +<< To Be Supplied >>
> +
> +
> +
> +
> Credits
> =======
> The following people have contributed to this document:
> diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/ 
> scsi/scsi_mid_low_api.txt
> --- a/Documentation/scsi/scsi_mid_low_api.txt   2008-09-23  
> 15:11:57.000000000 -0400
> +++ b/Documentation/scsi/scsi_mid_low_api.txt   2009-02-10  
> 10:34:42.000000000 -0500
> @@ -1271,6 +1271,11 @@ of interest:
>     hostdata[0]  - area reserved for LLD at end of struct Scsi_Host.  
> Size
>                    is set by the second argument (named 'xtr_bytes')  
> to
>                    scsi_host_alloc() or scsi_register().
> +    vendor_id    - a unique value that identifies the vendor  
> supplying
> +                   the LLD for the Scsi_Host.  Used most often in  
> validating
> +                   vendor-specific message requests.  Value  
> consists of an
> +                   identifier type and a vendor-specific value.
> +                   See scsi_netlink.h for a description of valid  
> formats.
>
> The scsi_host structure is defined in include/scsi/scsi_host.h
>
> diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/ 
> scsi_transport_fc.c
> --- a/drivers/scsi/scsi_transport_fc.c  2009-01-27  
> 09:44:31.000000000 -0500
> +++ b/drivers/scsi/scsi_transport_fc.c  2009-02-10  
> 14:58:39.000000000 -0500
> @@ -35,6 +35,7 @@
> #include <linux/netlink.h>
> #include <net/netlink.h>
> #include <scsi/scsi_netlink_fc.h>
> +#include <scsi/scsi_bsg_fc.h>
> #include "scsi_priv.h"
> #include "scsi_transport_fc_internal.h"
>
> @@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
> static int fc_vport_setup(struct Scsi_Host *shost, int channel,
>        struct device *pdev, struct fc_vport_identifiers  *ids,
>        struct fc_vport **vport);
> +static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs  
> *);
> +static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
> +static void fc_bsg_remove(struct request_queue *);
> +static void fc_bsg_goose_queue(struct fc_rport *);
>
> /*
>  * Redefine so that we can have same named attributes in the
> @@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
>                return -ENOMEM;
>        }
>
> +       fc_bsg_hostadd(shost, fc_host);
> +       /* ignore any bsg add error - we just can't do sgio */
> +
> +       return 0;
> +}
> +
> +static int fc_host_remove(struct transport_container *tc, struct  
> device *dev,
> +                        struct device *cdev)
> +{
> +       struct Scsi_Host *shost = dev_to_shost(dev);
> +       struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
> +
> +       fc_bsg_remove(fc_host->rqst_q);
>        return 0;
> }
>
> static DECLARE_TRANSPORT_CLASS(fc_host_class,
>                               "fc_host",
>                               fc_host_setup,
> -                              NULL,
> +                              fc_host_remove,
>                               NULL);
>
> /*
> @@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
>                scsi_flush_work(shost);
>
>        fc_terminate_rport_io(rport);
> +
>        /*
>         * Cancel any outstanding timers. These should really exist
>         * only when rmmod'ing the LLDD and we're asking for
> @@ -2415,6 +2434,8 @@ fc_rport_final_delete(struct work_struct
>            (i->f->dev_loss_tmo_callbk))
>                i->f->dev_loss_tmo_callbk(rport);
>
> +       fc_bsg_remove(rport->rqst_q);
> +
>        transport_remove_device(dev);
>        device_del(dev);
>        transport_destroy_device(dev);
> @@ -2502,6 +2523,9 @@ fc_rport_create(struct Scsi_Host *shost,
>        transport_add_device(dev);
>        transport_configure_device(dev);
>
> +       fc_bsg_rportadd(shost, rport);
> +       /* ignore any bsg add error - we just can't do sgio */
> +
>        if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
>                /* initiate a scan of the target */
>                rport->flags |= FC_RPORT_SCAN_PENDING;
> @@ -2666,6 +2690,8 @@ fc_remote_port_add(struct Scsi_Host *sho
>                                        spin_unlock_irqrestore(shost- 
> >host_lock,
>                                                        flags);
>
> +                               fc_bsg_goose_queue(rport);
> +
>                                return rport;
>                        }
>                }
> @@ -3351,6 +3377,592 @@ fc_vport_sched_delete(struct work_struct
> }
>
>
> +/*
> + * BSG support
> + */
> +
> +
> +/**
> + * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
> + * @job:       fc_bsg_job that is to be torn down
> + */
> +static void
> +fc_destroy_bsgjob(struct fc_bsg_job *job)
> +{
> +       unsigned long flags;
> +
> +       spin_lock_irqsave(&job->job_lock, flags);
> +       if (job->ref_cnt) {
> +               spin_unlock_irqrestore(&job->job_lock, flags);
> +               return;
> +       }
> +       spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +       put_device(job->dev);   /* release reference for the request  
> */
> +
> +       kfree(job->request_payload.sg_list);
> +       kfree(job->reply_payload.sg_list);
> +       kfree(job);
> +}
> +
> +
> +/**
> + * fc_bsg_jobdone - completion routine for bsg requests that the  
> LLD has
> + *                  completed
> + * @job:       fc_bsg_job that is complete
> + */
> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +       struct request *req = job->req;
> +       struct request *rsp = req->next_rq;
> +       unsigned long flags;
> +       unsigned rsp_len;
> +       int err;
> +
> +       spin_lock_irqsave(&job->job_lock, flags);
> +       job->state_flags |= FC_RQST_STATE_DONE;
> +       job->ref_cnt--;
> +       spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +       err = job->req->errors = job->reply->result;
> +       if (err < 0)
> +               /* we're only returning the result field in the  
> reply */
> +               job->req->sense_len = sizeof(uint32_t);
> +       else
> +               job->req->sense_len = job->reply_len;
> +
> +       if (rsp) {
> +               rsp_len = blk_rq_bytes(rsp);
> +               BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
> +               /* set reply (bidi) residual */
> +               rsp->data_len = (rsp_len - job->reply- 
> >reply_payload_rcv_len);
> +       }
> +
> +       /* we assume all request payload was transferred */
> +       blk_end_request(req, err, blk_rq_bytes(req));
> +
> +       fc_destroy_bsgjob(job);
> +}
> +
> +
> +/**
> + * fc_bsg_job_timeout - handler for when a bsg request timesout
> + * @req:       request that timed out
> + */
> +static enum blk_eh_timer_return
> +fc_bsg_job_timeout(struct request *req)
> +{
> +       struct fc_bsg_job *job = (void *) req->special;
> +       struct Scsi_Host *shost = job->shost;
> +       struct fc_internal *i = to_fc_internal(shost->transportt);
> +       unsigned long flags;
> +       int err = 0, done = 0;
> +
> +       if (job->rport && job->rport->port_state ==  
> FC_PORTSTATE_BLOCKED)
> +               return BLK_EH_RESET_TIMER;
> +
> +       spin_lock_irqsave(&job->job_lock, flags);
> +       if (job->state_flags & FC_RQST_STATE_DONE)
> +               done = 1;
> +       else
> +               job->ref_cnt++;
> +       spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +       if (!done && i->f->bsg_timeout) {
> +               /* call LLDD to abort the i/o as it has timed out */
> +               err = i->f->bsg_timeout(job);
> +               if (err)
> +                       printk(KERN_ERR "ERROR: FC BSG request  
> timeout - LLD "
> +                               "abort failed with status %d\n", err);
> +       }
> +
> +       if (!done) {
> +               spin_lock_irqsave(&job->job_lock, flags);
> +               job->ref_cnt--;
> +               spin_unlock_irqrestore(&job->job_lock, flags);
> +               fc_destroy_bsgjob(job);
> +       }
> +
> +       /* the blk_end_sync_io() doesn't check the error */
> +       return BLK_EH_HANDLED;
> +}
> +
> +
> +
> +static int
> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
> +{
> +       size_t sz = (sizeof(struct scatterlist) * req- 
> >nr_phys_segments);
> +
> +       BUG_ON(!req->nr_phys_segments);
> +
> +       buf->sg_list = kzalloc(sz, GFP_KERNEL);
> +       if (!buf->sg_list)
> +               return -ENOMEM;
> +       sg_init_table(buf->sg_list, req->nr_phys_segments);
> +       buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
> +       buf->payload_len = req->data_len;
> +       return 0;
> +}
> +
> +
> +/**
> + * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for  
> the
> + *                   bsg request
> + * @shost:     SCSI Host corresponding to the bsg object
> + * @rport:     (optional) FC Remote Port corresponding to the bsg  
> object
> + * @req:       BSG request that needs a job structure
> + */
> +static int
> +fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
> +       struct request *req)
> +{
> +       struct fc_internal *i = to_fc_internal(shost->transportt);
> +       struct request *rsp = req->next_rq;
> +       struct fc_bsg_job *job;
> +       int ret;
> +
> +       BUG_ON(req->special);
> +
> +       job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
> +                       GFP_KERNEL);
> +       if (!job)
> +               return -ENOMEM;
> +
> +       /*
> +        * Note: this is a bit silly.
> +        * The request gets formatted as a SGIO v4 ioctl request,  
> which
> +        * then gets reformatted as a blk request, which then gets
> +        * reformatted as a fc bsg request. And on completion, we have
> +        * to wrap return results such that SGIO v4 thinks it was a  
> scsi
> +        * status.  I hope this was all worth it.
> +        */
> +
> +       req->special = job;
> +       job->shost = shost;
> +       job->rport = rport;
> +       job->req = req;
> +       if (i->f->dd_bsg_size)
> +               job->dd_data = (void *)&job[1];
> +       spin_lock_init(&job->job_lock);
> +       job->request = (struct fc_bsg_request *)req->cmd;
> +       job->request_len = req->cmd_len;
> +       job->reply = req->sense;
> +       job->reply_len = SCSI_SENSE_BUFFERSIZE; /* Size of sense  
> buffer
> +                                                * allocated */
> +       if (req->bio) {
> +               ret = fc_bsg_map_buffer(&job->request_payload, req);
> +               if (ret)
> +                       goto failjob_rls_job;
> +       }
> +       if (rsp && rsp->bio) {
> +               ret = fc_bsg_map_buffer(&job->reply_payload, rsp);
> +               if (ret)
> +                       goto failjob_rls_rqst_payload;
> +       }
> +       job->job_done = fc_bsg_jobdone;
> +       if (rport)
> +               job->dev = &rport->dev;
> +       else
> +               job->dev = &shost->shost_gendev;
> +       get_device(job->dev);           /* take a reference for the  
> request */
> +
> +       job->ref_cnt = 1;
> +
> +       return 0;
> +
> +
> +failjob_rls_rqst_payload:
> +       kfree(job->request_payload.sg_list);
> +failjob_rls_job:
> +       kfree(job);
> +       return -ENOMEM;
> +}
> +
> +
> +enum fc_dispatch_result {
> +       FC_DISPATCH_BREAK,      /* on return, q is locked, break  
> from q loop */
> +       FC_DISPATCH_LOCKED,     /* on return, q is locked, continue  
> on */
> +       FC_DISPATCH_UNLOCKED,   /* on return, q is unlocked,  
> continue on */
> +};
> +
> +
> +/**
> + * fc_bsg_host_dispatch - process fc host bsg requests and dispatch  
> to LLDD
> + * @shost:     scsi host rport attached to
> + * @job:       bsg job to be processed
> + */
> +static enum fc_dispatch_result
> +fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host  
> *shost,
> +                        struct fc_bsg_job *job)
> +{
> +       struct fc_internal *i = to_fc_internal(shost->transportt);
> +       int cmdlen = sizeof(uint32_t);  /* start with length of  
> msgcode */
> +       int ret;
> +
> +       /* Validate the host command */
> +       switch (job->request->msgcode) {
> +       case FC_BSG_HST_ADD_RPORT:
> +               cmdlen += sizeof(struct fc_bsg_host_add_rport);
> +               break;
> +
> +       case FC_BSG_HST_DEL_RPORT:
> +               cmdlen += sizeof(struct fc_bsg_host_del_rport);
> +               break;
> +
> +       case FC_BSG_HST_ELS_NOLOGIN:
> +               cmdlen += sizeof(struct fc_bsg_host_els);
> +               /* there better be a xmt and rcv payloads */
> +               if ((!job->request_payload.payload_len) ||
> +                   (!job->reply_payload.payload_len)) {
> +                       ret = -EINVAL;
> +                       goto fail_host_msg;
> +               }
> +               break;
> +
> +       case FC_BSG_HST_CT:
> +               cmdlen += sizeof(struct fc_bsg_host_ct);
> +               /* there better be xmt and rcv payloads */
> +               if ((!job->request_payload.payload_len) ||
> +                   (!job->reply_payload.payload_len)) {
> +                       ret = -EINVAL;
> +                       goto fail_host_msg;
> +               }
> +               break;
> +
> +       case FC_BSG_HST_VENDOR:
> +               cmdlen += sizeof(struct fc_bsg_host_vendor);
> +               if ((shost->hostt->vendor_id == 0L) ||
> +                   (job->request->rqst_data.h_vendor.vendor_id !=
> +                       shost->hostt->vendor_id)) {
> +                       ret = -ESRCH;
> +                       goto fail_host_msg;
> +               }
> +               break;
> +
> +       default:
> +               ret = -EBADR;
> +               goto fail_host_msg;
> +       }
> +
> +       /* check if we really have all the request data needed */
> +       if (job->request_len < cmdlen) {
> +               ret = -ENOMSG;
> +               goto fail_host_msg;
> +       }
> +
> +       ret = i->f->bsg_request(job);
> +       if (!ret)
> +               return FC_DISPATCH_UNLOCKED;
> +
> +fail_host_msg:
> +       /* return the errno failure code as the only status */
> +       BUG_ON(job->reply_len < sizeof(uint32_t));
> +       job->reply->result = ret;
> +       job->reply_len = sizeof(uint32_t);
> +       fc_bsg_jobdone(job);
> +       return FC_DISPATCH_UNLOCKED;
> +}
> +
> +
> +/*
> + * fc_bsg_goose_queue - restart rport queue in case it was stopped
> + * @rport:     rport to be restarted
> + */
> +static void
> +fc_bsg_goose_queue(struct fc_rport *rport)
> +{
> +       int flagset;
> +
> +       if (!rport->rqst_q)
> +               return;
> +
> +       get_device(&rport->dev);
> +
> +       spin_lock(rport->rqst_q->queue_lock);
> +       flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
> >queue_flags) &&
> +                 !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
> >queue_flags);
> +       if (flagset)
> +               queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
> +       __blk_run_queue(rport->rqst_q);
> +       if (flagset)
> +               queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
> +       spin_unlock(rport->rqst_q->queue_lock);
> +
> +       put_device(&rport->dev);
> +}
> +
> +
> +/**
> + * fc_bsg_rport_dispatch - process rport bsg requests and dispatch  
> to LLDD
> + * @shost:     scsi host rport attached to
> + * @rport:     rport request destined to
> + * @job:       bsg job to be processed
> + */
> +static enum fc_dispatch_result
> +fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host  
> *shost,
> +                        struct fc_rport *rport, struct fc_bsg_job  
> *job)
> +{
> +       struct fc_internal *i = to_fc_internal(shost->transportt);
> +       int cmdlen = sizeof(uint32_t);  /* start with length of  
> msgcode */
> +       int ret;
> +
> +       /* Validate the rport command */
> +       switch (job->request->msgcode) {
> +       case FC_BSG_RPT_ELS:
> +               cmdlen += sizeof(struct fc_bsg_rport_els);
> +               goto check_bidi;
> +
> +       case FC_BSG_RPT_CT:
> +               cmdlen += sizeof(struct fc_bsg_rport_ct);
> +check_bidi:
> +               /* there better be xmt and rcv payloads */
> +               if ((!job->request_payload.payload_len) ||
> +                   (!job->reply_payload.payload_len)) {
> +                       ret = -EINVAL;
> +                       goto fail_rport_msg;
> +               }
> +               break;
> +       default:
> +               ret = -EBADR;
> +               goto fail_rport_msg;
> +       }
> +
> +       /* check if we really have all the request data needed */
> +       if (job->request_len < cmdlen) {
> +               ret = -ENOMSG;
> +               goto fail_rport_msg;
> +       }
> +
> +       ret = i->f->bsg_request(job);
> +       if (!ret)
> +               return FC_DISPATCH_UNLOCKED;
> +
> +fail_rport_msg:
> +       /* return the errno failure code as the only status */
> +       BUG_ON(job->reply_len < sizeof(uint32_t));
> +       job->reply->result = ret;
> +       job->reply_len = sizeof(uint32_t);
> +       fc_bsg_jobdone(job);
> +       return FC_DISPATCH_UNLOCKED;
> +}
> +
> +
> +/**
> + * fc_bsg_request_handler - generic handler for bsg requests
> + * @q:         request queue to manage
> + * @shost:     Scsi_Host related to the bsg object
> + * @rport:     FC remote port related to the bsg object (optional)
> + * @dev:       device structure for bsg object
> + */
> +static void
> +fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host  
> *shost,
> +                      struct fc_rport *rport, struct device *dev)
> +{
> +       struct request *req;
> +       struct fc_bsg_job *job;
> +       enum fc_dispatch_result ret;
> +
> +       if (!get_device(dev))
> +               return;
> +
> +       while (!blk_queue_plugged(q)) {
> +               req = elv_next_request(q);
> +               if (!req)
> +                       break;
> +
> +               if (rport && (rport->port_state ==  
> FC_PORTSTATE_BLOCKED))
> +                               break;
> +
> +               blkdev_dequeue_request(req);
> +
> +               if (rport && (rport->port_state !=  
> FC_PORTSTATE_ONLINE)) {
> +                       req->errors = -ENXIO;
> +                       spin_unlock_irq(q->queue_lock);
> +                       blk_end_request(req, -ENXIO,  
> blk_rq_bytes(req));
> +                       spin_lock_irq(q->queue_lock);
> +                       continue;
> +               }
> +
> +               spin_unlock_irq(q->queue_lock);
> +
> +               ret = fc_req_to_bsgjob(shost, rport, req);
> +               if (ret) {
> +                       req->errors = ret;
> +                       blk_end_request(req, ret, blk_rq_bytes(req));
> +                       spin_lock_irq(q->queue_lock);
> +                       continue;
> +               }
> +
> +               job = req->special;
> +
> +               /* check if we have the msgcode value at least */
> +               if (job->request_len < sizeof(uint32_t)) {
> +                       BUG_ON(job->reply_len < sizeof(uint32_t));
> +                       job->reply->result = -ENOMSG;
> +                       job->reply_len = sizeof(uint32_t);
> +                       fc_bsg_jobdone(job);
> +                       spin_lock_irq(q->queue_lock);
> +                       continue;
> +               }
> +
> +               /* the dispatch routines will unlock the queue_lock */
> +               if (rport)
> +                       ret = fc_bsg_rport_dispatch(q, shost, rport,  
> job);
> +               else
> +                       ret = fc_bsg_host_dispatch(q, shost, job);
> +
> +               /* did dispatcher hit state that can't process any  
> more */
> +               if (ret == FC_DISPATCH_BREAK)
> +                       break;
> +
> +               /* did dispatcher had released the lock */
> +               if (ret == FC_DISPATCH_UNLOCKED)
> +                       spin_lock_irq(q->queue_lock);
> +       }
> +
> +       spin_unlock_irq(q->queue_lock);
> +       put_device(dev);
> +       spin_lock_irq(q->queue_lock);
> +}
> +
> +
> +/**
> + * fc_bsg_host_handler - handler for bsg requests for a fc host
> + * @q:         fc host request queue
> + */
> +static void
> +fc_bsg_host_handler(struct request_queue *q)
> +{
> +       struct Scsi_Host *shost = q->queuedata;
> +
> +       fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
> +}
> +
> +
> +/**
> + * fc_bsg_rport_handler - handler for bsg requests for a fc rport
> + * @q:         rport request queue
> + */
> +static void
> +fc_bsg_rport_handler(struct request_queue *q)
> +{
> +       struct fc_rport *rport = q->queuedata;
> +       struct Scsi_Host *shost = rport_to_shost(rport);
> +
> +       fc_bsg_request_handler(q, shost, rport, &rport->dev);
> +}
> +
> +
> +/**
> + * fc_bsg_hostadd - Create and add the bsg hooks so we can receive  
> requests
> + * @shost:     shost for fc_host
> + * @fc_host:   fc_host adding the structures to
> + */
> +static int
> +fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs  
> *fc_host)
> +{
> +       struct device *dev = &shost->shost_gendev;
> +       struct fc_internal *i = to_fc_internal(shost->transportt);
> +       struct request_queue *q;
> +       int err;
> +       char bsg_name[BUS_ID_SIZE]; /*20*/
> +
> +       fc_host->rqst_q = NULL;
> +
> +       if (!i->f->bsg_request)
> +               return -ENOTSUPP;
> +
> +       snprintf(bsg_name, sizeof(bsg_name),
> +                "fc_host%d", shost->host_no);
> +
> +       q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
> +       if (!q) {
> +               printk(KERN_ERR "fc_host%d: bsg interface failed to "
> +                               "initialize - no request queue\n",
> +                                shost->host_no);
> +               return -ENOMEM;
> +       }
> +
> +       q->queuedata = shost;
> +       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
> +       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
> +       blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
> +
> +       err = bsg_register_queue(q, dev, bsg_name, NULL);
> +       if (err) {
> +               printk(KERN_ERR "fc_host%d: bsg interface failed to "
> +                               "initialize - register queue\n",
> +                               shost->host_no);
> +               blk_cleanup_queue(q);
> +               return err;
> +       }
> +
> +       fc_host->rqst_q = q;
> +       return 0;
> +}
> +
> +
> +/**
> + * fc_bsg_rportadd - Create and add the bsg hooks so we can receive  
> requests
> + * @shost:     shost that rport is attached to
> + * @rport:     rport that the bsg hooks are being attached to
> + */
> +static int
> +fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
> +{
> +       struct device *dev = &rport->dev;
> +       struct fc_internal *i = to_fc_internal(shost->transportt);
> +       struct request_queue *q;
> +       int err;
> +
> +       rport->rqst_q = NULL;
> +
> +       if (!i->f->bsg_request)
> +               return -ENOTSUPP;
> +
> +       q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
> +       if (!q) {
> +               printk(KERN_ERR "%s: bsg interface failed to "
> +                               "initialize - no request queue\n",
> +                                dev->bus_id);
> +               return -ENOMEM;
> +       }
> +
> +       q->queuedata = rport;
> +       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
> +       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
> +       blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
> +
> +       err = bsg_register_queue(q, dev, NULL, NULL);
> +       if (err) {
> +               printk(KERN_ERR "%s: bsg interface failed to "
> +                               "initialize - register queue\n",
> +                                dev->bus_id);
> +               blk_cleanup_queue(q);
> +               return err;
> +       }
> +
> +       rport->rqst_q = q;
> +       return 0;
> +}
> +
> +
> +/**
> + * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
> + * @q: the request_queue that is to be torn down.
> + */
> +static void
> +fc_bsg_remove(struct request_queue *q)
> +{
> +       if (q) {
> +               bsg_unregister_queue(q);
> +               blk_cleanup_queue(q);
> +       }
> +}
> +
> +
> /* Original Author:  Martin Hicks */
> MODULE_AUTHOR("James Smart");
> MODULE_DESCRIPTION("FC Transport Attributes");
> diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
> --- a/include/scsi/scsi_bsg_fc.h        1969-12-31  
> 19:00:00.000000000 -0500
> +++ b/include/scsi/scsi_bsg_fc.h        2009-02-10  
> 13:26:17.000000000 -0500
> @@ -0,0 +1,322 @@
> +/*
> + *  FC Transport BSG Interface
> + *
> + *  Copyright (C) 2008   James Smart, Emulex Corporation
> + *
> + *  This program is free software; you can redistribute it and/or  
> modify
> + *  it under the terms of the GNU General Public License as  
> published by
> + *  the Free Software Foundation; either version 2 of the License, or
> + *  (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with this program; if not, write to the Free Software
> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA   
> 02111-1307  USA
> + *
> + */
> +
> +#ifndef SCSI_BSG_FC_H
> +#define SCSI_BSG_FC_H
> +
> +/*
> + * This file intended to be included by both kernel and user space
> + */
> +
> +#include <scsi/scsi.h>
> +
> +/*
> + * FC Transport SGIO v4 BSG Message Support
> + */
> +
> +/* Default BSG request timeout (in seconds) */
> +#define FC_DEFAULT_BSG_TIMEOUT         (10 * HZ)
> +
> +
> +/*
> + * Request Message Codes supported by the FC Transport
> + */
> +
> +/* define the class masks for the message codes */
> +#define FC_BSG_CLS_MASK                0xF0000000      /* find  
> object class */
> +#define FC_BSG_HST_MASK                0x80000000      /* fc host  
> class */
> +#define FC_BSG_RPT_MASK                0x40000000      /* fc rport  
> class */
> +
> +       /* fc_host Message Codes */
> +#define FC_BSG_HST_ADD_RPORT           (FC_BSG_HST_MASK | 0x00000001)
> +#define FC_BSG_HST_DEL_RPORT           (FC_BSG_HST_MASK | 0x00000002)
> +#define FC_BSG_HST_ELS_NOLOGIN         (FC_BSG_HST_MASK | 0x00000003)
> +#define FC_BSG_HST_CT                  (FC_BSG_HST_MASK | 0x00000004)
> +#define FC_BSG_HST_VENDOR              (FC_BSG_HST_MASK | 0x000000FF)
> +
> +       /* fc_rport Message Codes */
> +#define FC_BSG_RPT_ELS                 (FC_BSG_RPT_MASK | 0x00000001)
> +#define FC_BSG_RPT_CT                  (FC_BSG_RPT_MASK | 0x00000002)
> +
> +
> +
> +/*
> + * FC Address Identifiers in Message Structures :
> + *
> + *   Whenever a command payload contains a FC Address Identifier
> + *   (aka port_id), the value is effectively in big-endian
> + *   order, thus the array elements are decoded as follows:
> + *     element [0] is bits 23:16 of the FC Address Identifier
> + *     element [1] is bits 15:8 of the FC Address Identifier
> + *     element [2] is bits 7:0 of the FC Address Identifier
> + */
> +
> +
> +/*
> + * FC Host Messages
> + */
> +
> +/* FC_BSG_HST_ADDR_PORT : */
> +
> +/* Request:
> + * This message requests the FC host to login to the remote port
> + * at the specified N_Port_Id.  The remote port is to be enumerated
> + * with the transport upon completion of the login.
> + */
> +struct fc_bsg_host_add_rport {
> +       uint8_t         reserved;
> +
> +       /* FC Address Identier of the remote port to login to */
> +       uint8_t         port_id[3];
> +};
> +
> +/* Response:
> + * There is no additional response data - fc_bsg_reply->result is  
> sufficient
> + */
> +
> +
> +/* FC_BSG_HST_DEL_RPORT : */
> +
> +/* Request:
> + * This message requests the FC host to remove an enumerated
> + * remote port and to terminate the login to it.
> + *
> + * Note: The driver is free to reject this request if it desires to
> + * remain logged in with the remote port.
> + */
> +struct fc_bsg_host_del_rport {
> +       uint8_t         reserved;
> +
> +       /* FC Address Identier of the remote port to logout of */
> +       uint8_t         port_id[3];
> +};
> +
> +/* Response:
> + * There is no additional response data - fc_bsg_reply->result is  
> sufficient
> + */
> +
> +
> +/* FC_BSG_HST_ELS_NOLOGIN : */
> +
> +/* Request:
> + * This message requests the FC_Host to send an ELS to a specific
> + * N_Port_ID. The host does not need to log into the remote port,
> + * nor does it need to enumerate the rport for further traffic
> + * (although, the FC host is free to do so if it desires).
> + */
> +struct fc_bsg_host_els {
> +       /*
> +        * ELS Command Code being sent (must be the same as byte 0
> +        * of the payload)
> +        */
> +       uint8_t         command_code;
> +
> +       /* FC Address Identier of the remote port to send the ELS to  
> */
> +       uint8_t         port_id[3];
> +};
> +
> +/* Response:
> + */
> +/* fc_bsg_ctels_reply->status values */
> +#define FC_CTELS_STATUS_OK     0x00000000
> +#define FC_CTELS_STATUS_REJECT 0x00000001
> +#define FC_CTELS_STATUS_P_RJT  0x00000002
> +#define FC_CTELS_STATUS_F_RJT  0x00000003
> +#define FC_CTELS_STATUS_P_BSY  0x00000004
> +#define FC_CTELS_STATUS_F_BSY  0x00000006
> +struct fc_bsg_ctels_reply {
> +       /*
> +        * Note: An ELS LS_RJT may be reported in 2 ways:
> +        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
> +        *     is to look into the ELS receive payload to determine
> +        *     LS_ACC or LS_RJT (by contents of word 0). The reject
> +        *     data will be in word 1.
> +        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
> +        *     rjt_data field will contain valid data.
> +        *
> +        * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK,  
> and
> +        *   the receive payload word 0 indicates LS_ACC
> +        *   (e.g. value is 0x02xxxxxx).
> +        *
> +        * Note: Similarly, a CT Reject may be reported in 2 ways:
> +        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
> +        *     is to look into the CT receive payload to determine
> +        *     Accept or Reject (by contents of word 2). The reject
> +        *     data will be in word 3.
> +        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
> +        *     rjt_data field will contain valid data.
> +        *
> +        * Note: x_RJT/BSY status will indicae that the rjt_data field
> +        *   is valid and contains the reason/explanation values.
> +        */
> +       uint32_t        status;         /* See FC_CTELS_STATUS_xxx */
> +
> +       /* valid if status is not FC_CTELS_STATUS_OK */
> +       struct  {
> +               uint8_t action;         /* fragment_id for CT REJECT  
> */
> +               uint8_t reason_code;
> +               uint8_t reason_explanation;
> +               uint8_t vendor_unique;
> +       } rjt_data;
> +};
> +
> +
> +/* FC_BSG_HST_CT : */
> +
> +/* Request:
> + * This message requests that a CT Request be performed with the
> + * indicated N_Port_ID. The driver is responsible for logging in with
> + * the fabric and/or N_Port_ID, etc as per FC rules. This request  
> does
> + * not mandate that the driver must enumerate the destination in the
> + * transport. The driver is allowed to decide whether to enumerate  
> it,
> + * and whether to tear it down after the request.
> + */
> +struct fc_bsg_host_ct {
> +       uint8_t         reserved;
> +
> +       /* FC Address Identier of the remote port to send the ELS to  
> */
> +       uint8_t         port_id[3];
> +
> +       /*
> +        * We need words 0-2 of the generic preamble for the LLD's
> +        */
> +       uint32_t        preamble_word0; /* revision & IN_ID */
> +       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  
> Options, Rsvd */
> +       uint32_t        preamble_word2; /* Cmd Code, Max Size */
> +
> +};
> +/* Response:
> + *
> + * The reply structure is an fc_bsg_ctels_reply structure
> + */
> +
> +
> +/* FC_BSG_HST_VENDOR : */
> +
> +/* Request:
> + * Note: When specifying vendor_id, be sure to read the Vendor Type  
> and ID
> + *   formatting requirements specified in scsi_netlink.h
> + */
> +struct fc_bsg_host_vendor {
> +       /*
> +        * Identifies the vendor that the message is formatted for.  
> This
> +        * should be the recipient of the message.
> +        */
> +       uint64_t vendor_id;
> +
> +       /* start of vendor command area */
> +       uint32_t vendor_cmd[0];
> +};
> +
> +/* Response:
> + */
> +struct fc_bsg_host_vendor_reply {
> +       /* start of vendor response area */
> +       uint32_t vendor_rsp[0];
> +};
> +
> +
> +
> +/*
> + * FC Remote Port Messages
> + */
> +
> +/* FC_BSG_RPT_ELS : */
> +
> +/* Request:
> + * This message requests that an ELS be performed with the rport.
> + */
> +struct fc_bsg_rport_els {
> +       /*
> +        * ELS Command Code being sent (must be the same as
> +        * byte 0 of the payload)
> +        */
> +       uint8_t els_code;
> +};
> +
> +/* Response:
> + *
> + * The reply structure is an fc_bsg_ctels_reply structure
> + */
> +
> +
> +/* FC_BSG_RPT_CT : */
> +
> +/* Request:
> + * This message requests that a CT Request be performed with the  
> rport.
> + */
> +struct fc_bsg_rport_ct {
> +       /*
> +        * We need words 0-2 of the generic preamble for the LLD's
> +        */
> +       uint32_t        preamble_word0; /* revision & IN_ID */
> +       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  
> Options, Rsvd */
> +       uint32_t        preamble_word2; /* Cmd Code, Max Size */
> +};
> +/* Response:
> + *
> + * The reply structure is an fc_bsg_ctels_reply structure
> + */
> +
> +
> +
> +
> +/* request (CDB) structure of the sg_io_v4 */
> +struct fc_bsg_request {
> +       uint32_t msgcode;
> +       union {
> +               struct fc_bsg_host_add_rport    h_addrport;
> +               struct fc_bsg_host_del_rport    h_delrport;
> +               struct fc_bsg_host_els          h_els;
> +               struct fc_bsg_host_ct           h_ct;
> +               struct fc_bsg_host_vendor       h_vendor;
> +
> +               struct fc_bsg_rport_els         r_els;
> +               struct fc_bsg_rport_ct          r_ct;
> +       } rqst_data;
> +};
> +
> +
> +/* response (request sense data) structure of the sg_io_v4 */
> +struct fc_bsg_reply {
> +       /*
> +        * The completion result. Result exists in two forms:
> +        *  if negative, it is an -Exxx system errno value. There will
> +        *    be no further reply information supplied.
> +        *  else, it's the 4-byte scsi error result, with driver,  
> host,
> +        *    msg and status fields. The per-msgcode reply structure
> +        *    will contain valid data.
> +        */
> +       uint32_t result;
> +
> +       /* If there was reply_payload, how much was recevied ? */
> +       uint32_t reply_payload_rcv_len;
> +
> +       union {
> +               struct fc_bsg_host_vendor_reply         vendor_reply;
> +
> +               struct fc_bsg_ctels_reply               ctels_reply;
> +       } reply_data;
> +};
> +
> +
> +#endif /* SCSI_BSG_FC_H */
> +
> diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
> --- a/include/scsi/scsi_host.h  2008-10-18 10:32:54.000000000 -0400
> +++ b/include/scsi/scsi_host.h  2009-02-10 10:34:42.000000000 -0500
> @@ -478,6 +478,15 @@ struct scsi_host_template {
>         * module_init/module_exit.
>         */
>        struct list_head legacy_hosts;
> +
> +       /*
> +        * Vendor Identifier associated with the host
> +        *
> +        * Note: When specifying vendor_id, be sure to read the
> +        *   Vendor Type and ID formatting requirements specified in
> +        *   scsi_netlink.h
> +        */
> +       u64 vendor_id;
> };
>
> /*
> diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/ 
> scsi_transport_fc.h
> --- a/include/scsi/scsi_transport_fc.h  2009-01-27  
> 09:44:40.000000000 -0500
> +++ b/include/scsi/scsi_transport_fc.h  2009-02-10  
> 13:12:20.000000000 -0500
> @@ -33,7 +33,6 @@
>
> struct scsi_transport_template;
>
> -
> /*
>  * FC Port definitions - Following FC HBAAPI guidelines
>  *
> @@ -352,6 +351,7 @@ struct fc_rport {   /* aka fc_starget_attr
>        struct delayed_work fail_io_work;
>        struct work_struct stgt_delete_work;
>        struct work_struct rport_delete_work;
> +       struct request_queue *rqst_q;   /* bsg support */
> } __attribute__((aligned(sizeof(unsigned long))));
>
> /* bit field values for struct fc_rport "flags" field: */
> @@ -514,6 +514,9 @@ struct fc_host_attrs {
>        struct workqueue_struct *work_q;
>        char devloss_work_q_name[20];
>        struct workqueue_struct *devloss_work_q;
> +
> +       /* bsg support */
> +       struct request_queue *rqst_q;
> };
>
> #define shost_to_fc_host(x) \
> @@ -579,6 +582,47 @@ struct fc_host_attrs {
>        (((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
>
>
> +struct fc_bsg_buffer {
> +       unsigned int payload_len;
> +       int sg_cnt;
> +       struct scatterlist *sg_list;
> +};
> +
> +/* Values for fc_bsg_job->state_flags (bitflags) */
> +#define FC_RQST_STATE_INPROGRESS       0
> +#define FC_RQST_STATE_DONE             1
> +
> +struct fc_bsg_job {
> +       struct Scsi_Host *shost;
> +       struct fc_rport *rport;
> +       struct device *dev;
> +       struct request *req;
> +       spinlock_t job_lock;
> +       unsigned int state_flags;
> +       unsigned int ref_cnt;
> +       void (*job_done)(struct fc_bsg_job *);
> +
> +       struct fc_bsg_request *request;
> +       struct fc_bsg_reply *reply;
> +       unsigned int request_len;
> +       unsigned int reply_len;
> +       /*
> +        * On entry : reply_len indicates the buffer size allocated  
> for
> +        * the reply.
> +        *
> +        * Upon completion : the message handler must set reply_len
> +        *  to indicates the size of the reply to be returned to the
> +        *  caller.
> +        */
> +
> +       /* DMA payloads for the request/response */
> +       struct fc_bsg_buffer request_payload;
> +       struct fc_bsg_buffer reply_payload;
> +
> +       void *dd_data;                  /* Used for driver-specific  
> storage */
> +};
> +
> +
> /* The functions by which the transport class and the driver  
> communicate */
> struct fc_function_template {
>        void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
> @@ -614,9 +658,14 @@ struct fc_function_template {
>        int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64,  
> int);
>        int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
>
> +       /* bsg support */
> +       int     (*bsg_request)(struct fc_bsg_job *);
> +       int     (*bsg_timeout)(struct fc_bsg_job *);
> +
>        /* allocation lengths for host-specific data */
>        u32                             dd_fcrport_size;
>        u32                             dd_fcvport_size;
> +       u32                             dd_bsg_size;
>
>        /*
>         * The driver sets these to tell the transport class it
> @@ -737,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
>        vport->vport_state = new_state;
> }
>
> -
> struct scsi_transport_template *fc_attach_transport(
>                        struct fc_function_template *);
> void fc_release_transport(struct scsi_transport_template *);
>
>

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

James Smart wrote:
> Trying to kick-start this again...  
> I've updated the prior RFC with the comments from Seokmann,
> SvenFujita, and Boaz. I would still like review on the
> blk_xxx completion calls in the std and error paths.
> 
> It currently expects that blk_end_request() has been updated
> by Fujita's patch to incorporate blk_end_bidi_request()
> functionality :
> http://marc.info/?l-linux-scsi&m=122785157116659&w=2
> 

I did not accept this patch and it did not go in right?
I still don't like it, it's a performance regression.

> -- james s
> 
> ======
> 
> All,
> 
> I've reworked Seokmann's patch for the following items:
> - Add an fchost interface for bsg requests
> 
> - Formalized the request/response structures that I expect
>   to have us stuff into the bsg cmd/sense data areas. These
>   are now genericized so we can essentially pass any kind of
>   transaction. It can be a request that has no transmit or
>   receive payload, and simply returns a response.
> 
> - A new file was created, scsi_bsg_fc.h, which contains the
>   request/response data structures that should be shared
>   between the application and the kernel entities. 
> 
> - I stripped out some things that were in the request
>   structure that were actually LLD fields. Instead, I added
>   a dd_bsgsize structure to the template, so the transport
>   will allocate LLD work space along with the job structure.
>   I expect the missing fields to move to this area.
> 
> - I've made a strong attempt at ensuring that the request
>   has all the information necessary for the LLD, so that
>   there is no need to have the LLD remap the transmit payload
>   to figure things out. Granted, this comes at the cost of
>   replicating some data items.
> 
>   Sven, I've added the CT information you needed as part of this.
> 
> - I've renamed things quite a bit, hoping to make it clarity
>   better. The "service" struct is now a job. I still have
>   headaches with "request" (is it the blk request, or the job
>   request, or what..)
> 
> - The CT/ELS response is a bit funky. I've noted that the
>   way Emulex returns a response, vs Qlogic is a bit different,
>   thus the 2 ways to indicate "reject".
> 
> - fixed a couple of bugs in Seokmann's code, in the teardown,
>   error flows, request que dma settings, etc.
> 
> - I added a "vendor_id" field to the scsi_host_template to
>   use when verifying that the recipient knows how to decode
>   vendor-specific message. I didn't do this with the netlink
>   things as I was prepping it to not break kabi in existing
>   and older kernels. But, I believe this is a good time to
>   add it.
> 
> - I've started the Documentation/scsi/scsi_transport_fc.txt
>   documentation, but punted finishing it in lieu of sending
>   this RFC. I'm starting from Seokman's original emails and
>   will be updating for this reformat.
> 
> I'm only starting to debug this, so user beware.
> 
> I could really use some code review from Fujita or Boaz, to
> make sure I'm calling the right blk_xx completion functions
> relative to the setup flow, and to ensure that the "goose"
> when I jump out while the rport is blocked is correct.
> 
> Comments welcome
> 
> -- james s
> 
> 
> 
> 
> 
>  Signed-off-by: James Smart <james.smart@emulex.com>
> 
>  ---
> 
>  Documentation/scsi/scsi_fc_transport.txt |   11 
>  Documentation/scsi/scsi_mid_low_api.txt  |    5 
>  drivers/scsi/scsi_transport_fc.c         |  614 ++++++++++++++++++++++++++++++-
>  include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
>  include/scsi/scsi_host.h                 |    9 
>  include/scsi/scsi_transport_fc.h         |   52 ++
>  6 files changed, 1009 insertions(+), 4 deletions(-)
> 
> 
> diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt
> --- a/Documentation/scsi/scsi_fc_transport.txt	2009-01-27 09:44:22.000000000 -0500
> +++ b/Documentation/scsi/scsi_fc_transport.txt	2009-02-10 10:34:42.000000000 -0500
> @@ -1,10 +1,11 @@
>                               SCSI FC Tansport
>                   =============================================
>  
> -Date:  4/12/2007
> +Date:  11/18/2008
>  Kernel Revisions for features:
>    rports : <<TBS>>
>    vports : 2.6.22 (? TBD)
> +  bsg support : 2.6.29 (?TBD)
>  
>  
>  Introduction
> @@ -15,6 +16,7 @@ The FC transport can be found at:
>    drivers/scsi/scsi_transport_fc.c
>    include/scsi/scsi_transport_fc.h
>    include/scsi/scsi_netlink_fc.h
> +  include/scsi/scsi_bsg_fc.h
>  
>  This file is found at Documentation/scsi/scsi_fc_transport.txt
>  
> @@ -472,6 +474,13 @@ int
>  fc_vport_terminate(struct fc_vport *vport)
>  
>  
> +FC BSG support (CT & ELS passthru, and more)
> +========================================================================
> +<< To Be Supplied >>
> +
> +
> +
> +
>  Credits
>  =======
>  The following people have contributed to this document:
> diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
> --- a/Documentation/scsi/scsi_mid_low_api.txt	2008-09-23 15:11:57.000000000 -0400
> +++ b/Documentation/scsi/scsi_mid_low_api.txt	2009-02-10 10:34:42.000000000 -0500
> @@ -1271,6 +1271,11 @@ of interest:
>      hostdata[0]  - area reserved for LLD at end of struct Scsi_Host. Size
>                     is set by the second argument (named 'xtr_bytes') to
>                     scsi_host_alloc() or scsi_register().
> +    vendor_id    - a unique value that identifies the vendor supplying
> +                   the LLD for the Scsi_Host.  Used most often in validating
> +                   vendor-specific message requests.  Value consists of an
> +                   identifier type and a vendor-specific value.
> +                   See scsi_netlink.h for a description of valid formats.
>  
>  The scsi_host structure is defined in include/scsi/scsi_host.h
>  
> diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c
> --- a/drivers/scsi/scsi_transport_fc.c	2009-01-27 09:44:31.000000000 -0500
> +++ b/drivers/scsi/scsi_transport_fc.c	2009-02-10 14:58:39.000000000 -0500
> @@ -35,6 +35,7 @@
>  #include <linux/netlink.h>
>  #include <net/netlink.h>
>  #include <scsi/scsi_netlink_fc.h>
> +#include <scsi/scsi_bsg_fc.h>
>  #include "scsi_priv.h"
>  #include "scsi_transport_fc_internal.h"
>  
> @@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
>  static int fc_vport_setup(struct Scsi_Host *shost, int channel,
>  	struct device *pdev, struct fc_vport_identifiers  *ids,
>  	struct fc_vport **vport);
> +static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs *);
> +static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
> +static void fc_bsg_remove(struct request_queue *);
> +static void fc_bsg_goose_queue(struct fc_rport *);
>  
>  /*
>   * Redefine so that we can have same named attributes in the
> @@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
>  		return -ENOMEM;
>  	}
>  
> +	fc_bsg_hostadd(shost, fc_host);
> +	/* ignore any bsg add error - we just can't do sgio */
> +
> +	return 0;
> +}
> +
> +static int fc_host_remove(struct transport_container *tc, struct device *dev,
> +			 struct device *cdev)
> +{
> +	struct Scsi_Host *shost = dev_to_shost(dev);
> +	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
> +
> +	fc_bsg_remove(fc_host->rqst_q);
>  	return 0;
>  }
>  
>  static DECLARE_TRANSPORT_CLASS(fc_host_class,
>  			       "fc_host",
>  			       fc_host_setup,
> -			       NULL,
> +			       fc_host_remove,
>  			       NULL);
>  
>  /*
> @@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
>  		scsi_flush_work(shost);
>  
>  	fc_terminate_rport_io(rport);
> +
>  	/*
>  	 * Cancel any outstanding timers. These should really exist
>  	 * only when rmmod'ing the LLDD and we're asking for
> @@ -2415,6 +2434,8 @@ fc_rport_final_delete(struct work_struct
>  	    (i->f->dev_loss_tmo_callbk))
>  		i->f->dev_loss_tmo_callbk(rport);
>  
> +	fc_bsg_remove(rport->rqst_q);
> +
>  	transport_remove_device(dev);
>  	device_del(dev);
>  	transport_destroy_device(dev);
> @@ -2502,6 +2523,9 @@ fc_rport_create(struct Scsi_Host *shost,
>  	transport_add_device(dev);
>  	transport_configure_device(dev);
>  
> +	fc_bsg_rportadd(shost, rport);
> +	/* ignore any bsg add error - we just can't do sgio */
> +
>  	if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
>  		/* initiate a scan of the target */
>  		rport->flags |= FC_RPORT_SCAN_PENDING;
> @@ -2666,6 +2690,8 @@ fc_remote_port_add(struct Scsi_Host *sho
>  					spin_unlock_irqrestore(shost->host_lock,
>  							flags);
>  
> +				fc_bsg_goose_queue(rport);
> +
>  				return rport;
>  			}
>  		}
> @@ -3351,6 +3377,592 @@ fc_vport_sched_delete(struct work_struct
>  }
>  
>  
> +/*
> + * BSG support
> + */
> +
> +
> +/**
> + * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
> + * @job:	fc_bsg_job that is to be torn down
> + */
> +static void
> +fc_destroy_bsgjob(struct fc_bsg_job *job)
> +{
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	if (job->ref_cnt) {
> +		spin_unlock_irqrestore(&job->job_lock, flags);
> +		return;
> +	}
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	put_device(job->dev);	/* release reference for the request */
> +
> +	kfree(job->request_payload.sg_list);
> +	kfree(job->reply_payload.sg_list);
> +	kfree(job);
> +}
> +
> +
> +/**
> + * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
> + *                  completed
> + * @job:	fc_bsg_job that is complete
> + */
> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +	struct request *req = job->req;
> +	struct request *rsp = req->next_rq;
> +	unsigned long flags;
> +	unsigned rsp_len;
-	unsigned rsp_len;
+	unsigned rsp_len = 0;
+	unsigned req_len = 0;

See below

> +	int err;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	job->state_flags |= FC_RQST_STATE_DONE;
> +	job->ref_cnt--;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	err = job->req->errors = job->reply->result;
> +	if (err < 0)
> +		/* we're only returning the result field in the reply */
> +		job->req->sense_len = sizeof(uint32_t);
> +	else
> +		job->req->sense_len = job->reply_len;
> +

Why use of  job->req when you have req = job->req above, it's confusing.

> +	if (rsp) {
> +		rsp_len = blk_rq_bytes(rsp);
> +		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
> +		/* set reply (bidi) residual */
> +		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
> +	}
> +
> +	/* we assume all request payload was transferred */
> +	blk_end_request(req, err, blk_rq_bytes(req));

Don't you need residual count in bsg caller?

This code will always return full request->data_len as residual count
of out/uni_in to caller of bsg SG_IO caller.

and it will only work with the un-accepted patch above just do:

+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	req_len = blk_rq_bytes(req);
+	/* we assume all request payload was transferred */
+	req->data_len = 0;
+	blk_end_bidi_request(req, err, req_len, rsp_len);

This will handle all cases today without need of above patch.

> +
> +	fc_destroy_bsgjob(job);
> +}
> +
> +
<snip>

Thanks
Boaz

Re: [RFC] FC pass thru - Rev V

[FUJITA Tomonori]

On Wed, 11 Feb 2009 18:15:07 +0200
Boaz Harrosh <bharrosh@panasas.com> wrote:

> James Smart wrote:
> > Trying to kick-start this again...  
> > I've updated the prior RFC with the comments from Seokmann,
> > SvenFujita, and Boaz. I would still like review on the
> > blk_xxx completion calls in the std and error paths.
> > 
> > It currently expects that blk_end_request() has been updated
> > by Fujita's patch to incorporate blk_end_bidi_request()
> > functionality :
> > http://marc.info/?l-linux-scsi&m=122785157116659&w=2
> > 
> 
> I did not accept this patch and it did not go in right?

I think that Jens has not merged mine or yours. I don't care about
either but I still think that it's better to kill
blk_end_bidi_request(). It's really confusing API.


> I still don't like it, it's a performance regression.

Hmm, I've not seen the figures. Please show the figures if you insist
a performance regression.

It's about a bidi request. We already have tons of loops, memory
allocations, etc in the path. Do you think that adding one more loop
leads to a notable performance regression?

Well, if you say that it's hacky then I would agree. But your patch
using ~0 is hacky too.

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

FUJITA Tomonori wrote:
> On Wed, 11 Feb 2009 18:15:07 +0200
> Boaz Harrosh <bharrosh@panasas.com> wrote:
> 
>> James Smart wrote:
>>> Trying to kick-start this again...  
>>> I've updated the prior RFC with the comments from Seokmann,
>>> SvenFujita, and Boaz. I would still like review on the
>>> blk_xxx completion calls in the std and error paths.
>>>
>>> It currently expects that blk_end_request() has been updated
>>> by Fujita's patch to incorporate blk_end_bidi_request()
>>> functionality :
>>> http://marc.info/?l-linux-scsi&m=122785157116659&w=2
>>>
>> I did not accept this patch and it did not go in right?
> 
> I think that Jens has not merged mine or yours. I don't care about
> either but I still think that it's better to kill
> blk_end_bidi_request(). It's really confusing API.
> 
> 
>> I still don't like it, it's a performance regression.
> 
> Hmm, I've not seen the figures. Please show the figures if you insist
> a performance regression.
> 

Come on man. this discussion all over again. You do a loop to find
information that was in a CPU register 10 cycles before. That is
plain bad programing, and just a cover up of bad API.

In my book it is: Someone got lazy.

> It's about a bidi request. We already have tons of loops, memory
> allocations, etc in the path. Do you think that adding one more loop
> leads to a notable performance regression?
> 
> Well, if you say that it's hacky then I would agree. But your patch
> using ~0 is hacky too.

It is an hack if used by an outside user, because it assumes knowledge
of block-internals. It is much less of an hack if done by block-internals
which knows for sure that this has no side effects.

But I agree that this is not clean. The clean solution is to add an extra
parameter to blk_end_request() and change all callers.

Or even cleaner is to add a new request->residual member and leave
request->data_len be in peace. Then change the few users that care
about residual, and one caller that sets it. I'll prepare a patch.

Boaz

Re: [RFC] FC pass thru - Rev V

[FUJITA Tomonori]

On Wed, 11 Feb 2009 18:55:00 +0200
Boaz Harrosh <bharrosh@panasas.com> wrote:

> > It's about a bidi request. We already have tons of loops, memory
> > allocations, etc in the path. Do you think that adding one more loop
> > leads to a notable performance regression?
> > 
> > Well, if you say that it's hacky then I would agree. But your patch
> > using ~0 is hacky too.
> 
> It is an hack if used by an outside user, because it assumes knowledge
> of block-internals. It is much less of an hack if done by block-internals
> which knows for sure that this has no side effects.
> 
> But I agree that this is not clean. The clean solution is to add an extra
> parameter to blk_end_request() and change all callers.

I don't agree.


> Or even cleaner is to add a new request->residual member and leave
> request->data_len be in peace. Then change the few users that care
> about residual, and one caller that sets it. I'll prepare a patch.

Yeah, it's clean but I'm not sure Jens would accept such patch since
fattening request struct leads to a notable performance regression.

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

FUJITA Tomonori wrote:
> On Wed, 11 Feb 2009 18:55:00 +0200
> Boaz Harrosh <bharrosh@panasas.com> wrote:
> 
>>> It's about a bidi request. We already have tons of loops, memory
>>> allocations, etc in the path. Do you think that adding one more loop
>>> leads to a notable performance regression?
>>>
>>> Well, if you say that it's hacky then I would agree. But your patch
>>> using ~0 is hacky too.
>> It is an hack if used by an outside user, because it assumes knowledge
>> of block-internals. It is much less of an hack if done by block-internals
>> which knows for sure that this has no side effects.
>>
>> But I agree that this is not clean. The clean solution is to add an extra
>> parameter to blk_end_request() and change all callers.
> 
> I don't agree.
> 
> 
>> Or even cleaner is to add a new request->residual member and leave
>> request->data_len be in peace. Then change the few users that care
>> about residual, and one caller that sets it. I'll prepare a patch.
> 
> Yeah, it's clean but I'm not sure Jens would accept such patch since
> fattening request struct leads to a notable performance regression.

OK, then give up, a proper comment next to blk_end_bidi_request()
that says:
"... can be used with any requests, in case of uni-requests the
second length will be ignored"

Actual FC code changes is:
-	blk_end_request(req, err, req_len);
+	blk_end_bidi_request(req, err, req_len, rsp_len);

That is not so hard to understand. There are much more complicated
things in kernel then that simple thing.

Boaz

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:

>
> On Feb 11, 2009, at 7:13 AM, James Smart wrote:
>
>> Trying to kick-start this again...
>> I've updated the prior RFC with the comments from Seokmann,
>> SvenFujita, and Boaz. I would still like review on the
>> blk_xxx completion calls in the std and error paths.
> I'm looking the driver changes to match it with latest updates in  
> the transport layer.
> The driver with the changes should be available within a couple of  
> weeks.
When the FC transport layer calls fc_bsg_map_buffer(), it eventually  
calls
blk_rq_map_sg() to build sg_list for mapped bio in the request  
structure.
I'm having trouble, however, in getting the sg_list for both dout_xferp
and din_xferp, which is mapped into the bio in the request structure.
With the pass-thru - Rev V, the sg_list returned from the  
fc_bsg_map_buffer()
seems not right, particularly, the dma_address and dma_length fields  
are NULL.
The sg_cnt and payload_len of the request_payload/reply_payload are  
correctly
set by the function, though.

I've been trying to figure out what causes those fields to be NULL  
but, no
luck.

Is there something that I need to be cautious in traveling these code  
path?

Any comments would be helpful.

Thank you,
Seokmann


>
>
> Thank you,
> Seokmann
>>
>>
>> It currently expects that blk_end_request() has been updated
>> by Fujita's patch to incorporate blk_end_bidi_request()
>> functionality :
>> http://marc.info/?l-linux-scsi&m=122785157116659&w=2
>
>>
>>
>> -- james s
>>
>> ======
>>
>> All,
>>
>> I've reworked Seokmann's patch for the following items:
>> - Add an fchost interface for bsg requests
>>
>> - Formalized the request/response structures that I expect
>> to have us stuff into the bsg cmd/sense data areas. These
>> are now genericized so we can essentially pass any kind of
>> transaction. It can be a request that has no transmit or
>> receive payload, and simply returns a response.
>>
>> - A new file was created, scsi_bsg_fc.h, which contains the
>> request/response data structures that should be shared
>> between the application and the kernel entities.
>>
>> - I stripped out some things that were in the request
>> structure that were actually LLD fields. Instead, I added
>> a dd_bsgsize structure to the template, so the transport
>> will allocate LLD work space along with the job structure.
>> I expect the missing fields to move to this area.
>>
>> - I've made a strong attempt at ensuring that the request
>> has all the information necessary for the LLD, so that
>> there is no need to have the LLD remap the transmit payload
>> to figure things out. Granted, this comes at the cost of
>> replicating some data items.
>>
>> Sven, I've added the CT information you needed as part of this.
>>
>> - I've renamed things quite a bit, hoping to make it clarity
>> better. The "service" struct is now a job. I still have
>> headaches with "request" (is it the blk request, or the job
>> request, or what..)
>>
>> - The CT/ELS response is a bit funky. I've noted that the
>> way Emulex returns a response, vs Qlogic is a bit different,
>> thus the 2 ways to indicate "reject".
>>
>> - fixed a couple of bugs in Seokmann's code, in the teardown,
>> error flows, request que dma settings, etc.
>>
>> - I added a "vendor_id" field to the scsi_host_template to
>> use when verifying that the recipient knows how to decode
>> vendor-specific message. I didn't do this with the netlink
>> things as I was prepping it to not break kabi in existing
>> and older kernels. But, I believe this is a good time to
>> add it.
>>
>> - I've started the Documentation/scsi/scsi_transport_fc.txt
>> documentation, but punted finishing it in lieu of sending
>> this RFC. I'm starting from Seokman's original emails and
>> will be updating for this reformat.
>>
>> I'm only starting to debug this, so user beware.
>>
>> I could really use some code review from Fujita or Boaz, to
>> make sure I'm calling the right blk_xx completion functions
>> relative to the setup flow, and to ensure that the "goose"
>> when I jump out while the rport is blocked is correct.
>>
>> Comments welcome
>>
>> -- james s
>>
>>
>>
>>
>>
>> Signed-off-by: James Smart <james.smart@emulex.com>
>>
>> ---
>>
>> Documentation/scsi/scsi_fc_transport.txt |   11
>> Documentation/scsi/scsi_mid_low_api.txt  |    5
>> drivers/scsi/scsi_transport_fc.c         |  614 ++++++++++++++++++++ 
>> ++++++++++-
>> include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
>> include/scsi/scsi_host.h                 |    9
>> include/scsi/scsi_transport_fc.h         |   52 ++
>> 6 files changed, 1009 insertions(+), 4 deletions(-)
>>
>>
>> diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/ 
>> Documentation/scsi/scsi_fc_transport.txt
>> --- a/Documentation/scsi/scsi_fc_transport.txt  2009-01-27  
>> 09:44:22.000000000 -0500
>> +++ b/Documentation/scsi/scsi_fc_transport.txt  2009-02-10  
>> 10:34:42.000000000 -0500
>> @@ -1,10 +1,11 @@
>>                             SCSI FC Tansport
>>                 =============================================
>>
>> -Date:  4/12/2007
>> +Date:  11/18/2008
>> Kernel Revisions for features:
>>  rports : <<TBS>>
>>  vports : 2.6.22 (? TBD)
>> +  bsg support : 2.6.29 (?TBD)
>>
>>
>> Introduction
>> @@ -15,6 +16,7 @@ The FC transport can be found at:
>>  drivers/scsi/scsi_transport_fc.c
>>  include/scsi/scsi_transport_fc.h
>>  include/scsi/scsi_netlink_fc.h
>> +  include/scsi/scsi_bsg_fc.h
>>
>> This file is found at Documentation/scsi/scsi_fc_transport.txt
>>
>> @@ -472,6 +474,13 @@ int
>> fc_vport_terminate(struct fc_vport *vport)
>>
>>
>> +FC BSG support (CT & ELS passthru, and more)
>> + 
>> = 
>> = 
>> = 
>> =====================================================================
>> +<< To Be Supplied >>
>> +
>> +
>> +
>> +
>> Credits
>> =======
>> The following people have contributed to this document:
>> diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/ 
>> Documentation/scsi/scsi_mid_low_api.txt
>> --- a/Documentation/scsi/scsi_mid_low_api.txt   2008-09-23  
>> 15:11:57.000000000 -0400
>> +++ b/Documentation/scsi/scsi_mid_low_api.txt   2009-02-10  
>> 10:34:42.000000000 -0500
>> @@ -1271,6 +1271,11 @@ of interest:
>>    hostdata[0]  - area reserved for LLD at end of struct Scsi_Host.  
>> Size
>>                   is set by the second argument (named 'xtr_bytes')  
>> to
>>                   scsi_host_alloc() or scsi_register().
>> +    vendor_id    - a unique value that identifies the vendor  
>> supplying
>> +                   the LLD for the Scsi_Host.  Used most often in  
>> validating
>> +                   vendor-specific message requests.  Value  
>> consists of an
>> +                   identifier type and a vendor-specific value.
>> +                   See scsi_netlink.h for a description of valid  
>> formats.
>>
>> The scsi_host structure is defined in include/scsi/scsi_host.h
>>
>> diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/ 
>> scsi_transport_fc.c
>> --- a/drivers/scsi/scsi_transport_fc.c  2009-01-27  
>> 09:44:31.000000000 -0500
>> +++ b/drivers/scsi/scsi_transport_fc.c  2009-02-10  
>> 14:58:39.000000000 -0500
>> @@ -35,6 +35,7 @@
>> #include <linux/netlink.h>
>> #include <net/netlink.h>
>> #include <scsi/scsi_netlink_fc.h>
>> +#include <scsi/scsi_bsg_fc.h>
>> #include "scsi_priv.h"
>> #include "scsi_transport_fc_internal.h"
>>
>> @@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
>> static int fc_vport_setup(struct Scsi_Host *shost, int channel,
>>       struct device *pdev, struct fc_vport_identifiers  *ids,
>>       struct fc_vport **vport);
>> +static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs  
>> *);
>> +static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
>> +static void fc_bsg_remove(struct request_queue *);
>> +static void fc_bsg_goose_queue(struct fc_rport *);
>>
>> /*
>> * Redefine so that we can have same named attributes in the
>> @@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
>>               return -ENOMEM;
>>       }
>>
>> +       fc_bsg_hostadd(shost, fc_host);
>> +       /* ignore any bsg add error - we just can't do sgio */
>> +
>> +       return 0;
>> +}
>> +
>> +static int fc_host_remove(struct transport_container *tc, struct  
>> device *dev,
>> +                        struct device *cdev)
>> +{
>> +       struct Scsi_Host *shost = dev_to_shost(dev);
>> +       struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
>> +
>> +       fc_bsg_remove(fc_host->rqst_q);
>>       return 0;
>> }
>>
>> static DECLARE_TRANSPORT_CLASS(fc_host_class,
>>                              "fc_host",
>>                              fc_host_setup,
>> -                              NULL,
>> +                              fc_host_remove,
>>                              NULL);
>>
>> /*
>> @@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
>>               scsi_flush_work(shost);
>>
>>       fc_terminate_rport_io(rport);
>> +
>>       /*
>>        * Cancel any outstanding timers. These should really exist
>>        * only when rmmod'ing the LLDD and we're asking for
>> @@ -2415,6 +2434,8 @@ fc_rport_final_delete(struct work_struct
>>           (i->f->dev_loss_tmo_callbk))
>>               i->f->dev_loss_tmo_callbk(rport);
>>
>> +       fc_bsg_remove(rport->rqst_q);
>> +
>>       transport_remove_device(dev);
>>       device_del(dev);
>>       transport_destroy_device(dev);
>> @@ -2502,6 +2523,9 @@ fc_rport_create(struct Scsi_Host *shost,
>>       transport_add_device(dev);
>>       transport_configure_device(dev);
>>
>> +       fc_bsg_rportadd(shost, rport);
>> +       /* ignore any bsg add error - we just can't do sgio */
>> +
>>       if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
>>               /* initiate a scan of the target */
>>               rport->flags |= FC_RPORT_SCAN_PENDING;
>> @@ -2666,6 +2690,8 @@ fc_remote_port_add(struct Scsi_Host *sho
>>                                       spin_unlock_irqrestore(shost- 
>> >host_lock,
>>                                                       flags);
>>
>> +                               fc_bsg_goose_queue(rport);
>> +
>>                               return rport;
>>                       }
>>               }
>> @@ -3351,6 +3377,592 @@ fc_vport_sched_delete(struct work_struct
>> }
>>
>>
>> +/*
>> + * BSG support
>> + */
>> +
>> +
>> +/**
>> + * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
>> + * @job:       fc_bsg_job that is to be torn down
>> + */
>> +static void
>> +fc_destroy_bsgjob(struct fc_bsg_job *job)
>> +{
>> +       unsigned long flags;
>> +
>> +       spin_lock_irqsave(&job->job_lock, flags);
>> +       if (job->ref_cnt) {
>> +               spin_unlock_irqrestore(&job->job_lock, flags);
>> +               return;
>> +       }
>> +       spin_unlock_irqrestore(&job->job_lock, flags);
>> +
>> +       put_device(job->dev);   /* release reference for the  
>> request */
>> +
>> +       kfree(job->request_payload.sg_list);
>> +       kfree(job->reply_payload.sg_list);
>> +       kfree(job);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_jobdone - completion routine for bsg requests that the  
>> LLD has
>> + *                  completed
>> + * @job:       fc_bsg_job that is complete
>> + */
>> +static void
>> +fc_bsg_jobdone(struct fc_bsg_job *job)
>> +{
>> +       struct request *req = job->req;
>> +       struct request *rsp = req->next_rq;
>> +       unsigned long flags;
>> +       unsigned rsp_len;
>> +       int err;
>> +
>> +       spin_lock_irqsave(&job->job_lock, flags);
>> +       job->state_flags |= FC_RQST_STATE_DONE;
>> +       job->ref_cnt--;
>> +       spin_unlock_irqrestore(&job->job_lock, flags);
>> +
>> +       err = job->req->errors = job->reply->result;
>> +       if (err < 0)
>> +               /* we're only returning the result field in the  
>> reply */
>> +               job->req->sense_len = sizeof(uint32_t);
>> +       else
>> +               job->req->sense_len = job->reply_len;
>> +
>> +       if (rsp) {
>> +               rsp_len = blk_rq_bytes(rsp);
>> +               BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
>> +               /* set reply (bidi) residual */
>> +               rsp->data_len = (rsp_len - job->reply- 
>> >reply_payload_rcv_len);
>> +       }
>> +
>> +       /* we assume all request payload was transferred */
>> +       blk_end_request(req, err, blk_rq_bytes(req));
>> +
>> +       fc_destroy_bsgjob(job);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_job_timeout - handler for when a bsg request timesout
>> + * @req:       request that timed out
>> + */
>> +static enum blk_eh_timer_return
>> +fc_bsg_job_timeout(struct request *req)
>> +{
>> +       struct fc_bsg_job *job = (void *) req->special;
>> +       struct Scsi_Host *shost = job->shost;
>> +       struct fc_internal *i = to_fc_internal(shost->transportt);
>> +       unsigned long flags;
>> +       int err = 0, done = 0;
>> +
>> +       if (job->rport && job->rport->port_state ==  
>> FC_PORTSTATE_BLOCKED)
>> +               return BLK_EH_RESET_TIMER;
>> +
>> +       spin_lock_irqsave(&job->job_lock, flags);
>> +       if (job->state_flags & FC_RQST_STATE_DONE)
>> +               done = 1;
>> +       else
>> +               job->ref_cnt++;
>> +       spin_unlock_irqrestore(&job->job_lock, flags);
>> +
>> +       if (!done && i->f->bsg_timeout) {
>> +               /* call LLDD to abort the i/o as it has timed out */
>> +               err = i->f->bsg_timeout(job);
>> +               if (err)
>> +                       printk(KERN_ERR "ERROR: FC BSG request  
>> timeout - LLD "
>> +                               "abort failed with status %d\n",  
>> err);
>> +       }
>> +
>> +       if (!done) {
>> +               spin_lock_irqsave(&job->job_lock, flags);
>> +               job->ref_cnt--;
>> +               spin_unlock_irqrestore(&job->job_lock, flags);
>> +               fc_destroy_bsgjob(job);
>> +       }
>> +
>> +       /* the blk_end_sync_io() doesn't check the error */
>> +       return BLK_EH_HANDLED;
>> +}
>> +
>> +
>> +
>> +static int
>> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
>> +{
>> +       size_t sz = (sizeof(struct scatterlist) * req- 
>> >nr_phys_segments);
>> +
>> +       BUG_ON(!req->nr_phys_segments);
>> +
>> +       buf->sg_list = kzalloc(sz, GFP_KERNEL);
>> +       if (!buf->sg_list)
>> +               return -ENOMEM;
>> +       sg_init_table(buf->sg_list, req->nr_phys_segments);
>> +       buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
>> +       buf->payload_len = req->data_len;
>> +       return 0;
>> +}
>> +
>> +
>> +/**
>> + * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for  
>> the
>> + *                   bsg request
>> + * @shost:     SCSI Host corresponding to the bsg object
>> + * @rport:     (optional) FC Remote Port corresponding to the bsg  
>> object
>> + * @req:       BSG request that needs a job structure
>> + */
>> +static int
>> +fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
>> +       struct request *req)
>> +{
>> +       struct fc_internal *i = to_fc_internal(shost->transportt);
>> +       struct request *rsp = req->next_rq;
>> +       struct fc_bsg_job *job;
>> +       int ret;
>> +
>> +       BUG_ON(req->special);
>> +
>> +       job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
>> +                       GFP_KERNEL);
>> +       if (!job)
>> +               return -ENOMEM;
>> +
>> +       /*
>> +        * Note: this is a bit silly.
>> +        * The request gets formatted as a SGIO v4 ioctl request,  
>> which
>> +        * then gets reformatted as a blk request, which then gets
>> +        * reformatted as a fc bsg request. And on completion, we  
>> have
>> +        * to wrap return results such that SGIO v4 thinks it was a  
>> scsi
>> +        * status.  I hope this was all worth it.
>> +        */
>> +
>> +       req->special = job;
>> +       job->shost = shost;
>> +       job->rport = rport;
>> +       job->req = req;
>> +       if (i->f->dd_bsg_size)
>> +               job->dd_data = (void *)&job[1];
>> +       spin_lock_init(&job->job_lock);
>> +       job->request = (struct fc_bsg_request *)req->cmd;
>> +       job->request_len = req->cmd_len;
>> +       job->reply = req->sense;
>> +       job->reply_len = SCSI_SENSE_BUFFERSIZE; /* Size of sense  
>> buffer
>> +                                                * allocated */
>> +       if (req->bio) {
>> +               ret = fc_bsg_map_buffer(&job->request_payload, req);
>> +               if (ret)
>> +                       goto failjob_rls_job;
>> +       }
>> +       if (rsp && rsp->bio) {
>> +               ret = fc_bsg_map_buffer(&job->reply_payload, rsp);
>> +               if (ret)
>> +                       goto failjob_rls_rqst_payload;
>> +       }
>> +       job->job_done = fc_bsg_jobdone;
>> +       if (rport)
>> +               job->dev = &rport->dev;
>> +       else
>> +               job->dev = &shost->shost_gendev;
>> +       get_device(job->dev);           /* take a reference for the  
>> request */
>> +
>> +       job->ref_cnt = 1;
>> +
>> +       return 0;
>> +
>> +
>> +failjob_rls_rqst_payload:
>> +       kfree(job->request_payload.sg_list);
>> +failjob_rls_job:
>> +       kfree(job);
>> +       return -ENOMEM;
>> +}
>> +
>> +
>> +enum fc_dispatch_result {
>> +       FC_DISPATCH_BREAK,      /* on return, q is locked, break  
>> from q loop */
>> +       FC_DISPATCH_LOCKED,     /* on return, q is locked, continue  
>> on */
>> +       FC_DISPATCH_UNLOCKED,   /* on return, q is unlocked,  
>> continue on */
>> +};
>> +
>> +
>> +/**
>> + * fc_bsg_host_dispatch - process fc host bsg requests and  
>> dispatch to LLDD
>> + * @shost:     scsi host rport attached to
>> + * @job:       bsg job to be processed
>> + */
>> +static enum fc_dispatch_result
>> +fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host  
>> *shost,
>> +                        struct fc_bsg_job *job)
>> +{
>> +       struct fc_internal *i = to_fc_internal(shost->transportt);
>> +       int cmdlen = sizeof(uint32_t);  /* start with length of  
>> msgcode */
>> +       int ret;
>> +
>> +       /* Validate the host command */
>> +       switch (job->request->msgcode) {
>> +       case FC_BSG_HST_ADD_RPORT:
>> +               cmdlen += sizeof(struct fc_bsg_host_add_rport);
>> +               break;
>> +
>> +       case FC_BSG_HST_DEL_RPORT:
>> +               cmdlen += sizeof(struct fc_bsg_host_del_rport);
>> +               break;
>> +
>> +       case FC_BSG_HST_ELS_NOLOGIN:
>> +               cmdlen += sizeof(struct fc_bsg_host_els);
>> +               /* there better be a xmt and rcv payloads */
>> +               if ((!job->request_payload.payload_len) ||
>> +                   (!job->reply_payload.payload_len)) {
>> +                       ret = -EINVAL;
>> +                       goto fail_host_msg;
>> +               }
>> +               break;
>> +
>> +       case FC_BSG_HST_CT:
>> +               cmdlen += sizeof(struct fc_bsg_host_ct);
>> +               /* there better be xmt and rcv payloads */
>> +               if ((!job->request_payload.payload_len) ||
>> +                   (!job->reply_payload.payload_len)) {
>> +                       ret = -EINVAL;
>> +                       goto fail_host_msg;
>> +               }
>> +               break;
>> +
>> +       case FC_BSG_HST_VENDOR:
>> +               cmdlen += sizeof(struct fc_bsg_host_vendor);
>> +               if ((shost->hostt->vendor_id == 0L) ||
>> +                   (job->request->rqst_data.h_vendor.vendor_id !=
>> +                       shost->hostt->vendor_id)) {
>> +                       ret = -ESRCH;
>> +                       goto fail_host_msg;
>> +               }
>> +               break;
>> +
>> +       default:
>> +               ret = -EBADR;
>> +               goto fail_host_msg;
>> +       }
>> +
>> +       /* check if we really have all the request data needed */
>> +       if (job->request_len < cmdlen) {
>> +               ret = -ENOMSG;
>> +               goto fail_host_msg;
>> +       }
>> +
>> +       ret = i->f->bsg_request(job);
>> +       if (!ret)
>> +               return FC_DISPATCH_UNLOCKED;
>> +
>> +fail_host_msg:
>> +       /* return the errno failure code as the only status */
>> +       BUG_ON(job->reply_len < sizeof(uint32_t));
>> +       job->reply->result = ret;
>> +       job->reply_len = sizeof(uint32_t);
>> +       fc_bsg_jobdone(job);
>> +       return FC_DISPATCH_UNLOCKED;
>> +}
>> +
>> +
>> +/*
>> + * fc_bsg_goose_queue - restart rport queue in case it was stopped
>> + * @rport:     rport to be restarted
>> + */
>> +static void
>> +fc_bsg_goose_queue(struct fc_rport *rport)
>> +{
>> +       int flagset;
>> +
>> +       if (!rport->rqst_q)
>> +               return;
>> +
>> +       get_device(&rport->dev);
>> +
>> +       spin_lock(rport->rqst_q->queue_lock);
>> +       flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
>> >queue_flags) &&
>> +                 !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
>> >queue_flags);
>> +       if (flagset)
>> +               queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
>> +       __blk_run_queue(rport->rqst_q);
>> +       if (flagset)
>> +               queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
>> +       spin_unlock(rport->rqst_q->queue_lock);
>> +
>> +       put_device(&rport->dev);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_rport_dispatch - process rport bsg requests and dispatch  
>> to LLDD
>> + * @shost:     scsi host rport attached to
>> + * @rport:     rport request destined to
>> + * @job:       bsg job to be processed
>> + */
>> +static enum fc_dispatch_result
>> +fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host  
>> *shost,
>> +                        struct fc_rport *rport, struct fc_bsg_job  
>> *job)
>> +{
>> +       struct fc_internal *i = to_fc_internal(shost->transportt);
>> +       int cmdlen = sizeof(uint32_t);  /* start with length of  
>> msgcode */
>> +       int ret;
>> +
>> +       /* Validate the rport command */
>> +       switch (job->request->msgcode) {
>> +       case FC_BSG_RPT_ELS:
>> +               cmdlen += sizeof(struct fc_bsg_rport_els);
>> +               goto check_bidi;
>> +
>> +       case FC_BSG_RPT_CT:
>> +               cmdlen += sizeof(struct fc_bsg_rport_ct);
>> +check_bidi:
>> +               /* there better be xmt and rcv payloads */
>> +               if ((!job->request_payload.payload_len) ||
>> +                   (!job->reply_payload.payload_len)) {
>> +                       ret = -EINVAL;
>> +                       goto fail_rport_msg;
>> +               }
>> +               break;
>> +       default:
>> +               ret = -EBADR;
>> +               goto fail_rport_msg;
>> +       }
>> +
>> +       /* check if we really have all the request data needed */
>> +       if (job->request_len < cmdlen) {
>> +               ret = -ENOMSG;
>> +               goto fail_rport_msg;
>> +       }
>> +
>> +       ret = i->f->bsg_request(job);
>> +       if (!ret)
>> +               return FC_DISPATCH_UNLOCKED;
>> +
>> +fail_rport_msg:
>> +       /* return the errno failure code as the only status */
>> +       BUG_ON(job->reply_len < sizeof(uint32_t));
>> +       job->reply->result = ret;
>> +       job->reply_len = sizeof(uint32_t);
>> +       fc_bsg_jobdone(job);
>> +       return FC_DISPATCH_UNLOCKED;
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_request_handler - generic handler for bsg requests
>> + * @q:         request queue to manage
>> + * @shost:     Scsi_Host related to the bsg object
>> + * @rport:     FC remote port related to the bsg object (optional)
>> + * @dev:       device structure for bsg object
>> + */
>> +static void
>> +fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host  
>> *shost,
>> +                      struct fc_rport *rport, struct device *dev)
>> +{
>> +       struct request *req;
>> +       struct fc_bsg_job *job;
>> +       enum fc_dispatch_result ret;
>> +
>> +       if (!get_device(dev))
>> +               return;
>> +
>> +       while (!blk_queue_plugged(q)) {
>> +               req = elv_next_request(q);
>> +               if (!req)
>> +                       break;
>> +
>> +               if (rport && (rport->port_state ==  
>> FC_PORTSTATE_BLOCKED))
>> +                               break;
>> +
>> +               blkdev_dequeue_request(req);
>> +
>> +               if (rport && (rport->port_state !=  
>> FC_PORTSTATE_ONLINE)) {
>> +                       req->errors = -ENXIO;
>> +                       spin_unlock_irq(q->queue_lock);
>> +                       blk_end_request(req, -ENXIO,  
>> blk_rq_bytes(req));
>> +                       spin_lock_irq(q->queue_lock);
>> +                       continue;
>> +               }
>> +
>> +               spin_unlock_irq(q->queue_lock);
>> +
>> +               ret = fc_req_to_bsgjob(shost, rport, req);
>> +               if (ret) {
>> +                       req->errors = ret;
>> +                       blk_end_request(req, ret, blk_rq_bytes(req));
>> +                       spin_lock_irq(q->queue_lock);
>> +                       continue;
>> +               }
>> +
>> +               job = req->special;
>> +
>> +               /* check if we have the msgcode value at least */
>> +               if (job->request_len < sizeof(uint32_t)) {
>> +                       BUG_ON(job->reply_len < sizeof(uint32_t));
>> +                       job->reply->result = -ENOMSG;
>> +                       job->reply_len = sizeof(uint32_t);
>> +                       fc_bsg_jobdone(job);
>> +                       spin_lock_irq(q->queue_lock);
>> +                       continue;
>> +               }
>> +
>> +               /* the dispatch routines will unlock the queue_lock  
>> */
>> +               if (rport)
>> +                       ret = fc_bsg_rport_dispatch(q, shost,  
>> rport, job);
>> +               else
>> +                       ret = fc_bsg_host_dispatch(q, shost, job);
>> +
>> +               /* did dispatcher hit state that can't process any  
>> more */
>> +               if (ret == FC_DISPATCH_BREAK)
>> +                       break;
>> +
>> +               /* did dispatcher had released the lock */
>> +               if (ret == FC_DISPATCH_UNLOCKED)
>> +                       spin_lock_irq(q->queue_lock);
>> +       }
>> +
>> +       spin_unlock_irq(q->queue_lock);
>> +       put_device(dev);
>> +       spin_lock_irq(q->queue_lock);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_host_handler - handler for bsg requests for a fc host
>> + * @q:         fc host request queue
>> + */
>> +static void
>> +fc_bsg_host_handler(struct request_queue *q)
>> +{
>> +       struct Scsi_Host *shost = q->queuedata;
>> +
>> +       fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_rport_handler - handler for bsg requests for a fc rport
>> + * @q:         rport request queue
>> + */
>> +static void
>> +fc_bsg_rport_handler(struct request_queue *q)
>> +{
>> +       struct fc_rport *rport = q->queuedata;
>> +       struct Scsi_Host *shost = rport_to_shost(rport);
>> +
>> +       fc_bsg_request_handler(q, shost, rport, &rport->dev);
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_hostadd - Create and add the bsg hooks so we can receive  
>> requests
>> + * @shost:     shost for fc_host
>> + * @fc_host:   fc_host adding the structures to
>> + */
>> +static int
>> +fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs  
>> *fc_host)
>> +{
>> +       struct device *dev = &shost->shost_gendev;
>> +       struct fc_internal *i = to_fc_internal(shost->transportt);
>> +       struct request_queue *q;
>> +       int err;
>> +       char bsg_name[BUS_ID_SIZE]; /*20*/
>> +
>> +       fc_host->rqst_q = NULL;
>> +
>> +       if (!i->f->bsg_request)
>> +               return -ENOTSUPP;
>> +
>> +       snprintf(bsg_name, sizeof(bsg_name),
>> +                "fc_host%d", shost->host_no);
>> +
>> +       q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
>> +       if (!q) {
>> +               printk(KERN_ERR "fc_host%d: bsg interface failed to "
>> +                               "initialize - no request queue\n",
>> +                                shost->host_no);
>> +               return -ENOMEM;
>> +       }
>> +
>> +       q->queuedata = shost;
>> +       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
>> +       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
>> +       blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
>> +
>> +       err = bsg_register_queue(q, dev, bsg_name, NULL);
>> +       if (err) {
>> +               printk(KERN_ERR "fc_host%d: bsg interface failed to "
>> +                               "initialize - register queue\n",
>> +                               shost->host_no);
>> +               blk_cleanup_queue(q);
>> +               return err;
>> +       }
>> +
>> +       fc_host->rqst_q = q;
>> +       return 0;
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_rportadd - Create and add the bsg hooks so we can  
>> receive requests
>> + * @shost:     shost that rport is attached to
>> + * @rport:     rport that the bsg hooks are being attached to
>> + */
>> +static int
>> +fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
>> +{
>> +       struct device *dev = &rport->dev;
>> +       struct fc_internal *i = to_fc_internal(shost->transportt);
>> +       struct request_queue *q;
>> +       int err;
>> +
>> +       rport->rqst_q = NULL;
>> +
>> +       if (!i->f->bsg_request)
>> +               return -ENOTSUPP;
>> +
>> +       q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
>> +       if (!q) {
>> +               printk(KERN_ERR "%s: bsg interface failed to "
>> +                               "initialize - no request queue\n",
>> +                                dev->bus_id);
>> +               return -ENOMEM;
>> +       }
>> +
>> +       q->queuedata = rport;
>> +       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
>> +       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
>> +       blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
>> +
>> +       err = bsg_register_queue(q, dev, NULL, NULL);
>> +       if (err) {
>> +               printk(KERN_ERR "%s: bsg interface failed to "
>> +                               "initialize - register queue\n",
>> +                                dev->bus_id);
>> +               blk_cleanup_queue(q);
>> +               return err;
>> +       }
>> +
>> +       rport->rqst_q = q;
>> +       return 0;
>> +}
>> +
>> +
>> +/**
>> + * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
>> + * @q: the request_queue that is to be torn down.
>> + */
>> +static void
>> +fc_bsg_remove(struct request_queue *q)
>> +{
>> +       if (q) {
>> +               bsg_unregister_queue(q);
>> +               blk_cleanup_queue(q);
>> +       }
>> +}
>> +
>> +
>> /* Original Author:  Martin Hicks */
>> MODULE_AUTHOR("James Smart");
>> MODULE_DESCRIPTION("FC Transport Attributes");
>> diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
>> --- a/include/scsi/scsi_bsg_fc.h        1969-12-31  
>> 19:00:00.000000000 -0500
>> +++ b/include/scsi/scsi_bsg_fc.h        2009-02-10  
>> 13:26:17.000000000 -0500
>> @@ -0,0 +1,322 @@
>> +/*
>> + *  FC Transport BSG Interface
>> + *
>> + *  Copyright (C) 2008   James Smart, Emulex Corporation
>> + *
>> + *  This program is free software; you can redistribute it and/or  
>> modify
>> + *  it under the terms of the GNU General Public License as  
>> published by
>> + *  the Free Software Foundation; either version 2 of the License,  
>> or
>> + *  (at your option) any later version.
>> + *
>> + *  This program is distributed in the hope that it will be useful,
>> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + *  GNU General Public License for more details.
>> + *
>> + *  You should have received a copy of the GNU General Public  
>> License
>> + *  along with this program; if not, write to the Free Software
>> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA   
>> 02111-1307  USA
>> + *
>> + */
>> +
>> +#ifndef SCSI_BSG_FC_H
>> +#define SCSI_BSG_FC_H
>> +
>> +/*
>> + * This file intended to be included by both kernel and user space
>> + */
>> +
>> +#include <scsi/scsi.h>
>> +
>> +/*
>> + * FC Transport SGIO v4 BSG Message Support
>> + */
>> +
>> +/* Default BSG request timeout (in seconds) */
>> +#define FC_DEFAULT_BSG_TIMEOUT         (10 * HZ)
>> +
>> +
>> +/*
>> + * Request Message Codes supported by the FC Transport
>> + */
>> +
>> +/* define the class masks for the message codes */
>> +#define FC_BSG_CLS_MASK                0xF0000000      /* find  
>> object class */
>> +#define FC_BSG_HST_MASK                0x80000000      /* fc host  
>> class */
>> +#define FC_BSG_RPT_MASK                0x40000000      /* fc rport  
>> class */
>> +
>> +       /* fc_host Message Codes */
>> +#define FC_BSG_HST_ADD_RPORT           (FC_BSG_HST_MASK |  
>> 0x00000001)
>> +#define FC_BSG_HST_DEL_RPORT           (FC_BSG_HST_MASK |  
>> 0x00000002)
>> +#define FC_BSG_HST_ELS_NOLOGIN         (FC_BSG_HST_MASK |  
>> 0x00000003)
>> +#define FC_BSG_HST_CT                  (FC_BSG_HST_MASK |  
>> 0x00000004)
>> +#define FC_BSG_HST_VENDOR              (FC_BSG_HST_MASK |  
>> 0x000000FF)
>> +
>> +       /* fc_rport Message Codes */
>> +#define FC_BSG_RPT_ELS                 (FC_BSG_RPT_MASK |  
>> 0x00000001)
>> +#define FC_BSG_RPT_CT                  (FC_BSG_RPT_MASK |  
>> 0x00000002)
>> +
>> +
>> +
>> +/*
>> + * FC Address Identifiers in Message Structures :
>> + *
>> + *   Whenever a command payload contains a FC Address Identifier
>> + *   (aka port_id), the value is effectively in big-endian
>> + *   order, thus the array elements are decoded as follows:
>> + *     element [0] is bits 23:16 of the FC Address Identifier
>> + *     element [1] is bits 15:8 of the FC Address Identifier
>> + *     element [2] is bits 7:0 of the FC Address Identifier
>> + */
>> +
>> +
>> +/*
>> + * FC Host Messages
>> + */
>> +
>> +/* FC_BSG_HST_ADDR_PORT : */
>> +
>> +/* Request:
>> + * This message requests the FC host to login to the remote port
>> + * at the specified N_Port_Id.  The remote port is to be enumerated
>> + * with the transport upon completion of the login.
>> + */
>> +struct fc_bsg_host_add_rport {
>> +       uint8_t         reserved;
>> +
>> +       /* FC Address Identier of the remote port to login to */
>> +       uint8_t         port_id[3];
>> +};
>> +
>> +/* Response:
>> + * There is no additional response data - fc_bsg_reply->result is  
>> sufficient
>> + */
>> +
>> +
>> +/* FC_BSG_HST_DEL_RPORT : */
>> +
>> +/* Request:
>> + * This message requests the FC host to remove an enumerated
>> + * remote port and to terminate the login to it.
>> + *
>> + * Note: The driver is free to reject this request if it desires to
>> + * remain logged in with the remote port.
>> + */
>> +struct fc_bsg_host_del_rport {
>> +       uint8_t         reserved;
>> +
>> +       /* FC Address Identier of the remote port to logout of */
>> +       uint8_t         port_id[3];
>> +};
>> +
>> +/* Response:
>> + * There is no additional response data - fc_bsg_reply->result is  
>> sufficient
>> + */
>> +
>> +
>> +/* FC_BSG_HST_ELS_NOLOGIN : */
>> +
>> +/* Request:
>> + * This message requests the FC_Host to send an ELS to a specific
>> + * N_Port_ID. The host does not need to log into the remote port,
>> + * nor does it need to enumerate the rport for further traffic
>> + * (although, the FC host is free to do so if it desires).
>> + */
>> +struct fc_bsg_host_els {
>> +       /*
>> +        * ELS Command Code being sent (must be the same as byte 0
>> +        * of the payload)
>> +        */
>> +       uint8_t         command_code;
>> +
>> +       /* FC Address Identier of the remote port to send the ELS  
>> to */
>> +       uint8_t         port_id[3];
>> +};
>> +
>> +/* Response:
>> + */
>> +/* fc_bsg_ctels_reply->status values */
>> +#define FC_CTELS_STATUS_OK     0x00000000
>> +#define FC_CTELS_STATUS_REJECT 0x00000001
>> +#define FC_CTELS_STATUS_P_RJT  0x00000002
>> +#define FC_CTELS_STATUS_F_RJT  0x00000003
>> +#define FC_CTELS_STATUS_P_BSY  0x00000004
>> +#define FC_CTELS_STATUS_F_BSY  0x00000006
>> +struct fc_bsg_ctels_reply {
>> +       /*
>> +        * Note: An ELS LS_RJT may be reported in 2 ways:
>> +        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
>> +        *     is to look into the ELS receive payload to determine
>> +        *     LS_ACC or LS_RJT (by contents of word 0). The reject
>> +        *     data will be in word 1.
>> +        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
>> +        *     rjt_data field will contain valid data.
>> +        *
>> +        * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK,  
>> and
>> +        *   the receive payload word 0 indicates LS_ACC
>> +        *   (e.g. value is 0x02xxxxxx).
>> +        *
>> +        * Note: Similarly, a CT Reject may be reported in 2 ways:
>> +        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
>> +        *     is to look into the CT receive payload to determine
>> +        *     Accept or Reject (by contents of word 2). The reject
>> +        *     data will be in word 3.
>> +        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
>> +        *     rjt_data field will contain valid data.
>> +        *
>> +        * Note: x_RJT/BSY status will indicae that the rjt_data  
>> field
>> +        *   is valid and contains the reason/explanation values.
>> +        */
>> +       uint32_t        status;         /* See FC_CTELS_STATUS_xxx */
>> +
>> +       /* valid if status is not FC_CTELS_STATUS_OK */
>> +       struct  {
>> +               uint8_t action;         /* fragment_id for CT  
>> REJECT */
>> +               uint8_t reason_code;
>> +               uint8_t reason_explanation;
>> +               uint8_t vendor_unique;
>> +       } rjt_data;
>> +};
>> +
>> +
>> +/* FC_BSG_HST_CT : */
>> +
>> +/* Request:
>> + * This message requests that a CT Request be performed with the
>> + * indicated N_Port_ID. The driver is responsible for logging in  
>> with
>> + * the fabric and/or N_Port_ID, etc as per FC rules. This request  
>> does
>> + * not mandate that the driver must enumerate the destination in the
>> + * transport. The driver is allowed to decide whether to enumerate  
>> it,
>> + * and whether to tear it down after the request.
>> + */
>> +struct fc_bsg_host_ct {
>> +       uint8_t         reserved;
>> +
>> +       /* FC Address Identier of the remote port to send the ELS  
>> to */
>> +       uint8_t         port_id[3];
>> +
>> +       /*
>> +        * We need words 0-2 of the generic preamble for the LLD's
>> +        */
>> +       uint32_t        preamble_word0; /* revision & IN_ID */
>> +       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  
>> Options, Rsvd */
>> +       uint32_t        preamble_word2; /* Cmd Code, Max Size */
>> +
>> +};
>> +/* Response:
>> + *
>> + * The reply structure is an fc_bsg_ctels_reply structure
>> + */
>> +
>> +
>> +/* FC_BSG_HST_VENDOR : */
>> +
>> +/* Request:
>> + * Note: When specifying vendor_id, be sure to read the Vendor  
>> Type and ID
>> + *   formatting requirements specified in scsi_netlink.h
>> + */
>> +struct fc_bsg_host_vendor {
>> +       /*
>> +        * Identifies the vendor that the message is formatted for.  
>> This
>> +        * should be the recipient of the message.
>> +        */
>> +       uint64_t vendor_id;
>> +
>> +       /* start of vendor command area */
>> +       uint32_t vendor_cmd[0];
>> +};
>> +
>> +/* Response:
>> + */
>> +struct fc_bsg_host_vendor_reply {
>> +       /* start of vendor response area */
>> +       uint32_t vendor_rsp[0];
>> +};
>> +
>> +
>> +
>> +/*
>> + * FC Remote Port Messages
>> + */
>> +
>> +/* FC_BSG_RPT_ELS : */
>> +
>> +/* Request:
>> + * This message requests that an ELS be performed with the rport.
>> + */
>> +struct fc_bsg_rport_els {
>> +       /*
>> +        * ELS Command Code being sent (must be the same as
>> +        * byte 0 of the payload)
>> +        */
>> +       uint8_t els_code;
>> +};
>> +
>> +/* Response:
>> + *
>> + * The reply structure is an fc_bsg_ctels_reply structure
>> + */
>> +
>> +
>> +/* FC_BSG_RPT_CT : */
>> +
>> +/* Request:
>> + * This message requests that a CT Request be performed with the  
>> rport.
>> + */
>> +struct fc_bsg_rport_ct {
>> +       /*
>> +        * We need words 0-2 of the generic preamble for the LLD's
>> +        */
>> +       uint32_t        preamble_word0; /* revision & IN_ID */
>> +       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  
>> Options, Rsvd */
>> +       uint32_t        preamble_word2; /* Cmd Code, Max Size */
>> +};
>> +/* Response:
>> + *
>> + * The reply structure is an fc_bsg_ctels_reply structure
>> + */
>> +
>> +
>> +
>> +
>> +/* request (CDB) structure of the sg_io_v4 */
>> +struct fc_bsg_request {
>> +       uint32_t msgcode;
>> +       union {
>> +               struct fc_bsg_host_add_rport    h_addrport;
>> +               struct fc_bsg_host_del_rport    h_delrport;
>> +               struct fc_bsg_host_els          h_els;
>> +               struct fc_bsg_host_ct           h_ct;
>> +               struct fc_bsg_host_vendor       h_vendor;
>> +
>> +               struct fc_bsg_rport_els         r_els;
>> +               struct fc_bsg_rport_ct          r_ct;
>> +       } rqst_data;
>> +};
>> +
>> +
>> +/* response (request sense data) structure of the sg_io_v4 */
>> +struct fc_bsg_reply {
>> +       /*
>> +        * The completion result. Result exists in two forms:
>> +        *  if negative, it is an -Exxx system errno value. There  
>> will
>> +        *    be no further reply information supplied.
>> +        *  else, it's the 4-byte scsi error result, with driver,  
>> host,
>> +        *    msg and status fields. The per-msgcode reply structure
>> +        *    will contain valid data.
>> +        */
>> +       uint32_t result;
>> +
>> +       /* If there was reply_payload, how much was recevied ? */
>> +       uint32_t reply_payload_rcv_len;
>> +
>> +       union {
>> +               struct fc_bsg_host_vendor_reply         vendor_reply;
>> +
>> +               struct fc_bsg_ctels_reply               ctels_reply;
>> +       } reply_data;
>> +};
>> +
>> +
>> +#endif /* SCSI_BSG_FC_H */
>> +
>> diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
>> --- a/include/scsi/scsi_host.h  2008-10-18 10:32:54.000000000 -0400
>> +++ b/include/scsi/scsi_host.h  2009-02-10 10:34:42.000000000 -0500
>> @@ -478,6 +478,15 @@ struct scsi_host_template {
>>        * module_init/module_exit.
>>        */
>>       struct list_head legacy_hosts;
>> +
>> +       /*
>> +        * Vendor Identifier associated with the host
>> +        *
>> +        * Note: When specifying vendor_id, be sure to read the
>> +        *   Vendor Type and ID formatting requirements specified in
>> +        *   scsi_netlink.h
>> +        */
>> +       u64 vendor_id;
>> };
>>
>> /*
>> diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/ 
>> scsi_transport_fc.h
>> --- a/include/scsi/scsi_transport_fc.h  2009-01-27  
>> 09:44:40.000000000 -0500
>> +++ b/include/scsi/scsi_transport_fc.h  2009-02-10  
>> 13:12:20.000000000 -0500
>> @@ -33,7 +33,6 @@
>>
>> struct scsi_transport_template;
>>
>> -
>> /*
>> * FC Port definitions - Following FC HBAAPI guidelines
>> *
>> @@ -352,6 +351,7 @@ struct fc_rport {   /* aka fc_starget_attr
>>       struct delayed_work fail_io_work;
>>       struct work_struct stgt_delete_work;
>>       struct work_struct rport_delete_work;
>> +       struct request_queue *rqst_q;   /* bsg support */
>> } __attribute__((aligned(sizeof(unsigned long))));
>>
>> /* bit field values for struct fc_rport "flags" field: */
>> @@ -514,6 +514,9 @@ struct fc_host_attrs {
>>       struct workqueue_struct *work_q;
>>       char devloss_work_q_name[20];
>>       struct workqueue_struct *devloss_work_q;
>> +
>> +       /* bsg support */
>> +       struct request_queue *rqst_q;
>> };
>>
>> #define shost_to_fc_host(x) \
>> @@ -579,6 +582,47 @@ struct fc_host_attrs {
>>       (((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
>>
>>
>> +struct fc_bsg_buffer {
>> +       unsigned int payload_len;
>> +       int sg_cnt;
>> +       struct scatterlist *sg_list;
>> +};
>> +
>> +/* Values for fc_bsg_job->state_flags (bitflags) */
>> +#define FC_RQST_STATE_INPROGRESS       0
>> +#define FC_RQST_STATE_DONE             1
>> +
>> +struct fc_bsg_job {
>> +       struct Scsi_Host *shost;
>> +       struct fc_rport *rport;
>> +       struct device *dev;
>> +       struct request *req;
>> +       spinlock_t job_lock;
>> +       unsigned int state_flags;
>> +       unsigned int ref_cnt;
>> +       void (*job_done)(struct fc_bsg_job *);
>> +
>> +       struct fc_bsg_request *request;
>> +       struct fc_bsg_reply *reply;
>> +       unsigned int request_len;
>> +       unsigned int reply_len;
>> +       /*
>> +        * On entry : reply_len indicates the buffer size allocated  
>> for
>> +        * the reply.
>> +        *
>> +        * Upon completion : the message handler must set reply_len
>> +        *  to indicates the size of the reply to be returned to the
>> +        *  caller.
>> +        */
>> +
>> +       /* DMA payloads for the request/response */
>> +       struct fc_bsg_buffer request_payload;
>> +       struct fc_bsg_buffer reply_payload;
>> +
>> +       void *dd_data;                  /* Used for driver-specific  
>> storage */
>> +};
>> +
>> +
>> /* The functions by which the transport class and the driver  
>> communicate */
>> struct fc_function_template {
>>       void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
>> @@ -614,9 +658,14 @@ struct fc_function_template {
>>       int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64,  
>> int);
>>       int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
>>
>> +       /* bsg support */
>> +       int     (*bsg_request)(struct fc_bsg_job *);
>> +       int     (*bsg_timeout)(struct fc_bsg_job *);
>> +
>>       /* allocation lengths for host-specific data */
>>       u32                             dd_fcrport_size;
>>       u32                             dd_fcvport_size;
>> +       u32                             dd_bsg_size;
>>
>>       /*
>>        * The driver sets these to tell the transport class it
>> @@ -737,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
>>       vport->vport_state = new_state;
>> }
>>
>> -
>> struct scsi_transport_template *fc_attach_transport(
>>                       struct fc_function_template *);
>> void fc_release_transport(struct scsi_transport_template *);
>>
>>
>

Re: [RFC] FC pass thru - Rev V

[James Smart]

Seokmann Ju wrote:
> Is there something that I need to be cautious in traveling these code
> path?

as mentioned in the original post... it has yet to be fully tested, so 
expect issues...

-- james s

Re: [RFC] FC pass thru - Rev V

[FUJITA Tomonori]

On Thu, 19 Feb 2009 18:33:43 -0800
Seokmann Ju <seokmann.ju@qlogic.com> wrote:

> 
> On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:
> 
> >
> > On Feb 11, 2009, at 7:13 AM, James Smart wrote:
> >
> >> Trying to kick-start this again...
> >> I've updated the prior RFC with the comments from Seokmann,
> >> SvenFujita, and Boaz. I would still like review on the
> >> blk_xxx completion calls in the std and error paths.
> > I'm looking the driver changes to match it with latest updates in  
> > the transport layer.
> > The driver with the changes should be available within a couple of  
> > weeks.
> When the FC transport layer calls fc_bsg_map_buffer(), it eventually  
> calls
> blk_rq_map_sg() to build sg_list for mapped bio in the request  
> structure.
> I'm having trouble, however, in getting the sg_list for both dout_xferp
> and din_xferp, which is mapped into the bio in the request structure.
> With the pass-thru - Rev V, the sg_list returned from the  
> fc_bsg_map_buffer()
> seems not right, particularly, the dma_address and dma_length fields  
> are NULL.
> The sg_cnt and payload_len of the request_payload/reply_payload are  
> correctly
> set by the function, though.

I've not read the code yet but the above behavior doesn't sound a bug.

dma_map_sg is a function that sets up the dma_address and dma_length
fields in a scatter list. LLDs gets a request from the transport layer
and need to call dma_map_sg.

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Feb 20, 2009, at 10:00 PM, FUJITA Tomonori wrote:

> On Thu, 19 Feb 2009 18:33:43 -0800
> Seokmann Ju <seokmann.ju@qlogic.com> wrote:
>
>>
>> On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:
>>
>>>
>>> On Feb 11, 2009, at 7:13 AM, James Smart wrote:
>>>
>>>> Trying to kick-start this again...
>>>> I've updated the prior RFC with the comments from Seokmann,
>>>> SvenFujita, and Boaz. I would still like review on the
>>>> blk_xxx completion calls in the std and error paths.
>>> I'm looking the driver changes to match it with latest updates in
>>> the transport layer.
>>> The driver with the changes should be available within a couple of
>>> weeks.
>> When the FC transport layer calls fc_bsg_map_buffer(), it eventually
>> calls
>> blk_rq_map_sg() to build sg_list for mapped bio in the request
>> structure.
>> I'm having trouble, however, in getting the sg_list for both  
>> dout_xferp
>> and din_xferp, which is mapped into the bio in the request structure.
>> With the pass-thru - Rev V, the sg_list returned from the
>> fc_bsg_map_buffer()
>> seems not right, particularly, the dma_address and dma_length fields
>> are NULL.
>> The sg_cnt and payload_len of the request_payload/reply_payload are
>> correctly
>> set by the function, though.
>
> I've not read the code yet but the above behavior doesn't sound a bug.
>
> dma_map_sg is a function that sets up the dma_address and dma_length
> fields in a scatter list. LLDs gets a request from the transport layer
> and need to call dma_map_sg.
Thanks for the point.
Actually, we talked about this before as below, somehow, I've forgotten.
http://markmail.org/message/vfxzsupax4xhozfy

RE: [RFC] FC pass thru - Rev V

[Seokmann Ju]

> From: James Smart [mailto:James.Smart@Emulex.Com]
> Sent: Wednesday, February 11, 2009 7:14 AM
> 
> Trying to kick-start this again...
> I've updated the prior RFC with the comments from Seokmann,
> SvenFujita, and Boaz. I would still like review on the
> blk_xxx completion calls in the std and error paths.
<snip>
> +static int
> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
> +{
> +	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
> +
> +	BUG_ON(!req->nr_phys_segments);
> +
> +	buf->sg_list = kzalloc(sz, GFP_KERNEL);
> +	if (!buf->sg_list)
> +		return -ENOMEM;
> +	sg_init_table(buf->sg_list, req->nr_phys_segments);
> +	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
> +	buf->payload_len = req->data_len;
> +	return 0;
> +}
It looks like that there is possibility that the dma buffer with more
than one sg element for ELS services - which is not allowed per FC spec.
I've seen a couple of times from the driver testing.
I'm looking for some ways to making sure the ELS services
to get single sg for both request/reply_payload.
Any comments/suggestion?

Thank you,
Seokmann

Re: [RFC] FC pass thru - Rev V

[James Smart]

Seokmann Ju wrote:
>> From: James Smart [mailto:James.Smart@Emulex.Com]
>> Sent: Wednesday, February 11, 2009 7:14 AM
>>
>> Trying to kick-start this again...
>> I've updated the prior RFC with the comments from Seokmann,
>> SvenFujita, and Boaz. I would still like review on the
>> blk_xxx completion calls in the std and error paths.
> <snip>
>> +static int
>> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
>> +{
>> +	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
>> +
>> +	BUG_ON(!req->nr_phys_segments);
>> +
>> +	buf->sg_list = kzalloc(sz, GFP_KERNEL);
>> +	if (!buf->sg_list)
>> +		return -ENOMEM;
>> +	sg_init_table(buf->sg_list, req->nr_phys_segments);
>> +	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
>> +	buf->payload_len = req->data_len;
>> +	return 0;
>> +}
> It looks like that there is possibility that the dma buffer with more
> than one sg element for ELS services - which is not allowed per FC spec.

How can the T11 FC spec ever reference anything about sg elements ? They 
are completely independent.

> I've seen a couple of times from the driver testing.
> I'm looking for some ways to making sure the ELS services
> to get single sg for both request/reply_payload.
> Any comments/suggestion?

For ELS's, the payload should be small - usually <= 256bytes so it fits 
within default login parameters. It should be 1 buffer, thus 1 sge. 
(Note: the routine above is for any request, which covers more than just 
ELS's).

If you're getting multiple sg's, then either the buffer spans some 
condition that crosses a boundary that blk_rq_map_sg() enforces (like a 
4G boundary, but I doubt that's it), or - and more likely - your 
application has a buffer that spans 2 pages which look contiguous to the 
app, but are actually virtual-memory mapped to 2 independent physical 
pages (thus the need for 2 sg's).  If your app knows the system page 
size, and was careful about buffer placement and length within page 
boundaries, it likely would never encounter this.  But, we would never 
want to force apps to be this smart, and this would be a good 
justification for supporting more than 1 sg.

-- james s

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Mar 7, 2009, at 6:44 AM, James Smart wrote:

>
>
> Seokmann Ju wrote:
>>> From: James Smart [mailto:James.Smart@Emulex.Com]
>>> Sent: Wednesday, February 11, 2009 7:14 AM
>>>
>>> Trying to kick-start this again...
>>> I've updated the prior RFC with the comments from Seokmann,
>>> SvenFujita, and Boaz. I would still like review on the
>>> blk_xxx completion calls in the std and error paths.
>> <snip>
>>> +static int
>>> +fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
>>> +{
>>> +	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
>>> +
>>> +	BUG_ON(!req->nr_phys_segments);
>>> +
>>> +	buf->sg_list = kzalloc(sz, GFP_KERNEL);
>>> +	if (!buf->sg_list)
>>> +		return -ENOMEM;
>>> +	sg_init_table(buf->sg_list, req->nr_phys_segments);
>>> +	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
>>> +	buf->payload_len = req->data_len;
>>> +	return 0;
>>> +}
>> It looks like that there is possibility that the dma buffer with more
>> than one sg element for ELS services - which is not allowed per FC  
>> spec.
>
> How can the T11 FC spec ever reference anything about sg elements ?  
> They
> are completely independent.
OK. Thanks for the clarification.

>
>
>> I've seen a couple of times from the driver testing.
>> I'm looking for some ways to making sure the ELS services
>> to get single sg for both request/reply_payload.
>> Any comments/suggestion?
>
> For ELS's, the payload should be small - usually <= 256bytes so it  
> fits
> within default login parameters. It should be 1 buffer, thus 1 sge.
> (Note: the routine above is for any request, which covers more than  
> just
> ELS's).
Yes, understood it.

>
>
> If you're getting multiple sg's, then either the buffer spans some
> condition that crosses a boundary that blk_rq_map_sg() enforces  
> (like a
> 4G boundary, but I doubt that's it), or - and more likely - your
> application has a buffer that spans 2 pages which look contiguous to  
> the
> app, but are actually virtual-memory mapped to 2 independent physical
> pages (thus the need for 2 sg's).  If your app knows the system page
> size, and was careful about buffer placement and length within page
> boundaries, it likely would never encounter this.  But, we would never
> want to force apps to be this smart, and this would be a good
> justification for supporting more than 1 sg.
OK. So the whole point that I brought up is implementation specific.
Thank you for the comment.

Seokmann

Re: [RFC] FC pass thru - Rev V

[James Smart]

>> If you're getting multiple sg's, then either the buffer spans some
>> condition that crosses a boundary that blk_rq_map_sg() enforces  
>> (like a
>> 4G boundary, but I doubt that's it), or - and more likely - your
>> application has a buffer that spans 2 pages which look contiguous to  
>> the
>> app, but are actually virtual-memory mapped to 2 independent physical
>> pages (thus the need for 2 sg's).  If your app knows the system page
>> size, and was careful about buffer placement and length within page
>> boundaries, it likely would never encounter this.  But, we would never
>> want to force apps to be this smart, and this would be a good
>> justification for supporting more than 1 sg.
> OK. So the whole point that I brought up is implementation specific.
> Thank you for the comment.

Yes, but I wouldn't be surprised to see it occur often enough to be a 
pain.  What I would recommend is :  if your hardware wants only one 
buffer descriptor for ELSs, and there's 2 BGs, that you double-buffer 
it. Allocate a local buffer, transfer the users data into the buffer, 
and then use the local buffer for the hardware. Same on the receive 
path. A pain, yes, but this isn't time critical stuff, and it's much 
better than just returning an error to the app based on it's buffer 
alignment (which it probably doesn't even realize).

The above works for ELS's (as its small), but I fully expect other 
things, like CT requests, to not be small and require multiple buffers. 
On those things, I hope the hardware supports multiple sg's as 
double-buffering it is ugly.

-- james s

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

James Smart wrote:
>>> If you're getting multiple sg's, then either the buffer spans some
>>> condition that crosses a boundary that blk_rq_map_sg() enforces  
>>> (like a
>>> 4G boundary, but I doubt that's it), or - and more likely - your
>>> application has a buffer that spans 2 pages which look contiguous to  
>>> the
>>> app, but are actually virtual-memory mapped to 2 independent physical
>>> pages (thus the need for 2 sg's).  If your app knows the system page
>>> size, and was careful about buffer placement and length within page
>>> boundaries, it likely would never encounter this.  But, we would never
>>> want to force apps to be this smart, and this would be a good
>>> justification for supporting more than 1 sg.
>> OK. So the whole point that I brought up is implementation specific.
>> Thank you for the comment.
> 
> Yes, but I wouldn't be surprised to see it occur often enough to be a 
> pain.  What I would recommend is :  if your hardware wants only one 
> buffer descriptor for ELSs, and there's 2 BGs, that you double-buffer 
> it. Allocate a local buffer, transfer the users data into the buffer, 
> and then use the local buffer for the hardware. Same on the receive 
> path. A pain, yes, but this isn't time critical stuff, and it's much 
> better than just returning an error to the app based on it's buffer 
> alignment (which it probably doesn't even realize).
> 

Or force a block bounce buffers by setting the queue alignment requirements
(Can only work for max transfer of PAGE_SIZE/2 )

> The above works for ELS's (as its small), but I fully expect other 
> things, like CT requests, to not be small and require multiple buffers. 
> On those things, I hope the hardware supports multiple sg's as 
> double-buffering it is ugly.
> 

> -- james s


Boaz

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:

>
> On Feb 11, 2009, at 7:13 AM, James Smart wrote:
>
>> Trying to kick-start this again...
>> I've updated the prior RFC with the comments from Seokmann,
>> SvenFujita, and Boaz. I would still like review on the
>> blk_xxx completion calls in the std and error paths.
> I'm looking the driver changes to match it with latest updates in  
> the transport layer.
> The driver with the changes should be available within a couple of  
> weeks.
Here is a change that I've made for transport layer while I'm testing  
the feature.
---
diff --git a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/ 
scsi_transport_fc.c
index 63ab7bc..4bca164 100644
--- a/drivers/scsi/scsi_transport_fc.c
+++ b/drivers/scsi/scsi_transport_fc.c
@@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
                 BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
                 /* set reply (bidi) residual */
                 rsp->data_len = (rsp_len - job->reply- 
 >reply_payload_rcv_len);
-       }
+               blk_end_bidi_request(req, err, blk_rq_bytes(req),
+                   job->reply->reply_payload_rcv_len);

-       /* we assume all request payload was transferred */
-       blk_end_request(req, err, blk_rq_bytes(req));
+       } else
+               blk_end_request(req, err, blk_rq_bytes(req));

         fc_destroy_bsgjob(job);
  }
---

With this change, I'm able to issue several ELS/CT services to the  
qla2xxx module for those registered to the transport layer and have  
rport associated with them.

And, below is the driver changes for just review purpose.

Do we have time table for this support to make it available in the  
upstream?
I think it would be good to have the feature in 2.6.30 kernel, if  
possible.
I will continue to work on and add rest of helper function in the  
driver.

Thank you,
Seokmann

---
diff -Naurp qla2xxx-org//qla_attr.c qla2xxx//qla_attr.c
--- qla2xxx-org//qla_attr.c	2009-03-13 06:15:53.000000000 -0700
+++ qla2xxx//qla_attr.c	2009-03-13 06:15:53.000000000 -0700
@@ -1054,6 +1054,457 @@ qla2x00_terminate_rport_io(struct fc_rpo
  }

  static int
+qla2x00_issue_els(struct fc_bsg_job *bsg_job)
+{
+	struct fc_rport *rport = bsg_job->rport;
+	fc_port_t *fcport = *(fc_port_t **) rport->dd_data;
+	struct Scsi_Host *host = rport_to_shost(rport);
+	scsi_qla_host_t *vha = shost_priv(host);
+	struct qla_hw_data *ha = vha->hw;
+	struct req_que *req;
+	struct rsp_que *rsp;
+	struct els_entry_24xx *els_iocb;
+	srb_t *sp;
+	unsigned long flags = 0;
+	uint32_t  handle;
+	int  index;
+	uint8_t  els_pkt[20];
+	size_t   copy_size;
+	uint16_t nextlid = 0;
+	uint16_t que_id;
+	int i;
+	int req_sg_cnt, rsp_sg_cnt;
+
+	/* ELS doesn't support multiple SG */
+	if (bsg_job->request_payload.sg_cnt > 1 ||
+	    bsg_job->reply_payload.sg_cnt > 1) {
+		qla_printk(KERN_WARNING, ha,
+		    "ERROR: multiple SG of request/response "
+		    "[%u/%u]  are not supported for ELS services\n",
+		    bsg_job->request_payload.sg_cnt,
+		    bsg_job->reply_payload.sg_cnt);
+		bsg_job->reply->result = -EIO;
+		goto pt_error0;
+	}
+
+	sp = mempool_alloc(ha->srb_mempool, GFP_ATOMIC);
+	if (!sp)
+		goto pt_error0;
+
+	sp->fcport = fcport;
+	sp->u.srv.bsg_job = bsg_job;
+	sp->flags = SRB_ELS_CT;
+
+	if (bsg_job->request->msgcode == FC_BSG_RPT_ELS) {
+		if (qla2x00_fabric_login(vha, fcport, &nextlid)) {
+			DEBUG2(qla_printk(KERN_WARNING, ha,
+			    "failed to login port %06X for ELS passthru\n",
+			    fcport->d_id.b24));
+			goto pt_error1;
+		}
+	}
+
+	que_id = vha->req_ques[0];
+	req = ha->req_q_map[que_id];
+	if (req->rsp)
+		rsp = req->rsp;
+	else
+		rsp = ha->rsp_q_map[que_id];
+
+	spin_lock_irqsave(&ha->hardware_lock, flags);
+	els_iocb = (struct els_entry_24xx *) qla2x00_req_pkt(vha, req, rsp);
+	if (els_iocb == NULL) {
+		qla_printk(KERN_WARNING, ha,
+		    "Passthru request failed to get request packet\n");
+		goto pt_error1;
+	}
+
+	req_sg_cnt = dma_map_sg(&ha->pdev->dev, bsg_job- 
 >request_payload.sg_list,
+	    bsg_job->request_payload.sg_cnt, DMA_TO_DEVICE);
+	if (!req_sg_cnt)
+		goto pt_error1;
+	rsp_sg_cnt = dma_map_sg(&ha->pdev->dev, bsg_job- 
 >reply_payload.sg_list,
+	    bsg_job->reply_payload.sg_cnt, DMA_FROM_DEVICE);
+	if (!rsp_sg_cnt)
+		goto pt_error2;
+
+	copy_size = sg_copy_to_buffer(bsg_job->request_payload.sg_list,
+	    bsg_job->request_payload.sg_cnt, els_pkt, 20);
+	if (!copy_size)
+		goto pt_error3;
+
+	printk("contents of ELS packet: ");
+	for (i=0;i<copy_size;i++)
+		printk("%2x ", els_pkt[i]);
+	printk("\n");
+
+	/* Check for room in outstanding command list. */
+	handle = req->current_outstanding_cmd;
+	for (index = 1; index < MAX_OUTSTANDING_COMMANDS; index++) {
+		handle++;
+		if (handle == MAX_OUTSTANDING_COMMANDS)
+			handle = 1;
+		if (!req->outstanding_cmds[handle])
+			break;
+	}
+
+	req->current_outstanding_cmd = handle;
+	req->outstanding_cmds[handle] = (srb_t *) sp;
+
+	els_iocb->handle = handle;
+
+	els_iocb->entry_type = ELS_IOCB_TYPE;
+	els_iocb->entry_count = 1;
+	els_iocb->sys_define = 0;
+	els_iocb->entry_status = 0;
+	els_iocb->nport_handle = cpu_to_le16(fcport->loop_id);
+	els_iocb->tx_dsd_count = __constant_cpu_to_le16(1);
+	els_iocb->vp_index = vha->vp_idx;
+	els_iocb->sof_type = EST_SOFI3;
+	els_iocb->rx_dsd_count = __constant_cpu_to_le16(1);
+	els_iocb->opcode = bsg_job->request->rqst_data.r_els.els_code;
+	els_iocb->port_id[0] = fcport->d_id.b.al_pa;
+	els_iocb->port_id[1] = fcport->d_id.b.area;
+	els_iocb->port_id[2] = fcport->d_id.b.domain;
+	els_iocb->control_flags = 0;
+	els_iocb->rx_byte_count =
+	    cpu_to_le32(bsg_job->reply_payload.payload_len);
+	els_iocb->tx_byte_count =
+	    cpu_to_le32(bsg_job->request_payload.payload_len);
+	els_iocb->tx_address[0] = cpu_to_le32(LSD(sg_dma_address
+	    (bsg_job->request_payload.sg_list)));
+	els_iocb->tx_address[1] = cpu_to_le32(MSD(sg_dma_address
+	    (bsg_job->request_payload.sg_list)));
+
+	els_iocb->tx_len = cpu_to_le32(sg_dma_len
+	    (bsg_job->request_payload.sg_list));
+
+	els_iocb->rx_address[0] = cpu_to_le32(LSD(sg_dma_address
+	    (bsg_job->reply_payload.sg_list)));
+	els_iocb->rx_address[1] = cpu_to_le32(MSD(sg_dma_address
+	    (bsg_job->reply_payload.sg_list)));
+
+	els_iocb->rx_len = cpu_to_le32(sg_dma_len
+	    (bsg_job->reply_payload.sg_list));
+
+	wmb();
+	qla2x00_isp_cmd(vha, req);
+
+	spin_unlock_irqrestore(&ha->hardware_lock, flags);
+	return 0;
+pt_error3:
+	dma_unmap_sg(&ha->pdev->dev, bsg_job->reply_payload.sg_list,
+	    bsg_job->reply_payload.sg_cnt, DMA_FROM_DEVICE);
+pt_error2:
+	dma_unmap_sg(&ha->pdev->dev, bsg_job->request_payload.sg_list,
+	    bsg_job->request_payload.sg_cnt, DMA_TO_DEVICE);
+pt_error1:
+	spin_unlock_irqrestore(&ha->hardware_lock, flags);
+	mempool_free(sp, ha->srb_mempool);
+pt_error0:
+	qla_printk(KERN_WARNING, ha, "ELS passthru failed\n");
+	bsg_job->reply->result = -EIO;
+ 	bsg_job->reply->reply_payload_rcv_len = 0;
+ 	bsg_job->job_done(bsg_job);
+
+	return -EINVAL;
+}
+
+static int
+qla2x00_issue_ct(struct fc_bsg_job *bsg_job)
+{
+	struct fc_rport *rport = bsg_job->rport;
+	fc_port_t *fcport = *(fc_port_t **) rport->dd_data;
+	struct Scsi_Host *host = rport_to_shost(rport);
+	scsi_qla_host_t *vha = shost_priv(host);
+	struct qla_hw_data *ha = vha->hw;
+	struct req_que *req;
+	struct rsp_que *rsp;
+	struct ct_entry_24xx *ct_iocb;
+	srb_t *sp;
+	unsigned long flags;
+	uint32_t  handle;
+	int  index;
+	uint8_t  ct_pkt[20];
+	size_t   copy_size;
+	int  req_sg_cnt, rsp_sg_cnt;
+	uint16_t avail_dsds;
+	uint32_t *cur_dsd;
+	struct scatterlist *sg;
+	uint16_t que_id;
+	int i;
+
+	sp = mempool_alloc(ha->srb_mempool, GFP_ATOMIC);
+	if (!sp)
+		goto pt_error0;
+
+	sp->fcport = fcport;
+	sp->u.srv.bsg_job = bsg_job;
+	sp->flags = SRB_ELS_CT;
+
+	que_id = vha->req_ques[0];
+	req = ha->req_q_map[que_id];
+	if (req->rsp)
+		rsp = req->rsp;
+	else
+		rsp = ha->rsp_q_map[que_id];
+
+	spin_lock_irqsave(&ha->hardware_lock, flags);
+	ct_iocb = (struct ct_entry_24xx *) qla2x00_req_pkt(vha, req, rsp);
+	if (ct_iocb == NULL) {
+		qla_printk(KERN_WARNING, ha,
+		    "Passthru request failed to get request packet\n");
+		goto pt_error1;
+	}
+
+	req_sg_cnt = dma_map_sg(&ha->pdev->dev, bsg_job- 
 >request_payload.sg_list,
+	    bsg_job->request_payload.sg_cnt, DMA_TO_DEVICE);
+	if (!req_sg_cnt)
+		goto pt_error1;
+	rsp_sg_cnt = dma_map_sg(&ha->pdev->dev, bsg_job- 
 >reply_payload.sg_list,
+	    bsg_job->reply_payload.sg_cnt, DMA_FROM_DEVICE);
+	if (!rsp_sg_cnt)
+		goto pt_error2;
+
+	copy_size = sg_copy_to_buffer(bsg_job->request_payload.sg_list,
+	    bsg_job->request_payload.sg_cnt, ct_pkt, 20);
+	if (!copy_size)
+		goto pt_error2;
+
+	printk("contents of CT packet: ");
+	for (i=0;i<copy_size;i++)
+		printk("%2x ", ct_pkt[i]);
+	printk("\n");
+
+	/* Check for room in outstanding command list. */
+	handle = req->current_outstanding_cmd;
+	for (index = 1; index < MAX_OUTSTANDING_COMMANDS; index++) {
+		handle++;
+		if (handle == MAX_OUTSTANDING_COMMANDS)
+			handle = 1;
+		if (!req->outstanding_cmds[handle])
+			break;
+	}
+
+	req->current_outstanding_cmd = handle;
+	req->outstanding_cmds[handle] = (srb_t *) sp;
+
+	ct_iocb->handle = handle;
+
+	ct_iocb->entry_type = CT_IOCB_TYPE;
+	ct_iocb->entry_count = 1;
+	ct_iocb->entry_status = 0;
+	ct_iocb->comp_status = __constant_cpu_to_le16(0);
+	if (IS_FWI2_CAPABLE(ha))
+		ct_iocb->nport_handle = cpu_to_le16(NPH_SNS);
+	else
+		ct_iocb->nport_handle =
+		    cpu_to_le16(SIMPLE_NAME_SERVER);
+	ct_iocb->cmd_dsd_count =
+	    __constant_cpu_to_le16(req_sg_cnt);
+	ct_iocb->vp_index = vha->vp_idx;
+	ct_iocb->timeout = 0;
+	ct_iocb->rsp_dsd_count =
+	    __constant_cpu_to_le16(rsp_sg_cnt);
+	ct_iocb->rsp_byte_count =
+	    cpu_to_le32(bsg_job->reply_payload.payload_len);
+	ct_iocb->cmd_byte_count =
+	    cpu_to_le32(bsg_job->request_payload.payload_len);
+	ct_iocb->dseg_0_address[0] = cpu_to_le32(LSD(sg_dma_address
+	    (bsg_job->request_payload.sg_list)));
+	ct_iocb->dseg_0_address[1] = cpu_to_le32(MSD(sg_dma_address
+	    (bsg_job->request_payload.sg_list)));
+
+	ct_iocb->dseg_0_len = cpu_to_le32(sg_dma_len
+	    (bsg_job->request_payload.sg_list));
+
+	rsp_sg_cnt = bsg_job->reply_payload.sg_cnt;
+	avail_dsds = 1;
+	cur_dsd = (uint32_t *)ct_iocb->dseg_1_address;
+	index = 0;
+	for_each_sg(bsg_job->reply_payload.sg_list, sg, rsp_sg_cnt, index) {
+		dma_addr_t       sle_dma;
+		cont_a64_entry_t *cont_pkt;
+
+		/* Allocate additional continuation packets? */
+		if (avail_dsds == 0) {
+			/*
+			 * Five DSDs are available in the Cont.
+			 * Type 1 IOCB.
+			 */
+			cont_pkt = qla2x00_prep_cont_type1_iocb(req, vha);
+			cur_dsd = (uint32_t *) cont_pkt->dseg_0_address;
+			avail_dsds = 5;
+		}
+
+		sle_dma = sg_dma_address(sg);
+		*cur_dsd++ = cpu_to_le32(LSD(sle_dma));
+		*cur_dsd++ = cpu_to_le32(MSD(sle_dma));
+		*cur_dsd++ = cpu_to_le32(sg_dma_len(sg));
+		avail_dsds--;
+	}
+
+	wmb();
+	qla2x00_isp_cmd(vha, req);
+
+	spin_unlock_irqrestore(&ha->hardware_lock, flags);
+	return 0;
+
+pt_error2:
+	dma_unmap_sg(&ha->pdev->dev, bsg_job->request_payload.sg_list,
+	    bsg_job->request_payload.sg_cnt, DMA_TO_DEVICE);
+pt_error1:
+	spin_unlock_irqrestore(&ha->hardware_lock, flags);
+	mempool_free(sp, ha->srb_mempool);
+pt_error0:
+	qla_printk(KERN_WARNING, ha, "CT passthru failed\n");
+	bsg_job->reply->result = -EIO;
+ 	bsg_job->job_done(bsg_job);
+
+	return -EINVAL;
+}
+
+static int
+qla2x00_issue_add_rport(struct fc_bsg_job *bsg_job)
+{
+	printk("%s is under construction\n", __func__);
+	return -EINVAL;
+}
+
+static int
+qla2x00_issue_delete_rport(struct fc_bsg_job *bsg_job)
+{
+	printk("%s is under construction\n", __func__);
+	return -EINVAL;
+}
+
+static int
+qla2x00_issue_vendor_specific(struct fc_bsg_job *bsg_job)
+{
+	printk("%s is under construction\n", __func__);
+	return -EINVAL;
+}
+
+static int
+qla24xx_bsg_request(struct fc_bsg_job *bsg_job)
+{
+	int ret = 0;
+	fc_port_t *fcport;
+	struct Scsi_Host *host;
+	scsi_qla_host_t *vha;
+	struct qla_hw_data *ha;
+
+	if (bsg_job->rport) {
+		if (bsg_job->rport->port_state != FC_PORTSTATE_ONLINE)
+			goto pt_error;
+
+		fcport = *(fc_port_t **) bsg_job->rport->dd_data;
+		host = rport_to_shost(bsg_job->rport);
+	} else {
+		printk("PASS-THRU: %s: HOST under construction\n", __func__);
+		ret = -EINVAL;
+		goto pt_error;
+	}
+
+	/* if fcport is NULL */
+	if (!fcport)
+		goto pt_error;
+
+	vha = shost_priv(host);
+	ha = vha->hw;
+
+	/* At this time, pass throug supported by 4Gb or higher */
+	if (!IS_FWI2_CAPABLE(ha))
+		goto pt_error;
+
+	switch (bsg_job->request->msgcode) {
+	case FC_BSG_RPT_ELS:
+		ret = qla2x00_issue_els(bsg_job);
+		break;
+	case FC_BSG_RPT_CT:
+		ret = qla2x00_issue_ct(bsg_job);
+		break;
+	case FC_BSG_HST_ADD_RPORT:
+		ret = qla2x00_issue_add_rport(bsg_job);
+		break;
+	case FC_BSG_HST_DEL_RPORT:
+		ret = qla2x00_issue_delete_rport(bsg_job);
+		break;
+	case FC_BSG_HST_ELS_NOLOGIN:
+		ret = qla2x00_issue_els(bsg_job);
+		break;
+	case FC_BSG_HST_VENDOR:
+		ret = qla2x00_issue_vendor_specific(bsg_job);
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	if (ret)
+		goto pt_error;
+
+	return ret;
+
+pt_error:
+	printk(KERN_WARNING "ERROR: failed to issue job\n");
+	return ret;
+}
+
+static int
+qla24xx_bsg_timeout(struct fc_bsg_job *bsg_job)
+{
+	scsi_qla_host_t *vha = shost_priv(bsg_job->shost);
+	srb_t *sp;
+	int ret, i;
+	unsigned long flags;
+	int wait = 0;
+	struct qla_hw_data *ha = vha->hw;
+	uint16_t que_id;
+	struct req_que *req;
+	struct rsp_que *rsp;
+
+	ret = SUCCESS;
+
+	que_id = vha->req_ques[0];
+	req = ha->req_q_map[que_id];
+	if (req->rsp)
+		rsp = req->rsp;
+	else
+		rsp = ha->rsp_q_map[que_id];
+
+	/* Check active list for command command. */
+	spin_lock_irqsave(&ha->hardware_lock, flags);
+	for (i = 1; i < MAX_OUTSTANDING_COMMANDS; i++) {
+		sp = req->outstanding_cmds[i];
+
+		if (sp == NULL)
+			continue;
+
+		if ((sp->flags != SRB_ELS_CT) ||
+		    (sp->u.srv.bsg_job != bsg_job))
+			continue;
+
+		spin_unlock_irqrestore(&ha->hardware_lock, flags);
+		if (ha->isp_ops->abort_command(vha, sp, req)) {
+			DEBUG2(printk("%s(%ld): abort_command "
+			    "mbx failed.\n", __func__, vha->host_no));
+			ret = FAILED;
+		} else {
+			DEBUG3(printk("%s(%ld): abort_command "
+			    "mbx success.\n", __func__, vha->host_no));
+			wait = 1;
+		}
+		spin_lock_irqsave(&ha->hardware_lock, flags);
+
+		break;
+	}
+	spin_unlock_irqrestore(&ha->hardware_lock, flags);
+
+	return ret;
+}
+
+static int
  qla2x00_issue_lip(struct Scsi_Host *shost)
  {
  	scsi_qla_host_t *vha = shost_priv(shost);
@@ -1353,6 +1804,9 @@ struct fc_function_template qla2xxx_tran
  	.vport_create = qla24xx_vport_create,
  	.vport_disable = qla24xx_vport_disable,
  	.vport_delete = qla24xx_vport_delete,
+
+	.bsg_request = qla24xx_bsg_request,
+	.bsg_timeout = qla24xx_bsg_timeout,
  };

  struct fc_function_template qla2xxx_transport_vport_functions = {
@@ -1393,6 +1847,9 @@ struct fc_function_template qla2xxx_tran
  	.dev_loss_tmo_callbk = qla2x00_dev_loss_tmo_callbk,
  	.terminate_rport_io = qla2x00_terminate_rport_io,
  	.get_fc_host_stats = qla2x00_get_fc_host_stats,
+
+	.bsg_request = qla24xx_bsg_request,
+	.bsg_timeout = qla24xx_bsg_timeout,
  };

  void
diff -Naurp qla2xxx-org//qla_def.h qla2xxx//qla_def.h
--- qla2xxx-org//qla_def.h	2009-03-13 06:15:53.000000000 -0700
+++ qla2xxx//qla_def.h	2009-03-13 06:15:53.000000000 -0700
@@ -31,6 +31,7 @@
  #include <scsi/scsi_device.h>
  #include <scsi/scsi_cmnd.h>
  #include <scsi/scsi_transport_fc.h>
+#include <scsi/scsi_bsg_fc.h>

  #define QLA2XXX_DRIVER_NAME  "qla2xxx"

diff -Naurp qla2xxx-org//qla_isr.c qla2xxx//qla_isr.c
--- qla2xxx-org//qla_isr.c	2009-03-13 06:15:53.000000000 -0700
+++ qla2xxx//qla_isr.c	2009-03-13 06:15:53.000000000 -0700
@@ -1536,10 +1536,74 @@ qla24xx_process_response_queue(struct rs
  		case STATUS_CONT_TYPE:
  			qla2x00_status_cont_entry(vha, (sts_cont_entry_t *)pkt);
  			break;
+ 		case MS_IOCB_TYPE:
+ 		case ELS_IOCB_TYPE:
+ 			if (pkt->handle < MAX_OUTSTANDING_COMMANDS) {
+ 				sp = req->outstanding_cmds[pkt->handle];
+				if (sp->flags != SRB_ELS_CT)
+					break;
+ 				bsg_job = sp->u.srv.bsg_job;
+ 				req->outstanding_cmds[pkt->handle] = NULL;
+ 			} else {
+ 				bsg_job = NULL;
+ 				break;
+ 			}
+
+ 			bsg_job->reply->result = 0;
+ 			bsg_job->reply->reply_payload_rcv_len =
+ 			    bsg_job->reply_payload.payload_len;
+
+ 			if (comp_status) {
+ 				bsg_job->reply->result = -EIO;
+ 				printk(KERN_WARNING "scsi(%ld): ELS/CT: "
+ 				    "comp_status = %x\n", vha->host_no,
+ 				    comp_status);
+ 				bsg_job->reply->reply_data.vendor_reply.vendor_rsp[0] =  
comp_status;
+ 				if ((comp_status >> 8) == CS_DATA_UNDERRUN)
+ 					bsg_job->reply->reply_payload_rcv_len =
+ 					    bsg_job->reply_payload.payload_len;
+ 				else if ((comp_status >> 8) == CS_ABORTED) {
+ 					bsg_job->reply->result = -EIO;
+ 				}
+ 			}
+
+			dma_unmap_sg(&ha->pdev->dev,
+			    bsg_job->request_payload.sg_list,
+			    bsg_job->request_payload.sg_cnt, DMA_TO_DEVICE);
+			dma_unmap_sg(&ha->pdev->dev,
+			    bsg_job->reply_payload.sg_list,
+			    bsg_job->reply_payload.sg_cnt, DMA_FROM_DEVICE);
+
+ 			mempool_free(sp, ha->srb_mempool);
+ 			bsg_job->job_done(bsg_job);
+ 			break;
  		case VP_RPT_ID_IOCB_TYPE:
  			qla24xx_report_id_acquisition(vha,
  			    (struct vp_rpt_id_entry_24xx *)pkt);
  			break;
+ 		case ABORT_IOCB_TYPE:
+ 			if (comp_status) {
+ 				if (pkt->handle < MAX_OUTSTANDING_COMMANDS) {
+ 					sp = req->outstanding_cmds[pkt->handle];
+ 					req->outstanding_cmds[pkt->handle] =
+ 					    NULL;
+ 					if (sp->flags & SRB_ELS_CT) {
+ 						bsg_job =
+						    (struct fc_bsg_job *)
+						    sp->u.srv.bsg_job;
+ 						qla_printk(KERN_ERR, ha,
+ 						    "failed to abort"
+ 						    " FC service %p\n",
+ 						    bsg_job);
+						/* TBD - what else to do? */
+ 					} else
+ 						qla_printk(KERN_ERR, ha,
+ 						    "failed to abort"
+ 						    " SCSI CMD[0] = %2x\n",
+ 						    sp->u.io.cmd->cmnd[0]);
+ 				}
+ 			}
+ 			break;
  		default:
  			/* Type Not Supported. */
  			DEBUG4(printk(KERN_WARNING
---

Re: [RFC] FC pass thru - Rev V

[Sven Schuetz]

Hi Seokmann,
>
> Do we have time table for this support to make it available in the 
> upstream?
> I think it would be good to have the feature in 2.6.30 kernel, if 
> possible.
> I will continue to work on and add rest of helper function in the driver.
>
> Thank you,
> Seokmann
>
> ---

I am preparing a patch for our zfcp driver and would love to see both 
James&your fc_transport patch as well as the patches for our drivers in 
2.6.30 too :)

Tomo, Boaz - since the current implemantation is based on that patch:
http://marc.info/?l-linux-scsi&m=122785157116659&w=2
which is not agreed upon - can we start the discussion again?
My understanding from the discussion back then was that Boaz proposed a 
change to the patch which Tomo agreed to. See here: 
http://marc.info/?l=linux-scsi&m=122812818916080&w=2
However, a new version of the patch was never posted. So would a patch 
with the proposed change be ok for the two of you which could be posted 
to Jens?
I really appreciate all of your efforts, thanks a lot for that :-)
Have a nice weekend,
Sven

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

Hi Sven,

On Mar 13, 2009, at 9:47 AM, Sven Schuetz wrote:

> Hi Seokmann,
>>
>> Do we have time table for this support to make it available in the
>> upstream?
>> I think it would be good to have the feature in 2.6.30 kernel, if
>> possible.
>> I will continue to work on and add rest of helper function in the  
>> driver.
>>
>> Thank you,
>> Seokmann
>>
>> ---
>
> I am preparing a patch for our zfcp driver and would love to see both
> James&your fc_transport patch as well as the patches for our drivers  
> in
> 2.6.30 too :)
Yeah, that is also my hope, too.
As I get move forward further, I will post updates for rest of work.

Thank you and have a great weekend.

Seokmann
> Tomo, Boaz - since the current implemantation is based on that patch:
> http://marc.info/?l-linux-scsi&m=122785157116659&w=2
> which is not agreed upon - can we start the discussion again?
> My understanding from the discussion back then was that Boaz  
> proposed a
> change to the patch which Tomo agreed to. See here:
> http://marc.info/?l=linux-scsi&m=122812818916080&w=2
> However, a new version of the patch was never posted. So would a patch
> with the proposed change be ok for the two of you which could be  
> posted
> to Jens?
> I really appreciate all of your efforts, thanks a lot for that :-)
> Have a nice weekend,
> Sven

Re: [RFC] FC pass thru - Rev V

[James Smart]

Thank you Seokmann. I'm glad the RFC worked well for you.  Your patch is 
appreciated, it was the portion that needed to be reverted as the 
collapsed blk_end_request() patch wasn't taken.  To double check - there 
are no patches required with the RFC now - right ?

I'll repost as a non-RFC, and we'll see where James takes it.

-- james s

Seokmann Ju wrote:
> On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:
>   
> Here is a change that I've made for transport layer while I'm testing
> the feature.
> ---
> diff --git a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/
> scsi_transport_fc.c
> index 63ab7bc..4bca164 100644
>   
<snip...>
> With this change, I'm able to issue several ELS/CT services to the
> qla2xxx module for those registered to the transport layer and have
> rport associated with them.
>
> And, below is the driver changes for just review purpose.
>
> Do we have time table for this support to make it available in the
> upstream?
> I think it would be good to have the feature in 2.6.30 kernel, if
> possible.
> I will continue to work on and add rest of helper function in the
> driver.
>
> Thank you,
> Seokmann
>   

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

Seokmann Ju wrote:
> On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:
> 
>> On Feb 11, 2009, at 7:13 AM, James Smart wrote:
>>
>>> Trying to kick-start this again...
>>> I've updated the prior RFC with the comments from Seokmann,
>>> SvenFujita, and Boaz. I would still like review on the
>>> blk_xxx completion calls in the std and error paths.
>> I'm looking the driver changes to match it with latest updates in  
>> the transport layer.
>> The driver with the changes should be available within a couple of  
>> weeks.
> Here is a change that I've made for transport layer while I'm testing  
> the feature.
> ---
> diff --git a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/ 
> scsi_transport_fc.c
> index 63ab7bc..4bca164 100644
> --- a/drivers/scsi/scsi_transport_fc.c
> +++ b/drivers/scsi/scsi_transport_fc.c
> @@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
>                  BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
>                  /* set reply (bidi) residual */
>                  rsp->data_len = (rsp_len - job->reply- 
>  >reply_payload_rcv_len);

This part looks a bit white-space broken.

> -       }
> +               blk_end_bidi_request(req, err, blk_rq_bytes(req),
> +                   job->reply->reply_payload_rcv_len);
> 
> -       /* we assume all request payload was transferred */
> -       blk_end_request(req, err, blk_rq_bytes(req));
> +       } else
> +               blk_end_request(req, err, blk_rq_bytes(req));

I still do not like this version for more then one reason. There is
a more proper simpler way.

Where is the code-base this is patching? I'll make a proper patch for you.

> 
>          fc_destroy_bsgjob(job);
>   }
> ---
> 
> With this change, I'm able to issue several ELS/CT services to the  
> qla2xxx module for those registered to the transport layer and have  
> rport associated with them.
> 
<snip>

Boaz

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

Sven Schuetz wrote:
> Hi Seokmann,
>> Do we have time table for this support to make it available in the 
>> upstream?
>> I think it would be good to have the feature in 2.6.30 kernel, if 
>> possible.
>> I will continue to work on and add rest of helper function in the driver.
>>
>> Thank you,
>> Seokmann
>>
>> ---
> 
> I am preparing a patch for our zfcp driver and would love to see both 
> James&your fc_transport patch as well as the patches for our drivers in 
> 2.6.30 too :)
> 
> Tomo, Boaz - since the current implemantation is based on that patch:
> http://marc.info/?l-linux-scsi&m=122785157116659&w=2
> which is not agreed upon - can we start the discussion again?
> My understanding from the discussion back then was that Boaz proposed a 
> change to the patch which Tomo agreed to. See here: 
> http://marc.info/?l=linux-scsi&m=122812818916080&w=2
> However, a new version of the patch was never posted. So would a patch 
> with the proposed change be ok for the two of you which could be posted 
> to Jens?
> I really appreciate all of your efforts, thanks a lot for that :-)
> Have a nice weekend,
> Sven

My humble opinion is: That none of the patches should go in. It is easy
enough to use the API as it is, and it is more correct then any of the
suggested alternatives.

Please point me to the latest code-base and I'll fix you a patch that I
think looks very simple and will work well.

Cheers, have a good day
Boaz

Re: [RFC] FC pass thru - Rev V

[James Smart]

Current patch is the RFC, at :
http://marc.info/?l=linux-scsi&m=123436574018579&w=2

-- james s

Boaz Harrosh wrote:
> My humble opinion is: That none of the patches should go in. It is easy
> enough to use the API as it is, and it is more correct then any of the
> suggested alternatives.
>
> Please point me to the latest code-base and I'll fix you a patch that I
> think looks very simple and will work well.
>
> Cheers, have a good day
> Boaz
>   

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

James Smart wrote:
> Current patch is the RFC, at :
> http://marc.info/?l=linux-scsi&m=123436574018579&w=2
> 
> -- james s
> 
> Boaz Harrosh wrote:
>> My humble opinion is: That none of the patches should go in. It is easy
>> enough to use the API as it is, and it is more correct then any of the
>> suggested alternatives.
>>
>> Please point me to the latest code-base and I'll fix you a patch that I
>> think looks very simple and will work well.
>>
>> Cheers, have a good day
>> Boaz
>>   

My changes below

> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +	struct request *req = job->req;
> +	struct request *rsp = req->next_rq;
> +	unsigned long flags;
> +	unsigned rsp_len;

-	unsigned rsp_len;
+	unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

> +	int err;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	job->state_flags |= FC_RQST_STATE_DONE;
> +	job->ref_cnt--;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	err = job->req->errors = job->reply->result;
> +	if (err < 0)
> +		/* we're only returning the result field in the reply */
> +		job->req->sense_len = sizeof(uint32_t);
> +	else
> +		job->req->sense_len = job->reply_len;
> +
> +	if (rsp) {
> +		rsp_len = blk_rq_bytes(rsp);
> +		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
> +		/* set reply (bidi) residual */
> +		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
> +	}
> +
> +	/* we assume all request payload was transferred */

-	/* we assume all request payload was transferred */
+	/* we assume all request payload was transferred, residual == 0 */
+	req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer
residual of all payload. On the other hand you might have more precise residual
information here, as received from the scsi transport? (Specially in the error
case, perhaps in the future).

> +	blk_end_request(req, err, blk_rq_bytes(req));

-	blk_end_request(req, err, blk_rq_bytes(req));
+	blk_end_bidi_request(req, err, req_len, rsp_len);

> +
> +	fc_destroy_bsgjob(job);
> +}

This should take care of all possible cases, (bidi or otherwise), and will
return the proper transfer residual count of 0 to user-mode.

Thanks
Boaz

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

James Smart wrote:
> Current patch is the RFC, at :
> http://marc.info/?l=linux-scsi&m=123436574018579&w=2
> 
> -- james s
> 
> Boaz Harrosh wrote:
>> My humble opinion is: That none of the patches should go in. It is easy
>> enough to use the API as it is, and it is more correct then any of the
>> suggested alternatives.
>>
>> Please point me to the latest code-base and I'll fix you a patch that I
>> think looks very simple and will work well.
>>
>> Cheers, have a good day
>> Boaz
>>   

My changes below

> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +	struct request *req = job->req;
> +	struct request *rsp = req->next_rq;
> +	unsigned long flags;
> +	unsigned rsp_len;

-	unsigned rsp_len;
+	unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

> +	int err;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	job->state_flags |= FC_RQST_STATE_DONE;
> +	job->ref_cnt--;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	err = job->req->errors = job->reply->result;
> +	if (err < 0)
> +		/* we're only returning the result field in the reply */
> +		job->req->sense_len = sizeof(uint32_t);
> +	else
> +		job->req->sense_len = job->reply_len;
> +
> +	if (rsp) {
> +		rsp_len = blk_rq_bytes(rsp);
> +		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
> +		/* set reply (bidi) residual */
> +		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
> +	}
> +
> +	/* we assume all request payload was transferred */

-	/* we assume all request payload was transferred */
+	/* we assume all request payload was transferred, residual == 0 */
+	req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer
residual of all payload. On the other hand you might have more precise residual
information here, as received from the scsi transport? (Specially in the error
case, perhaps in the future).

> +	blk_end_request(req, err, blk_rq_bytes(req));

-	blk_end_request(req, err, blk_rq_bytes(req));
+	blk_end_bidi_request(req, err, req_len, rsp_len);

> +
> +	fc_destroy_bsgjob(job);
> +}

This should take care of all possible cases, (bidi or otherwise), and will
return the proper transfer residual count of 0 to user-mode.

Thanks
Boaz

(Sorry if it's a repost, we had a small email outage)

Re: [RFC] FC pass thru - Rev V

[James Smart]

Boaz Harrosh wrote:
<snip..>
>> +	if (rsp) {
>> +		rsp_len = blk_rq_bytes(rsp);
>> +		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
>> +		/* set reply (bidi) residual */
>> +		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
>> +	}
>> +
>> +	/* we assume all request payload was transferred */
> 
> -	/* we assume all request payload was transferred */
> +	/* we assume all request payload was transferred, residual == 0 */
> +	req->data_len = 0;
> 
> Note: this is the proper procedure. Otherwise bsg user will receive transfer
> residual of all payload. On the other hand you might have more precise residual
> information here, as received from the scsi transport? (Specially in the error
> case, perhaps in the future).
> 
>> +	blk_end_request(req, err, blk_rq_bytes(req));
> 
> -	blk_end_request(req, err, blk_rq_bytes(req));
> +	blk_end_bidi_request(req, err, req_len, rsp_len);
> 
>> +
>> +	fc_destroy_bsgjob(job);
>> +}
> 
> This should take care of all possible cases, (bidi or otherwise), and will
> return the proper transfer residual count of 0 to user-mode.

We should be calling blk_end_bidi_request() even in cases where there wasn't a 
req->next_rq ?

Here's what I would have done. Has your changes for setting req_len and 
req->data_len, and Seokman's mods for calling either blk_end_bidi_request() or 
blk_end_request() depending on whether there is a rsp request.

-- james s


/**
  * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
  *                  completed
  * @job:	fc_bsg_job that is complete
  */
static void
fc_bsg_jobdone(struct fc_bsg_job *job)
{
	struct request *req = job->req;
	struct request *rsp = req->next_rq;
	unsigned long flags;
	unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

	int err;

	spin_lock_irqsave(&job->job_lock, flags);
	job->state_flags |= FC_RQST_STATE_DONE;
	job->ref_cnt--;
	spin_unlock_irqrestore(&job->job_lock, flags);

	err = job->req->errors = job->reply->result;
	if (err < 0)
		/* we're only returning the result field in the reply */
		job->req->sense_len = sizeof(uint32_t);
	else
		job->req->sense_len = job->reply_len;

	/* we assume all request payload was transferred, residual == 0 */
	req->data_len = 0;

	if (rsp) {
		rsp_len = blk_rq_bytes(rsp);
		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
		/* set reply (bidi) residual */
		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
		blk_end_bidi_request(req, err, req_len, rsp_len);
	} else
		blk_end_request(req, err, req_len);

	fc_destroy_bsgjob(job);
}

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

James Smart wrote:
> 
> Boaz Harrosh wrote:
> <snip..>
>>> +	if (rsp) {
>>> +		rsp_len = blk_rq_bytes(rsp);
>>> +		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
>>> +		/* set reply (bidi) residual */
>>> +		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
>>> +	}
>>> +
>>> +	/* we assume all request payload was transferred */
>> -	/* we assume all request payload was transferred */
>> +	/* we assume all request payload was transferred, residual == 0 */
>> +	req->data_len = 0;
>>
>> Note: this is the proper procedure. Otherwise bsg user will receive transfer
>> residual of all payload. On the other hand you might have more precise residual
>> information here, as received from the scsi transport? (Specially in the error
>> case, perhaps in the future).
>>
>>> +	blk_end_request(req, err, blk_rq_bytes(req));
>> -	blk_end_request(req, err, blk_rq_bytes(req));
>> +	blk_end_bidi_request(req, err, req_len, rsp_len);
>>
>>> +
>>> +	fc_destroy_bsgjob(job);
>>> +}
>> This should take care of all possible cases, (bidi or otherwise), and will
>> return the proper transfer residual count of 0 to user-mode.
> 
> We should be calling blk_end_bidi_request() even in cases where there wasn't a 
> req->next_rq ?
> 
> Here's what I would have done. Has your changes for setting req_len and 
> req->data_len, and Seokman's mods for calling either blk_end_bidi_request() or 
> blk_end_request() depending on whether there is a rsp request.
> 
> -- james s
> 
> 
> /**
>   * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
>   *                  completed
>   * @job:	fc_bsg_job that is complete
>   */
> static void
> fc_bsg_jobdone(struct fc_bsg_job *job)
> {
> 	struct request *req = job->req;
> 	struct request *rsp = req->next_rq;
> 	unsigned long flags;
> 	unsigned rsp_len = 0, req_len = blk_rq_bytes(req);
> 
> 	int err;
> 
> 	spin_lock_irqsave(&job->job_lock, flags);
> 	job->state_flags |= FC_RQST_STATE_DONE;
> 	job->ref_cnt--;
> 	spin_unlock_irqrestore(&job->job_lock, flags);
> 
> 	err = job->req->errors = job->reply->result;
> 	if (err < 0)
> 		/* we're only returning the result field in the reply */
> 		job->req->sense_len = sizeof(uint32_t);
> 	else
> 		job->req->sense_len = job->reply_len;
> 
> 	/* we assume all request payload was transferred, residual == 0 */
> 	req->data_len = 0;
> 
> 	if (rsp) {
> 		rsp_len = blk_rq_bytes(rsp);
> 		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
> 		/* set reply (bidi) residual */
> 		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
> 		blk_end_bidi_request(req, err, req_len, rsp_len);
> 	} else
> 		blk_end_request(req, err, req_len);
> 
> 	fc_destroy_bsgjob(job);
> }

No! Please most certainly don't. this is the all issue here. blk_end_request
is just a wrapper for an internal blk_end_bidi_request(). They are essentially
the same exact call when the first just puts a zero at bidi_bytes.

My code is the preferred and easiest and most correct way. Again calling
blk_end_bidi_request() is absolutely always safe, only thing missing is a proper
comment for blk_end_bidi_request() that says so.

Cheers
Boaz

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Mar 14, 2009, at 3:16 PM, James Smart wrote:

> Thank you Seokmann. I'm glad the RFC worked well for you.  Your  
> patch is
> appreciated, it was the portion that needed to be reverted as the
> collapsed blk_end_request() patch wasn't taken.  To double check -  
> there
> are no patches required with the RFC now - right ?
Yes, for those ELS/CT service handling through rports, it works for me.

Thank you,
Seokmann

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Mar 15, 2009, at 2:30 AM, Boaz Harrosh wrote:

> Seokmann Ju wrote:
>> On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:
>>
>>> On Feb 11, 2009, at 7:13 AM, James Smart wrote:
>>>
>>>> Trying to kick-start this again...
>>>> I've updated the prior RFC with the comments from Seokmann,
>>>> SvenFujita, and Boaz. I would still like review on the
>>>> blk_xxx completion calls in the std and error paths.
>>> I'm looking the driver changes to match it with latest updates in
>>> the transport layer.
>>> The driver with the changes should be available within a couple of
>>> weeks.
>> Here is a change that I've made for transport layer while I'm testing
>> the feature.
>> ---
>> diff --git a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/
>> scsi_transport_fc.c
>> index 63ab7bc..4bca164 100644
>> --- a/drivers/scsi/scsi_transport_fc.c
>> +++ b/drivers/scsi/scsi_transport_fc.c
>> @@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
>>                 BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
>>                 /* set reply (bidi) residual */
>>                 rsp->data_len = (rsp_len - job->reply-
>>> reply_payload_rcv_len);
>
> This part looks a bit white-space broken.
>
>> -       }
>> +               blk_end_bidi_request(req, err, blk_rq_bytes(req),
>> +                   job->reply->reply_payload_rcv_len);
>>
>> -       /* we assume all request payload was transferred */
>> -       blk_end_request(req, err, blk_rq_bytes(req));
>> +       } else
>> +               blk_end_request(req, err, blk_rq_bytes(req));
>
> I still do not like this version for more then one reason. There is
> a more proper simpler way.
>
> Where is the code-base this is patching? I'll make a proper patch  
> for you.
I took James S's patch from here.
http://lwn.net/Articles/318731/

Thank you,
Seokmann

Re: [RFC] FC pass thru - Rev V

[Boaz Harrosh]

Seokmann Ju wrote:
> On Mar 15, 2009, at 2:30 AM, Boaz Harrosh wrote:
> 
>> Seokmann Ju wrote:
>>> On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:
>>>
>>>> On Feb 11, 2009, at 7:13 AM, James Smart wrote:
>>>>
>>>>> Trying to kick-start this again...
>>>>> I've updated the prior RFC with the comments from Seokmann,
>>>>> SvenFujita, and Boaz. I would still like review on the
>>>>> blk_xxx completion calls in the std and error paths.
>>>> I'm looking the driver changes to match it with latest updates in
>>>> the transport layer.
>>>> The driver with the changes should be available within a couple of
>>>> weeks.
>>> Here is a change that I've made for transport layer while I'm testing
>>> the feature.
>>> ---
>>> diff --git a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/
>>> scsi_transport_fc.c
>>> index 63ab7bc..4bca164 100644
>>> --- a/drivers/scsi/scsi_transport_fc.c
>>> +++ b/drivers/scsi/scsi_transport_fc.c
>>> @@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
>>>                 BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
>>>                 /* set reply (bidi) residual */
>>>                 rsp->data_len = (rsp_len - job->reply-
>>>> reply_payload_rcv_len);
>> This part looks a bit white-space broken.
>>
>>> -       }
>>> +               blk_end_bidi_request(req, err, blk_rq_bytes(req),
>>> +                   job->reply->reply_payload_rcv_len);
>>>
>>> -       /* we assume all request payload was transferred */
>>> -       blk_end_request(req, err, blk_rq_bytes(req));
>>> +       } else
>>> +               blk_end_request(req, err, blk_rq_bytes(req));
>> I still do not like this version for more then one reason. There is
>> a more proper simpler way.
>>
>> Where is the code-base this is patching? I'll make a proper patch  
>> for you.
> I took James S's patch from here.
> http://lwn.net/Articles/318731/
> 

Not good, sorry. Quoted the original patch in link above please change
lines below

James Smart wrote:
> +static void
> +fc_bsg_jobdone(struct fc_bsg_job *job)
> +{
> +	struct request *req = job->req;
> +	struct request *rsp = req->next_rq;
> +	unsigned long flags;
> +	unsigned rsp_len;

-	unsigned rsp_len;
+	unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

> +	int err;
> +
> +	spin_lock_irqsave(&job->job_lock, flags);
> +	job->state_flags |= FC_RQST_STATE_DONE;
> +	job->ref_cnt--;
> +	spin_unlock_irqrestore(&job->job_lock, flags);
> +
> +	err = job->req->errors = job->reply->result;
> +	if (err < 0)
> +		/* we're only returning the result field in the reply */
> +		job->req->sense_len = sizeof(uint32_t);
> +	else
> +		job->req->sense_len = job->reply_len;
> +
> +	if (rsp) {
> +		rsp_len = blk_rq_bytes(rsp);
> +		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
> +		/* set reply (bidi) residual */
> +		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
> +	}
> +
> +	/* we assume all request payload was transferred */

-	/* we assume all request payload was transferred */
+	/* we assume all request payload was transferred, residual == 0 */
+	req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer
residual of all payload. On the other hand you might have more precise residual
information here, as received from the scsi transport? (Specially in the error
case, perhaps in the future).

> +	blk_end_request(req, err, blk_rq_bytes(req));

-	blk_end_request(req, err, blk_rq_bytes(req));
+	blk_end_bidi_request(req, err, req_len, rsp_len);


> +
> +	fc_destroy_bsgjob(job);
> +}
> Thank you,
> Seokmann
> 

This should take care of all possible cases, (bidi or otherwise), and will
return the proper transfer residual count of 0 to user-mode.

Thanks
Boaz

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Mar 16, 2009, at 6:38 AM, Boaz Harrosh wrote:

> Seokmann Ju wrote:
<snip>
>> I took James S's patch from here.
>> http://lwn.net/Articles/318731/
>>
>
> Not good, sorry. Quoted the original patch in link above please change
> lines below
Yes, Now I've got all relevant emails that you've been posted over the  
weekend.
My email seems a bit slow today for some reason.

Thank you,
Seokmann

Re: [RFC] FC pass thru - Rev V

[FUJITA Tomonori]

On Sun, 15 Mar 2009 11:34:59 +0200
Boaz Harrosh <bharrosh@panasas.com> wrote:

> Sven Schuetz wrote:
> > Hi Seokmann,
> >> Do we have time table for this support to make it available in the 
> >> upstream?
> >> I think it would be good to have the feature in 2.6.30 kernel, if 
> >> possible.
> >> I will continue to work on and add rest of helper function in the driver.
> >>
> >> Thank you,
> >> Seokmann
> >>
> >> ---
> > 
> > I am preparing a patch for our zfcp driver and would love to see both 
> > James&your fc_transport patch as well as the patches for our drivers in 
> > 2.6.30 too :)
> > 
> > Tomo, Boaz - since the current implemantation is based on that patch:
> > http://marc.info/?l-linux-scsi&m=122785157116659&w=2
> > which is not agreed upon - can we start the discussion again?
> > My understanding from the discussion back then was that Boaz proposed a 
> > change to the patch which Tomo agreed to. See here: 
> > http://marc.info/?l=linux-scsi&m=122812818916080&w=2
> > However, a new version of the patch was never posted. So would a patch 
> > with the proposed change be ok for the two of you which could be posted 
> > to Jens?
> > I really appreciate all of your efforts, thanks a lot for that :-)
> > Have a nice weekend,
> > Sven
> 
> My humble opinion is: That none of the patches should go in. It is easy
> enough to use the API as it is, and it is more correct then any of the
> suggested alternatives.

It's pretty clear that the current API is pretty confusing. If not,
why do you need to tell SCSI developers how to use it again and again?

Please don't say 'add comments along with the API'. Commenting doesn't
fix the confusing API. People can use good API without any
explanation.

But it's better to live with the current API for now. We can fix the
API latter. I'll read Tejun's block cleanup patchset and figure out
what to do.

Re: [RFC] FC pass thru - Rev V

[Seokmann Ju]

On Mar 14, 2009, at 3:16 PM, James Smart wrote:

> Thank you Seokmann. I'm glad the RFC worked well for you.  Your  
> patch is
> appreciated, it was the portion that needed to be reverted as the
> collapsed blk_end_request() patch wasn't taken.  To double check -  
> there
> are no patches required with the RFC now - right ?
>
> I'll repost as a non-RFC, and we'll see where James takes it.
Could you share your plan on this?
Just trying to figure out when we will have it in the upstream tree,  
hoping that 2.6.30 is viable target.

Thank you,
Seokmann