* [PATCH v4 0/9] snic:initial submission of snic driver for Cisco SCSI HBA
@ 2015-04-09 11:49 Narsimhulu Musini
2015-04-09 11:49 ` [PATCH v4 1/9] snic: snic module infrastructure Narsimhulu Musini
` (8 more replies)
0 siblings, 9 replies; 24+ messages in thread
From: Narsimhulu Musini @ 2015-04-09 11:49 UTC (permalink / raw)
To: JBottomley, linux-scsi, hare, hch; +Cc: Narsimhulu Musini
The following patch series adds snic driver for Cisco SCSI HBA
Cisco has developed a new PCI HBA interface called sNIC, which stands for
SCSI NIC. This is a new storage feature supported on specialized network
adapter. The new PCI function provides a uniform host interface and abstracts
backend storage.
* v4
- Incorporated Christoph Hellwig's comments.
* v3
- Incorporated Dr. Hannes Reinecke's review comments.
* v2
- Incorporated Dr. Hannes Reinecke's review comments.
Thanks
Narsimhulu Musini & Sesidhar Baddela
Narsimhulu Musini (9):
snic: snic module infrastructure
snic:Add interrupt, resource firmware interfaces
snic:Add meta request, handling of meta requests.
snic:Add snic target discovery
snic:add SCSI handling, AEN, and fwreset handling
snic:Add low level queuing interfaces
snic:Add sysfs entries to list stats and trace data
snic:Add event tracing to capture IO events.
snic:Add Makefile, patch Kconfig, MAINTAINERS
MAINTAINERS | 7 +
drivers/scsi/Kconfig | 17 +
drivers/scsi/Makefile | 1 +
drivers/scsi/snic/Makefile | 21 +
drivers/scsi/snic/cq_desc.h | 76 ++
drivers/scsi/snic/cq_enet_desc.h | 38 +
drivers/scsi/snic/snic.h | 415 ++++++
drivers/scsi/snic/snic_attrs.c | 79 ++
drivers/scsi/snic/snic_ctl.c | 276 ++++
drivers/scsi/snic/snic_debugfs.c | 562 ++++++++
drivers/scsi/snic/snic_disc.c | 602 +++++++++
drivers/scsi/snic/snic_disc.h | 124 ++
drivers/scsi/snic/snic_fwint.h | 525 ++++++++
drivers/scsi/snic/snic_io.c | 519 ++++++++
drivers/scsi/snic/snic_io.h | 118 ++
drivers/scsi/snic/snic_isr.c | 209 +++
drivers/scsi/snic/snic_main.c | 1039 +++++++++++++++
drivers/scsi/snic/snic_res.c | 297 +++++
drivers/scsi/snic/snic_res.h | 96 ++
drivers/scsi/snic/snic_scsi.c | 2638 +++++++++++++++++++++++++++++++++++++
drivers/scsi/snic/snic_stats.h | 123 ++
drivers/scsi/snic/snic_trc.c | 182 +++
drivers/scsi/snic/snic_trc.h | 121 ++
drivers/scsi/snic/vnic_cq.c | 86 ++
drivers/scsi/snic/vnic_cq.h | 120 ++
drivers/scsi/snic/vnic_cq_fw.h | 62 +
drivers/scsi/snic/vnic_dev.c | 749 +++++++++++
drivers/scsi/snic/vnic_dev.h | 140 ++
drivers/scsi/snic/vnic_devcmd.h | 270 ++++
drivers/scsi/snic/vnic_intr.c | 59 +
drivers/scsi/snic/vnic_intr.h | 119 ++
drivers/scsi/snic/vnic_resource.h | 68 +
drivers/scsi/snic/vnic_snic.h | 54 +
drivers/scsi/snic/vnic_stats.h | 68 +
drivers/scsi/snic/vnic_wq.c | 236 ++++
drivers/scsi/snic/vnic_wq.h | 187 +++
drivers/scsi/snic/wq_enet_desc.h | 91 ++
37 files changed, 10394 insertions(+)
create mode 100644 drivers/scsi/snic/Makefile
create mode 100644 drivers/scsi/snic/cq_desc.h
create mode 100644 drivers/scsi/snic/cq_enet_desc.h
create mode 100644 drivers/scsi/snic/snic.h
create mode 100644 drivers/scsi/snic/snic_attrs.c
create mode 100644 drivers/scsi/snic/snic_ctl.c
create mode 100644 drivers/scsi/snic/snic_debugfs.c
create mode 100644 drivers/scsi/snic/snic_disc.c
create mode 100644 drivers/scsi/snic/snic_disc.h
create mode 100644 drivers/scsi/snic/snic_fwint.h
create mode 100644 drivers/scsi/snic/snic_io.c
create mode 100644 drivers/scsi/snic/snic_io.h
create mode 100644 drivers/scsi/snic/snic_isr.c
create mode 100644 drivers/scsi/snic/snic_main.c
create mode 100644 drivers/scsi/snic/snic_res.c
create mode 100644 drivers/scsi/snic/snic_res.h
create mode 100644 drivers/scsi/snic/snic_scsi.c
create mode 100644 drivers/scsi/snic/snic_stats.h
create mode 100644 drivers/scsi/snic/snic_trc.c
create mode 100644 drivers/scsi/snic/snic_trc.h
create mode 100644 drivers/scsi/snic/vnic_cq.c
create mode 100644 drivers/scsi/snic/vnic_cq.h
create mode 100644 drivers/scsi/snic/vnic_cq_fw.h
create mode 100644 drivers/scsi/snic/vnic_dev.c
create mode 100644 drivers/scsi/snic/vnic_dev.h
create mode 100644 drivers/scsi/snic/vnic_devcmd.h
create mode 100644 drivers/scsi/snic/vnic_intr.c
create mode 100644 drivers/scsi/snic/vnic_intr.h
create mode 100644 drivers/scsi/snic/vnic_resource.h
create mode 100644 drivers/scsi/snic/vnic_snic.h
create mode 100644 drivers/scsi/snic/vnic_stats.h
create mode 100644 drivers/scsi/snic/vnic_wq.c
create mode 100644 drivers/scsi/snic/vnic_wq.h
create mode 100644 drivers/scsi/snic/wq_enet_desc.h
--
1.8.5.4
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v4 1/9] snic: snic module infrastructure
2015-04-09 11:49 [PATCH v4 0/9] snic:initial submission of snic driver for Cisco SCSI HBA Narsimhulu Musini
@ 2015-04-09 11:49 ` Narsimhulu Musini
2015-04-09 12:40 ` Hannes Reinecke
2015-04-09 11:49 ` [PATCH v4 2/9] snic:Add interrupt, resource firmware interfaces Narsimhulu Musini
` (7 subsequent siblings)
8 siblings, 1 reply; 24+ messages in thread
From: Narsimhulu Musini @ 2015-04-09 11:49 UTC (permalink / raw)
To: JBottomley, linux-scsi, hare, hch; +Cc: Narsimhulu Musini, Sesidhar Baddela
snic_main.c contains module load and unload, global driver context,
PCI Registration, PCI probe and remove, SCSI ML registration functionality.
snic.h contains snic structure definition, snic global context, and
prototypes.
snic_attrs.c contains device attributes to list snic state, link state,
and driver version under /sys/class/scsi_host/host<id>/
Signed-off-by: Narsimhulu Musini <nmusini@cisco.com>
Signed-off-by: Sesidhar Baddela <sebaddel@cisco.com>
---
* v4
- Replaced scratch pad usage with private data structure for internal use.
- Changed cmd_per_lun to default queue depth.
- Removed snic_os.h file and replaced the wrappers with direct linux API.
* v3
- Cleaned up unused functions, structure members, and redundant comments.
- Fixed a typo in time computation.
* v2
- Added Compile time macro for debugfs dependent functionality.
drivers/scsi/snic/snic.h | 415 ++++++++++++++++
drivers/scsi/snic/snic_attrs.c | 79 +++
drivers/scsi/snic/snic_main.c | 1039 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 1533 insertions(+)
create mode 100644 drivers/scsi/snic/snic.h
create mode 100644 drivers/scsi/snic/snic_attrs.c
create mode 100644 drivers/scsi/snic/snic_main.c
diff --git a/drivers/scsi/snic/snic.h b/drivers/scsi/snic/snic.h
new file mode 100644
index 0000000..533abf0
--- /dev/null
+++ b/drivers/scsi/snic/snic.h
@@ -0,0 +1,415 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _SNIC_H_
+#define _SNIC_H_
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/workqueue.h>
+#include <linux/bitops.h>
+#include <linux/mempool.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_host.h>
+
+#include "snic_disc.h"
+#include "snic_io.h"
+#include "snic_res.h"
+#include "snic_trc.h"
+#include "snic_stats.h"
+#include "vnic_dev.h"
+#include "vnic_wq.h"
+#include "vnic_cq.h"
+#include "vnic_intr.h"
+#include "vnic_stats.h"
+#include "vnic_snic.h"
+
+#define SNIC_DRV_NAME "snic"
+#define SNIC_DRV_DESCRIPTION "Cisco SCSI NIC Driver"
+#define SNIC_DRV_VERSION "0.0.1.18"
+#define PFX SNIC_DRV_NAME ":"
+#define DFX SNIC_DRV_NAME "%d: "
+
+#define DESC_CLEAN_LOW_WATERMARK 8
+#define SNIC_UCSM_DFLT_THROTTLE_CNT_BLD 16 /* UCSM default throttle count */
+#define SNIC_MAX_IO_REQ 50 /* scsi_cmnd tag map entries */
+#define SNIC_MIN_IO_REQ 8 /* Min IO throttle count */
+#define SNIC_IO_LOCKS 64 /* IO locks: power of 2 */
+#define SNIC_DFLT_QUEUE_DEPTH 32 /* Default Queue Depth */
+#define SNIC_MAX_QUEUE_DEPTH 64 /* Max Queue Depth */
+#define SNIC_DFLT_CMD_TIMEOUT 90 /* Extended tmo for FW */
+
+/*
+ * Tag bits used for special requests.
+ */
+#define SNIC_TAG_ABORT BIT(30) /* Tag indicating abort */
+#define SNIC_TAG_DEV_RST BIT(29) /* Tag for device reset */
+#define SNIC_TAG_IOCTL_DEV_RST BIT(28) /* Tag for User Device Reset */
+#define SNIC_TAG_MASK (BIT(24) - 1) /* Mask for lookup */
+#define SNIC_NO_TAG -1
+
+/*
+ * Command flags to identify the type of command and for other future use
+ */
+#define SNIC_NO_FLAGS 0
+#define SNIC_IO_INITIALIZED BIT(0)
+#define SNIC_IO_ISSUED BIT(1)
+#define SNIC_IO_DONE BIT(2)
+#define SNIC_IO_REQ_NULL BIT(3)
+#define SNIC_IO_ABTS_PENDING BIT(4)
+#define SNIC_IO_ABORTED BIT(5)
+#define SNIC_IO_ABTS_ISSUED BIT(6)
+#define SNIC_IO_TERM_ISSUED BIT(7)
+#define SNIC_IO_ABTS_TIMEDOUT BIT(8)
+#define SNIC_IO_ABTS_TERM_DONE BIT(9)
+#define SNIC_IO_ABTS_TERM_REQ_NULL BIT(10)
+#define SNIC_IO_ABTS_TERM_TIMEDOUT BIT(11)
+#define SNIC_IO_INTERNAL_TERM_PENDING BIT(12)
+#define SNIC_IO_INTERNAL_TERM_ISSUED BIT(13)
+#define SNIC_DEVICE_RESET BIT(14)
+#define SNIC_DEV_RST_ISSUED BIT(15)
+#define SNIC_DEV_RST_TIMEDOUT BIT(16)
+#define SNIC_DEV_RST_ABTS_ISSUED BIT(17)
+#define SNIC_DEV_RST_TERM_ISSUED BIT(18)
+#define SNIC_DEV_RST_DONE BIT(19)
+#define SNIC_DEV_RST_REQ_NULL BIT(20)
+#define SNIC_DEV_RST_ABTS_DONE BIT(21)
+#define SNIC_DEV_RST_TERM_DONE BIT(22)
+#define SNIC_DEV_RST_ABTS_PENDING BIT(23)
+#define SNIC_DEV_RST_PENDING BIT(24)
+#define SNIC_DEV_RST_NOTSUP BIT(25)
+#define SNIC_SCSI_CLEANUP BIT(26)
+#define SNIC_HOST_RESET_ISSUED BIT(27)
+
+#define SNIC_ABTS_TIMEOUT 30000 /* msec */
+#define SNIC_LUN_RESET_TIMEOUT 30000 /* msec */
+#define SNIC_HOST_RESET_TIMEOUT 30000 /* msec */
+
+
+/*
+ * These are protected by the hashed req_lock.
+ */
+#define CMD_SP(Cmnd) \
+ (((struct snic_internal_io_state *)scsi_cmd_priv(Cmnd))->rqi)
+#define CMD_STATE(Cmnd) \
+ (((struct snic_internal_io_state *)scsi_cmd_priv(Cmnd))->state)
+#define CMD_ABTS_STATUS(Cmnd) \
+ (((struct snic_internal_io_state *)scsi_cmd_priv(Cmnd))->abts_status)
+#define CMD_LR_STATUS(Cmnd) \
+ (((struct snic_internal_io_state *)scsi_cmd_priv(Cmnd))->lr_status)
+#define CMD_FLAGS(Cmnd) \
+ (((struct snic_internal_io_state *)scsi_cmd_priv(Cmnd))->flags)
+
+#define SNIC_INVALID_CODE 0x100 /* Hdr Status val unused by firmware */
+
+#define SNIC_MAX_TARGET 256
+#define SNIC_FLAGS_NONE (0)
+
+/* snic module params */
+extern unsigned int snic_max_qdepth;
+
+/* snic debugging */
+extern unsigned int snic_log_level;
+
+#define SNIC_MAIN_LOGGING 0x1
+#define SNIC_SCSI_LOGGING 0x2
+#define SNIC_ISR_LOGGING 0x8
+#define SNIC_DESC_LOGGING 0x10
+
+#define SNIC_CHECK_LOGGING(LEVEL, CMD) \
+do { \
+ if (unlikely(snic_log_level & LEVEL)) \
+ do { \
+ CMD; \
+ } while (0); \
+} while (0)
+
+#define SNIC_MAIN_DBG(host, fmt, args...) \
+ SNIC_CHECK_LOGGING(SNIC_MAIN_LOGGING, \
+ shost_printk(KERN_INFO, host, fmt, ## args);)
+
+#define SNIC_SCSI_DBG(host, fmt, args...) \
+ SNIC_CHECK_LOGGING(SNIC_SCSI_LOGGING, \
+ shost_printk(KERN_INFO, host, fmt, ##args);)
+
+#define SNIC_DISC_DBG(host, fmt, args...) \
+ SNIC_CHECK_LOGGING(SNIC_SCSI_LOGGING, \
+ shost_printk(KERN_INFO, host, fmt, ##args);)
+
+#define SNIC_ISR_DBG(host, fmt, args...) \
+ SNIC_CHECK_LOGGING(SNIC_ISR_LOGGING, \
+ shost_printk(KERN_INFO, host, fmt, ##args);)
+
+#define SNIC_HOST_ERR(host, fmt, args...) \
+ shost_printk(KERN_ERR, host, fmt, ##args)
+
+#define SNIC_HOST_INFO(host, fmt, args...) \
+ shost_printk(KERN_INFO, host, fmt, ##args)
+
+#define SNIC_INFO(fmt, args...) \
+ pr_info(PFX fmt, ## args)
+
+#define SNIC_DBG(fmt, args...) \
+ pr_info(PFX fmt, ## args)
+
+#define SNIC_ERR(fmt, args...) \
+ pr_err(PFX fmt, ## args)
+
+#ifdef DEBUG
+#define SNIC_BUG_ON(EXPR) \
+ ({ \
+ if (EXPR) { \
+ SNIC_ERR("SNIC BUG(%s)\n", #EXPR); \
+ BUG_ON(EXPR); \
+ } \
+ })
+#else
+#define SNIC_BUG_ON(EXPR) \
+ ({ \
+ if (EXPR) { \
+ SNIC_ERR("SNIC BUG(%s) at %s : %d\n", \
+ #EXPR, __func__, __LINE__); \
+ WARN_ON_ONCE(EXPR); \
+ } \
+ })
+#endif
+
+/* Soft assert */
+#define SNIC_ASSERT_NOT_IMPL(EXPR) \
+ ({ \
+ if (EXPR) {\
+ SNIC_INFO("Functionality not impl'ed at %s:%d\n", \
+ __func__, __LINE__); \
+ WARN_ON_ONCE(EXPR); \
+ } \
+ })
+
+
+extern const char *snic_state_str[];
+
+enum snic_intx_intr_index {
+ SNIC_INTX_WQ_RQ_COPYWQ,
+ SNIC_INTX_ERR,
+ SNIC_INTX_NOTIFY,
+ SNIC_INTX_INTR_MAX,
+};
+
+enum snic_msix_intr_index {
+ SNIC_MSIX_WQ,
+ SNIC_MSIX_IO_CMPL,
+ SNIC_MSIX_ERR_NOTIFY,
+ SNIC_MSIX_INTR_MAX,
+};
+
+struct snic_msix_entry {
+ int requested;
+ char devname[IFNAMSIZ];
+ irqreturn_t (*isr)(int, void *);
+ void *devid;
+};
+
+enum snic_state {
+ SNIC_INIT = 0,
+ SNIC_ERROR,
+ SNIC_ONLINE,
+ SNIC_OFFLINE,
+ SNIC_FWRESET,
+};
+
+#define SNIC_WQ_MAX 1
+#define SNIC_CQ_IO_CMPL_MAX 1
+#define SNIC_CQ_MAX (SNIC_WQ_MAX + SNIC_CQ_IO_CMPL_MAX)
+
+/* firmware version information */
+struct snic_fw_info {
+ u32 fw_ver;
+ u32 hid; /* u16 hid | u16 vnic id */
+ u32 max_concur_ios; /* max concurrent ios */
+ u32 max_sgs_per_cmd; /* max sgls per IO */
+ u32 max_io_sz; /* max io size supported */
+ u32 hba_cap; /* hba capabilities */
+ u32 max_tgts; /* max tgts supported */
+ u16 io_tmo; /* FW Extended timeout */
+ struct completion *wait; /* protected by snic lock*/
+};
+
+/*
+ * snic_work item : defined to process asynchronous events
+ */
+struct snic_work {
+ struct work_struct work;
+ u16 ev_id;
+ u64 *ev_data;
+};
+
+/*
+ * snic structure to represent SCSI vNIC
+ */
+struct snic {
+ /* snic specific members */
+ struct list_head list;
+ char name[IFNAMSIZ];
+ atomic_t state;
+ spinlock_t snic_lock;
+ struct completion *remove_wait;
+ bool in_remove;
+ bool stop_link_events; /* stop processing link events */
+
+ /* discovery related */
+ struct snic_disc disc;
+
+ /* Scsi Host info */
+ struct Scsi_Host *shost;
+
+ /* vnic related structures */
+ struct vnic_dev_bar bar0;
+
+ struct vnic_stats *stats;
+ unsigned long stats_time;
+ unsigned long stats_reset_time;
+
+ struct vnic_dev *vdev;
+
+ /* hw resource info */
+ unsigned int wq_count;
+ unsigned int cq_count;
+ unsigned int intr_count;
+ unsigned int err_intr_offset;
+
+ int link_status; /* retrieved from vnic_dev_link_status() */
+ u32 link_down_cnt;
+
+ /* pci related */
+ struct pci_dev *pdev;
+ struct msix_entry msix_entry[SNIC_MSIX_INTR_MAX];
+ struct snic_msix_entry msix[SNIC_MSIX_INTR_MAX];
+
+ /* io related info */
+ mempool_t *req_pool[SNIC_REQ_MAX_CACHES]; /* (??) */
+ ____cacheline_aligned spinlock_t io_req_lock[SNIC_IO_LOCKS];
+
+ /* Maintain snic specific commands, cmds with no tag in spl_cmd_list */
+ ____cacheline_aligned spinlock_t spl_cmd_lock;
+ struct list_head spl_cmd_list;
+
+ unsigned int max_tag_id;
+ atomic_t ios_inflight; /* io in flight counter */
+
+ struct vnic_snic_config config;
+
+ struct work_struct link_work;
+
+ /* firmware information */
+ struct snic_fw_info fwinfo;
+
+ /* Work for processing Target related work */
+ struct work_struct tgt_work;
+
+ /* Work for processing Discovery */
+ struct work_struct disc_work;
+
+ /* stats related */
+ unsigned int reset_stats;
+ atomic64_t io_cmpl_skip;
+ struct snic_stats s_stats; /* Per SNIC driver stats */
+
+ /* platform specific */
+#ifdef SNIC_DEBUG_FS
+ struct dentry *stats_host; /* Per snic debugfs root */
+ struct dentry *stats_file; /* Per snic debugfs file */
+ struct dentry *reset_stats_file;/* Per snic reset stats file */
+#endif
+
+ /* completion queue cache line section */
+ ____cacheline_aligned struct vnic_cq cq[SNIC_CQ_MAX];
+
+ /* work queue cache line section */
+ ____cacheline_aligned struct vnic_wq wq[SNIC_WQ_MAX];
+ spinlock_t wq_lock[SNIC_WQ_MAX];
+
+ /* interrupt resource cache line section */
+ ____cacheline_aligned struct vnic_intr intr[SNIC_MSIX_INTR_MAX];
+}; /* end of snic structure */
+
+/*
+ * SNIC Driver's Global Data
+ */
+struct snic_global {
+ struct list_head snic_list;
+ spinlock_t snic_list_lock;
+
+ struct kmem_cache *req_cache[SNIC_REQ_MAX_CACHES];
+
+ struct workqueue_struct *event_q;
+
+#ifdef SNIC_DEBUG_FS
+ /* debugfs related global data */
+ struct dentry *trc_root;
+ struct dentry *stats_root;
+
+ struct snic_trc trc ____cacheline_aligned;
+
+#endif
+};
+
+extern struct snic_global *snic_glob;
+
+int snic_glob_init(void);
+void snic_glob_cleanup(void);
+
+extern struct workqueue_struct *snic_event_queue;
+extern struct device_attribute *snic_attrs[];
+
+int snic_queuecommand(struct Scsi_Host *, struct scsi_cmnd *);
+int snic_abort_cmd(struct scsi_cmnd *);
+int snic_device_reset(struct scsi_cmnd *);
+int snic_host_reset(struct scsi_cmnd *);
+int snic_reset(struct Scsi_Host *, struct scsi_cmnd *);
+void snic_shutdown_scsi_cleanup(struct snic *);
+
+
+int snic_request_intr(struct snic *);
+void snic_free_intr(struct snic *);
+int snic_set_intr_mode(struct snic *);
+void snic_clear_intr_mode(struct snic *);
+
+int snic_fwcq_cmpl_handler(struct snic *, int);
+int snic_wq_cmpl_handler(struct snic *, int);
+void snic_free_wq_buf(struct vnic_wq *, struct vnic_wq_buf *);
+
+
+void snic_log_q_error(struct snic *);
+void snic_handle_link_event(struct snic *);
+void snic_handle_link(struct work_struct *);
+
+int snic_queue_exch_ver_req(struct snic *);
+int snic_io_exch_ver_cmpl_handler(struct snic *, struct snic_fw_req *);
+
+int snic_queue_wq_desc(struct snic *, void *os_buf, u16 len);
+
+void snic_handle_untagged_req(struct snic *, struct snic_req_info *);
+void snic_release_untagged_req(struct snic *, struct snic_req_info *);
+void snic_free_all_untagged_reqs(struct snic *);
+int snic_get_conf(struct snic *);
+void snic_set_state(struct snic *, enum snic_state);
+int snic_get_state(struct snic *);
+const char *snic_state_to_str(unsigned int);
+void snic_hex_dump(char *, char *, int);
+void snic_print_desc(const char *fn, char *os_buf, int len);
+const char *show_opcode_name(int val);
+#endif /* _SNIC_H */
diff --git a/drivers/scsi/snic/snic_attrs.c b/drivers/scsi/snic/snic_attrs.c
new file mode 100644
index 0000000..88822ad
--- /dev/null
+++ b/drivers/scsi/snic/snic_attrs.c
@@ -0,0 +1,79 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ * [Insert appropriate license here when releasing outside of Cisco]
+ *
+ */
+
+#include <linux/string.h>
+#include <linux/device.h>
+
+#include "snic.h"
+
+static ssize_t
+snic_show_sym_name(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct snic *snic = shost_priv(class_to_shost(dev));
+
+ return snprintf(buf, PAGE_SIZE, "%s\n", snic->name);
+}
+
+static ssize_t
+snic_show_state(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct snic *snic = shost_priv(class_to_shost(dev));
+
+ return snprintf(buf, PAGE_SIZE, "%s\n",
+ snic_state_str[snic_get_state(snic)]);
+}
+
+static ssize_t
+snic_show_drv_version(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%s\n", SNIC_DRV_VERSION);
+}
+
+static ssize_t
+snic_show_link_state(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct snic *snic = shost_priv(class_to_shost(dev));
+
+ if (snic->config.xpt_type == SNIC_DAS)
+ snic->link_status = vnic_dev_link_status(snic->vdev);
+
+ return snprintf(buf, PAGE_SIZE, "%s\n",
+ (snic->link_status) ? "Link Up" : "Link Down");
+}
+
+static DEVICE_ATTR(snic_sym_name, S_IRUGO, snic_show_sym_name, NULL);
+static DEVICE_ATTR(snic_state, S_IRUGO, snic_show_state, NULL);
+static DEVICE_ATTR(drv_version, S_IRUGO, snic_show_drv_version, NULL);
+static DEVICE_ATTR(link_state, S_IRUGO, snic_show_link_state, NULL);
+
+struct device_attribute *snic_attrs[] = {
+ &dev_attr_snic_sym_name,
+ &dev_attr_snic_state,
+ &dev_attr_drv_version,
+ &dev_attr_link_state,
+ NULL,
+};
diff --git a/drivers/scsi/snic/snic_main.c b/drivers/scsi/snic/snic_main.c
new file mode 100644
index 0000000..a815377
--- /dev/null
+++ b/drivers/scsi/snic/snic_main.c
@@ -0,0 +1,1039 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/module.h>
+#include <linux/mempool.h>
+#include <linux/string.h>
+#include <linux/slab.h>
+#include <linux/errno.h>
+#include <linux/init.h>
+#include <linux/pci.h>
+#include <linux/skbuff.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/workqueue.h>
+#include <scsi/scsi_host.h>
+#include <scsi/scsi_tcq.h>
+
+#include "snic.h"
+#include "snic_fwint.h"
+
+#define PCI_DEVICE_ID_CISCO_SNIC 0x0046
+
+/* Supported devices by snic module */
+static struct pci_device_id snic_id_table[] = {
+ {PCI_DEVICE(0x1137, PCI_DEVICE_ID_CISCO_SNIC) },
+ { 0, } /* end of table */
+};
+
+unsigned int snic_log_level = 0x0;
+module_param(snic_log_level, int, S_IRUGO|S_IWUSR);
+MODULE_PARM_DESC(snic_log_level, "bitmask for snic logging levels");
+
+#ifdef SNIC_DEBUG_FS
+unsigned int snic_trace_max_pages = 16;
+module_param(snic_trace_max_pages, uint, S_IRUGO|S_IWUSR);
+MODULE_PARM_DESC(snic_trace_max_pages,
+ "Total allocated memory pages for snic trace buffer");
+
+#endif
+unsigned int snic_max_qdepth = SNIC_DFLT_QUEUE_DEPTH;
+module_param(snic_max_qdepth, uint, S_IRUGO | S_IWUSR);
+MODULE_PARM_DESC(snic_max_qdepth, "Queue depth to report for each LUN");
+
+/*
+ * snic_slave_alloc : callback function to SCSI Mid Layer, called on
+ * scsi device initialization.
+ */
+static int
+snic_slave_alloc(struct scsi_device *sdev)
+{
+ struct snic_tgt *tgt = starget_to_tgt(scsi_target(sdev));
+
+ if (!tgt || snic_tgt_chkready(tgt))
+ return -ENXIO;
+
+ return 0;
+}
+
+/*
+ * snic_slave_configure : callback function to SCSI Mid Layer, called on
+ * scsi device initialization.
+ */
+static int
+snic_slave_configure(struct scsi_device *sdev)
+{
+ struct snic *snic = shost_priv(sdev->host);
+ u32 qdepth = 0, max_ios = 0;
+ int tmo = SNIC_DFLT_CMD_TIMEOUT * HZ;
+
+ /* Set Queue Depth */
+ max_ios = snic_max_qdepth;
+ qdepth = min_t(u32, max_ios, SNIC_MAX_QUEUE_DEPTH);
+ scsi_change_queue_depth(sdev, qdepth);
+
+ if (snic->fwinfo.io_tmo > 1)
+ tmo = snic->fwinfo.io_tmo * HZ;
+
+ /* FW requires extended timeouts */
+ blk_queue_rq_timeout(sdev->request_queue, tmo);
+
+ return 0;
+}
+
+static int
+snic_change_queue_depth(struct scsi_device *sdev, int qdepth)
+{
+ int qsz = 0;
+
+ qsz = min_t(u32, qdepth, SNIC_MAX_QUEUE_DEPTH);
+ scsi_change_queue_depth(sdev, qsz);
+ SNIC_INFO("QDepth Changed to %d\n", sdev->queue_depth);
+
+ return sdev->queue_depth;
+}
+
+static struct scsi_host_template snic_host_template = {
+ .module = THIS_MODULE,
+ .name = SNIC_DRV_NAME,
+ .queuecommand = snic_queuecommand,
+ .eh_abort_handler = snic_abort_cmd,
+ .eh_device_reset_handler = snic_device_reset,
+ .eh_host_reset_handler = snic_host_reset,
+ .slave_alloc = snic_slave_alloc,
+ .slave_configure = snic_slave_configure,
+ .change_queue_depth = snic_change_queue_depth,
+ .this_id = -1,
+ .cmd_per_lun = SNIC_DFLT_QUEUE_DEPTH,
+ .can_queue = SNIC_MAX_IO_REQ,
+ .use_clustering = ENABLE_CLUSTERING,
+ .sg_tablesize = SNIC_MAX_SG_DESC_CNT,
+ .max_sectors = 0x800,
+ .shost_attrs = snic_attrs,
+ .use_blk_tags = 1,
+ .track_queue_depth = 1,
+ .cmd_size = sizeof(struct snic_internal_io_state),
+ .proc_name = "snic_scsi",
+};
+
+/*
+ * snic_handle_link_event : Handles link events such as link up/down/error
+ */
+void
+snic_handle_link_event(struct snic *snic)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&snic->snic_lock, flags);
+ if (snic->stop_link_events) {
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+
+ return;
+ }
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+
+ queue_work(snic_glob->event_q, &snic->link_work);
+} /* end of snic_handle_link_event */
+
+/*
+ * snic_notify_set : sets notification area
+ * This notification area is to receive events from fw
+ * Note: snic supports only MSIX interrupts, in which we can just call
+ * vnic_dev_notify_set directly
+ */
+static int
+snic_notify_set(struct snic *snic)
+{
+ int ret = 0;
+ enum vnic_dev_intr_mode intr_mode;
+
+ intr_mode = vnic_dev_get_intr_mode(snic->vdev);
+
+ if (intr_mode == VNIC_DEV_INTR_MODE_MSIX) {
+ ret = vnic_dev_notify_set(snic->vdev, SNIC_MSIX_ERR_NOTIFY);
+ } else {
+ SNIC_HOST_ERR(snic->shost,
+ "Interrupt mode should be setup before devcmd notify set %d\n",
+ intr_mode);
+ ret = -1;
+ }
+
+ return ret;
+} /* end of snic_notify_set */
+
+/*
+ * snic_dev_wait : polls vnic open status.
+ */
+static int
+snic_dev_wait(struct vnic_dev *vdev,
+ int (*start)(struct vnic_dev *, int),
+ int (*finished)(struct vnic_dev *, int *),
+ int arg)
+{
+ unsigned long time;
+ int ret, done;
+ int retry_cnt = 0;
+
+ ret = start(vdev, arg);
+ if (ret)
+ return ret;
+
+ /*
+ * Wait for func to complete...2 seconds max.
+ *
+ * Sometimes schedule_timeout_uninterruptible take long time
+ * to wakeup, which results skipping retry. The retry counter
+ * ensures to retry at least two times.
+ */
+ time = jiffies + (HZ * 2);
+ do {
+ ret = finished(vdev, &done);
+ if (ret)
+ return ret;
+
+ if (done)
+ return 0;
+ schedule_timeout_uninterruptible(HZ/10);
+ ++retry_cnt;
+ } while (time_after(time, jiffies) || (retry_cnt < 3));
+
+ return -ETIMEDOUT;
+} /* end of snic_dev_wait */
+
+/*
+ * snic_cleanup: called by snic_remove
+ * Stops the snic device, masks all interrupts, Completed CQ entries are
+ * drained. Posted WQ/RQ/Copy-WQ entries are cleanup
+ */
+static int
+snic_cleanup(struct snic *snic)
+{
+ unsigned int i;
+ int ret;
+
+ vnic_dev_disable(snic->vdev);
+ for (i = 0; i < snic->intr_count; i++)
+ vnic_intr_mask(&snic->intr[i]);
+
+ for (i = 0; i < snic->wq_count; i++) {
+ ret = vnic_wq_disable(&snic->wq[i]);
+ if (ret)
+ return ret;
+ }
+
+ /* Clean up completed IOs */
+ snic_fwcq_cmpl_handler(snic, -1);
+
+ snic_wq_cmpl_handler(snic, -1);
+
+ /* Clean up the IOs that have not completed */
+ for (i = 0; i < snic->wq_count; i++)
+ vnic_wq_clean(&snic->wq[i], snic_free_wq_buf);
+
+ for (i = 0; i < snic->cq_count; i++)
+ vnic_cq_clean(&snic->cq[i]);
+
+ for (i = 0; i < snic->intr_count; i++)
+ vnic_intr_clean(&snic->intr[i]);
+
+ /* Cleanup snic specific requests */
+ snic_free_all_untagged_reqs(snic);
+
+ /* Cleanup Pending SCSI commands */
+ snic_shutdown_scsi_cleanup(snic);
+
+ for (i = 0; i < SNIC_REQ_MAX_CACHES; i++)
+ mempool_destroy(snic->req_pool[i]);
+
+ return 0;
+} /* end of snic_cleanup */
+
+
+static void
+snic_iounmap(struct snic *snic)
+{
+ if (snic->bar0.vaddr)
+ iounmap(snic->bar0.vaddr);
+}
+
+/*
+ * snic_vdev_open_done : polls for vnic_dev_open cmd completion.
+ */
+static int
+snic_vdev_open_done(struct vnic_dev *vdev, int *done)
+{
+ struct snic *snic = vnic_dev_priv(vdev);
+ int ret;
+ int nretries = 5;
+
+ do {
+ ret = vnic_dev_open_done(vdev, done);
+ if (ret == 0)
+ break;
+
+ SNIC_HOST_INFO(snic->shost, "VNIC_DEV_OPEN Timedout.\n");
+ } while (nretries--);
+
+ return ret;
+} /* end of snic_vdev_open_done */
+
+/*
+ * snic_add_host : registers scsi host with ML
+ */
+static int
+snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
+{
+ int ret = 0;
+
+ ret = scsi_add_host(shost, &pdev->dev);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "snic: scsi_add_host failed. %d\n",
+ ret);
+
+ return ret;
+ }
+
+ SNIC_BUG_ON(shost->work_q != NULL);
+ snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d",
+ shost->host_no);
+ shost->work_q = create_singlethread_workqueue(shost->work_q_name);
+ if (!shost->work_q) {
+ SNIC_HOST_ERR(shost, "Failed to Create ScsiHost wq.\n");
+
+ ret = -ENOMEM;
+ }
+
+ return ret;
+} /* end of snic_add_host */
+
+static void
+snic_del_host(struct Scsi_Host *shost)
+{
+ if (!shost->work_q)
+ return;
+
+ destroy_workqueue(shost->work_q);
+ shost->work_q = NULL;
+ scsi_remove_host(shost);
+}
+
+int
+snic_get_state(struct snic *snic)
+{
+ return atomic_read(&snic->state);
+}
+
+void
+snic_set_state(struct snic *snic, enum snic_state state)
+{
+ SNIC_HOST_INFO(snic->shost, "snic state change from %s to %s\n",
+ snic_state_to_str(snic_get_state(snic)),
+ snic_state_to_str(state));
+
+ atomic_set(&snic->state, state);
+}
+
+/*
+ * snic_probe : Initialize the snic interface.
+ */
+static int
+snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ struct Scsi_Host *shost;
+ struct snic *snic;
+ mempool_t *pool;
+ unsigned long flags;
+ u32 max_ios = 0;
+ int ret, i;
+
+ /* Device Information */
+ SNIC_INFO("snic device %4x:%4x:%4x:%4x: ",
+ pdev->vendor, pdev->device, pdev->subsystem_vendor,
+ pdev->subsystem_device);
+
+ SNIC_INFO("snic device bus %x: slot %x: fn %x\n",
+ pdev->bus->number, PCI_SLOT(pdev->devfn),
+ PCI_FUNC(pdev->devfn));
+
+ /*
+ * Allocate SCSI Host and setup association between host, and snic
+ */
+ shost = scsi_host_alloc(&snic_host_template, sizeof(struct snic));
+ if (!shost) {
+ SNIC_ERR("Unable to alloc scsi_host\n");
+ ret = -ENOMEM;
+
+ goto prob_end;
+ }
+ snic = shost_priv(shost);
+ snic->shost = shost;
+
+ snprintf(snic->name, sizeof(snic->name) - 1, "%s%d", SNIC_DRV_NAME,
+ shost->host_no);
+
+ SNIC_HOST_INFO(shost,
+ "snic%d = %p shost = %p device bus %x: slot %x: fn %x\n",
+ shost->host_no, snic, shost, pdev->bus->number,
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn));
+#ifdef SNIC_DEBUG_FS
+ /* Per snic debugfs init */
+ ret = snic_stats_debugfs_init(snic);
+ if (ret) {
+ SNIC_HOST_ERR(snic->shost,
+ "Failed to initialize debugfs stats\n");
+ snic_stats_debugfs_remove(snic);
+ }
+#endif
+
+ /* Setup PCI Resources */
+ pci_set_drvdata(pdev, snic);
+ snic->pdev = pdev;
+
+ ret = pci_enable_device(pdev);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Cannot enable PCI Resources, aborting : %d\n",
+ ret);
+
+ goto err_free_snic;
+ }
+
+ ret = pci_request_regions(pdev, SNIC_DRV_NAME);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Cannot obtain PCI Resources, aborting : %d\n",
+ ret);
+
+ goto err_pci_disable;
+ }
+
+ pci_set_master(pdev);
+
+ /*
+ * Query PCI Controller on system for DMA addressing
+ * limitation for the device. Try 43-bit first, and
+ * fail to 32-bit.
+ */
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(43));
+ if (ret) {
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "No Usable DMA Configuration, aborting %d\n",
+ ret);
+
+ goto err_rel_regions;
+ }
+
+ ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Unable to obtain 32-bit DMA for consistent allocations, aborting: %d\n",
+ ret);
+
+ goto err_rel_regions;
+ }
+ } else {
+ ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(43));
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Unable to obtain 43-bit DMA for consistent allocations. aborting: %d\n",
+ ret);
+
+ goto err_rel_regions;
+ }
+ }
+
+
+ /* Map vNIC resources from BAR0 */
+ if (!(pci_resource_flags(pdev, 0) & IORESOURCE_MEM)) {
+ SNIC_HOST_ERR(shost, "BAR0 not memory mappable aborting.\n");
+
+ ret = -ENODEV;
+ goto err_rel_regions;
+ }
+
+ snic->bar0.vaddr = pci_iomap(pdev, 0, 0);
+ if (!snic->bar0.vaddr) {
+ SNIC_HOST_ERR(shost,
+ "Cannot memory map BAR0 res hdr aborting.\n");
+
+ ret = -ENODEV;
+ goto err_rel_regions;
+ }
+
+ snic->bar0.bus_addr = pci_resource_start(pdev, 0);
+ snic->bar0.len = pci_resource_len(pdev, 0);
+ SNIC_BUG_ON(snic->bar0.bus_addr == 0);
+
+ /* Devcmd2 Resource Allocation and Initialization */
+ snic->vdev = vnic_dev_alloc_discover(NULL, snic, pdev, &snic->bar0, 1);
+ if (!snic->vdev) {
+ SNIC_HOST_ERR(shost, "vNIC Resource Discovery Failed.\n");
+
+ ret = -ENODEV;
+ goto err_iounmap;
+ }
+
+ ret = vnic_dev_cmd_init(snic->vdev, 0);
+ if (ret) {
+ SNIC_HOST_INFO(shost, "Devcmd2 Init Failed. err = %d\n", ret);
+
+ goto err_vnic_unreg;
+ }
+
+ ret = snic_dev_wait(snic->vdev, vnic_dev_open, snic_vdev_open_done, 0);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "vNIC dev open failed, aborting. %d\n",
+ ret);
+
+ goto err_vnic_unreg;
+ }
+
+ ret = vnic_dev_init(snic->vdev, 0);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "vNIC dev init failed. aborting. %d\n",
+ ret);
+
+ goto err_dev_close;
+ }
+
+ /* Get vNIC information */
+ ret = snic_get_vnic_config(snic);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Get vNIC configuration failed, aborting. %d\n",
+ ret);
+
+ goto err_dev_close;
+ }
+
+ /* Configure Maximum Outstanding IO reqs */
+ max_ios = snic->config.io_throttle_count;
+ if (max_ios != SNIC_UCSM_DFLT_THROTTLE_CNT_BLD)
+ shost->can_queue = min_t(u32, SNIC_MAX_IO_REQ,
+ max_t(u32, SNIC_MIN_IO_REQ, max_ios));
+
+ snic->max_tag_id = shost->can_queue;
+
+ ret = scsi_init_shared_tag_map(shost, snic->max_tag_id);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Unable to alloc shared tag map. %d\n",
+ ret);
+
+ goto err_dev_close;
+ }
+
+ shost->max_lun = snic->config.luns_per_tgt;
+ shost->max_id = SNIC_MAX_TARGET;
+
+ shost->max_cmd_len = MAX_COMMAND_SIZE; /*defined in scsi_cmnd.h*/
+
+ snic_get_res_counts(snic);
+
+ /*
+ * Assumption: Only MSIx is supported
+ */
+ ret = snic_set_intr_mode(snic);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Failed to set intr mode aborting. %d\n",
+ ret);
+
+ goto err_dev_close;
+ }
+
+ ret = snic_alloc_vnic_res(snic);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Failed to alloc vNIC resources aborting. %d\n",
+ ret);
+
+ goto err_clear_intr;
+ }
+
+ /* Initialize specific lists */
+ INIT_LIST_HEAD(&snic->list);
+
+ /*
+ * spl_cmd_list for maintaining snic specific cmds
+ * such as EXCH_VER_REQ, REPORT_TARGETS etc
+ */
+ INIT_LIST_HEAD(&snic->spl_cmd_list);
+ spin_lock_init(&snic->spl_cmd_lock);
+
+ /* initialize all snic locks */
+ spin_lock_init(&snic->snic_lock);
+
+ for (i = 0; i < SNIC_WQ_MAX; i++)
+ spin_lock_init(&snic->wq_lock[i]);
+
+ for (i = 0; i < SNIC_IO_LOCKS; i++)
+ spin_lock_init(&snic->io_req_lock[i]);
+
+ pool = mempool_create_slab_pool(2,
+ snic_glob->req_cache[SNIC_REQ_CACHE_DFLT_SGL]);
+ if (!pool) {
+ SNIC_HOST_ERR(shost, "dflt sgl pool creation failed\n");
+
+ goto err_free_res;
+ }
+
+ snic->req_pool[SNIC_REQ_CACHE_DFLT_SGL] = pool;
+
+ pool = mempool_create_slab_pool(2,
+ snic_glob->req_cache[SNIC_REQ_CACHE_MAX_SGL]);
+ if (!pool) {
+ SNIC_HOST_ERR(shost, "max sgl pool creation failed\n");
+
+ goto err_free_dflt_sgl_pool;
+ }
+
+ snic->req_pool[SNIC_REQ_CACHE_MAX_SGL] = pool;
+
+ pool = mempool_create_slab_pool(2,
+ snic_glob->req_cache[SNIC_REQ_TM_CACHE]);
+ if (!pool) {
+ SNIC_HOST_ERR(shost, "snic tmreq info pool creation failed.\n");
+
+ goto err_free_max_sgl_pool;
+ }
+
+ snic->req_pool[SNIC_REQ_TM_CACHE] = pool;
+
+ /* Initialize snic state */
+ atomic_set(&snic->state, SNIC_INIT);
+
+ atomic_set(&snic->ios_inflight, 0);
+
+ /* Setup notification buffer area */
+ ret = snic_notify_set(snic);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Failed to alloc notify buffer aborting. %d\n",
+ ret);
+
+ goto err_free_tmreq_pool;
+ }
+
+ /*
+ * Initialization done with PCI system, hardware, firmware.
+ * Add shost to SCSI
+ */
+ ret = snic_add_host(shost, pdev);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Adding scsi host Failed ... exiting. %d\n",
+ ret);
+
+ goto err_notify_unset;
+ }
+
+ spin_lock_irqsave(&snic_glob->snic_list_lock, flags);
+ list_add_tail(&snic->list, &snic_glob->snic_list);
+ spin_unlock_irqrestore(&snic_glob->snic_list_lock, flags);
+
+ snic_disc_init(&snic->disc);
+ INIT_WORK(&snic->tgt_work, snic_handle_tgt_disc);
+ INIT_WORK(&snic->disc_work, snic_handle_disc);
+ INIT_WORK(&snic->link_work, snic_handle_link);
+
+ /* Enable all queues */
+ for (i = 0; i < snic->wq_count; i++)
+ vnic_wq_enable(&snic->wq[i]);
+
+ ret = vnic_dev_enable_wait(snic->vdev);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "vNIC dev enable failed w/ error %d\n",
+ ret);
+
+ goto err_vdev_enable;
+ }
+
+ ret = snic_request_intr(snic);
+ if (ret) {
+ SNIC_HOST_ERR(shost, "Unable to request irq. %d\n", ret);
+
+ goto err_req_intr;
+ }
+
+ for (i = 0; i < snic->intr_count; i++)
+ vnic_intr_unmask(&snic->intr[i]);
+
+ snic_set_state(snic, SNIC_ONLINE);
+
+ /* Get snic params */
+ ret = snic_get_conf(snic);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Failed to get snic io config from FW w err %d\n",
+ ret);
+
+ goto err_get_conf;
+ }
+
+ ret = snic_disc_start(snic);
+ if (ret) {
+ SNIC_HOST_ERR(shost, "snic_probe:Discovery Failed w err = %d\n",
+ ret);
+
+ goto err_get_conf;
+ }
+
+ SNIC_HOST_INFO(shost, "SNIC Device Probe Successful.\n");
+
+ return 0;
+
+err_get_conf:
+ snic_free_all_untagged_reqs(snic);
+
+ for (i = 0; i < snic->intr_count; i++)
+ vnic_intr_mask(&snic->intr[i]);
+
+ snic_free_intr(snic);
+
+err_req_intr:
+ vnic_dev_disable(snic->vdev);
+
+err_vdev_enable:
+ for (i = 0; i < snic->wq_count; i++) {
+ int rc = 0;
+
+ rc = vnic_wq_disable(&snic->wq[i]);
+ if (rc) {
+ SNIC_HOST_ERR(shost,
+ "WQ Disable Failed w/ err = %d\n", rc);
+
+ break;
+ }
+ }
+ snic_del_host(snic->shost);
+
+err_notify_unset:
+ vnic_dev_notify_unset(snic->vdev);
+
+err_free_tmreq_pool:
+ mempool_destroy(snic->req_pool[SNIC_REQ_TM_CACHE]);
+
+err_free_max_sgl_pool:
+ mempool_destroy(snic->req_pool[SNIC_REQ_CACHE_MAX_SGL]);
+
+err_free_dflt_sgl_pool:
+ mempool_destroy(snic->req_pool[SNIC_REQ_CACHE_DFLT_SGL]);
+
+err_free_res:
+ snic_free_vnic_res(snic);
+
+err_clear_intr:
+ snic_clear_intr_mode(snic);
+
+err_dev_close:
+ vnic_dev_close(snic->vdev);
+
+err_vnic_unreg:
+ vnic_dev_unregister(snic->vdev);
+
+err_iounmap:
+ snic_iounmap(snic);
+
+err_rel_regions:
+ pci_release_regions(pdev);
+
+err_pci_disable:
+ pci_disable_device(pdev);
+
+err_free_snic:
+#ifdef SNIC_DEBUG_FS
+ snic_stats_debugfs_remove(snic);
+#endif
+ scsi_host_put(shost);
+ pci_set_drvdata(pdev, NULL);
+
+prob_end:
+ SNIC_INFO("sNIC device : bus %d: slot %d: fn %d Registration Failed.\n",
+ pdev->bus->number, PCI_SLOT(pdev->devfn),
+ PCI_FUNC(pdev->devfn));
+
+ return ret;
+} /* end of snic_probe */
+
+
+/*
+ * snic_remove : invoked on unbinding the interface to cleanup the
+ * resources allocated in snic_probe on initialization.
+ */
+static void
+snic_remove(struct pci_dev *pdev)
+{
+ struct snic *snic = pci_get_drvdata(pdev);
+ unsigned long flags;
+
+ if (!snic) {
+ SNIC_INFO("sNIC dev: bus %d slot %d fn %d snic inst is null.\n",
+ pdev->bus->number, PCI_SLOT(pdev->devfn),
+ PCI_FUNC(pdev->devfn));
+
+ return;
+ }
+
+ /*
+ * Mark state so that the workqueue thread stops forwarding
+ * received frames and link events. ISR and other threads
+ * that can queue work items will also stop creating work
+ * items on the snic workqueue
+ */
+ snic_set_state(snic, SNIC_OFFLINE);
+ spin_lock_irqsave(&snic->snic_lock, flags);
+ snic->stop_link_events = 1;
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+
+ flush_workqueue(snic_glob->event_q);
+ snic_disc_term(snic);
+
+ spin_lock_irqsave(&snic->snic_lock, flags);
+ snic->in_remove = 1;
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+
+ /*
+ * This stops the snic device, masks all interrupts, Completed
+ * CQ entries are drained. Posted WQ/RQ/Copy-WQ entries are
+ * cleanup
+ */
+ snic_cleanup(snic);
+
+ spin_lock_irqsave(&snic_glob->snic_list_lock, flags);
+ list_del(&snic->list);
+ spin_unlock_irqrestore(&snic_glob->snic_list_lock, flags);
+
+ snic_tgt_del_all(snic);
+#ifdef SNIC_DEBUG_FS
+ snic_stats_debugfs_remove(snic);
+#endif
+ snic_del_host(snic->shost);
+
+ vnic_dev_notify_unset(snic->vdev);
+ snic_free_intr(snic);
+ snic_free_vnic_res(snic);
+ snic_clear_intr_mode(snic);
+ vnic_dev_close(snic->vdev);
+ vnic_dev_unregister(snic->vdev);
+ snic_iounmap(snic);
+ pci_release_regions(pdev);
+ pci_disable_device(pdev);
+ pci_set_drvdata(pdev, NULL);
+
+ /* this frees Scsi_Host and snic memory (continuous chunk) */
+ scsi_host_put(snic->shost);
+} /* end of snic_remove */
+
+
+struct snic_global *snic_glob;
+
+/*
+ * snic_global_data_init: Initialize SNIC Global Data
+ * Notes: All the global lists, variables should be part of global data
+ * this helps in debugging.
+ */
+static int
+snic_global_data_init(void)
+{
+ int ret = 0;
+ struct kmem_cache *cachep;
+ ssize_t len = 0;
+
+ snic_glob = kzalloc(sizeof(*snic_glob), GFP_KERNEL);
+
+ if (!snic_glob) {
+ SNIC_ERR("Failed to allocate Global Context.\n");
+
+ ret = -ENOMEM;
+ goto gdi_end;
+ }
+
+#ifdef SNIC_DEBUG_FS
+ /* Debugfs related Initialization */
+ /* Create debugfs entries for snic */
+ ret = snic_debugfs_init();
+ if (ret < 0) {
+ SNIC_ERR("Failed to create sysfs dir for tracing and stats.\n");
+ snic_debugfs_term();
+ /* continue even if it fails */
+ }
+
+ /* Trace related Initialization */
+ /* Allocate memory for trace buffer */
+ ret = snic_trc_init();
+ if (ret < 0) {
+ SNIC_ERR("Trace buffer init failed, SNIC tracing disabled\n");
+ snic_trc_free();
+ /* continue even if it fails */
+ }
+
+#endif
+ INIT_LIST_HEAD(&snic_glob->snic_list);
+ spin_lock_init(&snic_glob->snic_list_lock);
+
+ /* Create a cache for allocation of snic_host_req+default size ESGLs */
+ len = sizeof(struct snic_req_info);
+ len += sizeof(struct snic_host_req) + sizeof(struct snic_dflt_sgl);
+ cachep = kmem_cache_create("snic_req_dfltsgl", len, SNIC_SG_DESC_ALIGN,
+ SLAB_HWCACHE_ALIGN, NULL);
+ if (!cachep) {
+ SNIC_ERR("Failed to create snic default sgl slab\n");
+ ret = -ENOMEM;
+
+ goto err_dflt_req_slab;
+ }
+ snic_glob->req_cache[SNIC_REQ_CACHE_DFLT_SGL] = cachep;
+
+ /* Create a cache for allocation of max size Extended SGLs */
+ len = sizeof(struct snic_req_info);
+ len += sizeof(struct snic_host_req) + sizeof(struct snic_max_sgl);
+ cachep = kmem_cache_create("snic_req_maxsgl", len, SNIC_SG_DESC_ALIGN,
+ SLAB_HWCACHE_ALIGN, NULL);
+ if (!cachep) {
+ SNIC_ERR("Failed to create snic max sgl slab\n");
+ ret = -ENOMEM;
+
+ goto err_max_req_slab;
+ }
+ snic_glob->req_cache[SNIC_REQ_CACHE_MAX_SGL] = cachep;
+
+ len = sizeof(struct snic_host_req);
+ cachep = kmem_cache_create("snic_req_maxsgl", len, SNIC_SG_DESC_ALIGN,
+ SLAB_HWCACHE_ALIGN, NULL);
+ if (!cachep) {
+ SNIC_ERR("Failed to create snic tm req slab\n");
+ ret = -ENOMEM;
+
+ goto err_tmreq_slab;
+ }
+ snic_glob->req_cache[SNIC_REQ_TM_CACHE] = cachep;
+
+ /* snic_event queue */
+ snic_glob->event_q = create_singlethread_workqueue("snic_event_wq");
+ if (!snic_glob->event_q) {
+ SNIC_ERR("snic event queue create failed\n");
+ ret = -ENOMEM;
+
+ goto err_eventq;
+ }
+
+ return ret;
+
+err_eventq:
+ kmem_cache_destroy(snic_glob->req_cache[SNIC_REQ_TM_CACHE]);
+
+err_tmreq_slab:
+ kmem_cache_destroy(snic_glob->req_cache[SNIC_REQ_CACHE_MAX_SGL]);
+
+err_max_req_slab:
+ kmem_cache_destroy(snic_glob->req_cache[SNIC_REQ_CACHE_DFLT_SGL]);
+
+err_dflt_req_slab:
+#ifdef SNIC_DEBUG_FS
+ snic_trc_free();
+ snic_debugfs_term();
+#endif
+ kfree(snic_glob);
+ snic_glob = NULL;
+
+gdi_end:
+ return ret;
+} /* end of snic_glob_init */
+
+/*
+ * snic_global_data_cleanup : Frees SNIC Global Data
+ */
+static void
+snic_global_data_cleanup(void)
+{
+ SNIC_BUG_ON(snic_glob == NULL);
+
+ destroy_workqueue(snic_glob->event_q);
+ kmem_cache_destroy(snic_glob->req_cache[SNIC_REQ_TM_CACHE]);
+ kmem_cache_destroy(snic_glob->req_cache[SNIC_REQ_CACHE_MAX_SGL]);
+ kmem_cache_destroy(snic_glob->req_cache[SNIC_REQ_CACHE_DFLT_SGL]);
+
+#ifdef SNIC_DEBUG_FS
+ /* Freeing Trace Resources */
+ snic_trc_free();
+
+ /* Freeing Debugfs Resources */
+ snic_debugfs_term();
+#endif
+ kfree(snic_glob);
+ snic_glob = NULL;
+} /* end of snic_glob_cleanup */
+
+static struct pci_driver snic_driver = {
+ .name = SNIC_DRV_NAME,
+ .id_table = snic_id_table,
+ .probe = snic_probe,
+ .remove = snic_remove,
+};
+
+static int __init
+snic_init_module(void)
+{
+ int ret = 0;
+
+ SNIC_INFO("%s, ver %s\n", SNIC_DRV_DESCRIPTION, SNIC_DRV_VERSION);
+
+ ret = snic_global_data_init();
+ if (ret) {
+ SNIC_ERR("Failed to Initialize Global Data.\n");
+
+ return ret;
+ }
+
+ ret = pci_register_driver(&snic_driver);
+ if (ret < 0) {
+ SNIC_ERR("PCI driver register error\n");
+
+ goto err_pci_reg;
+ }
+
+ return ret;
+
+err_pci_reg:
+ snic_global_data_cleanup();
+
+ return ret;
+}
+
+static void __exit
+snic_cleanup_module(void)
+{
+ pci_unregister_driver(&snic_driver);
+ snic_global_data_cleanup();
+}
+
+module_init(snic_init_module);
+module_exit(snic_cleanup_module);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION(SNIC_DRV_DESCRIPTION);
+MODULE_VERSION(SNIC_DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, snic_id_table);
+MODULE_AUTHOR("Narsimhulu Musini <nmusini@cisco.com>, "
+ "Sesidhar Baddela <sebaddel@cisco.com>");
--
1.8.5.4
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v4 2/9] snic:Add interrupt, resource firmware interfaces
2015-04-09 11:49 [PATCH v4 0/9] snic:initial submission of snic driver for Cisco SCSI HBA Narsimhulu Musini
2015-04-09 11:49 ` [PATCH v4 1/9] snic: snic module infrastructure Narsimhulu Musini
@ 2015-04-09 11:49 ` Narsimhulu Musini
2015-04-09 12:46 ` Hannes Reinecke
2015-04-09 11:49 ` [PATCH v4 3/9] snic:Add meta request, handling of meta requests Narsimhulu Musini
` (6 subsequent siblings)
8 siblings, 1 reply; 24+ messages in thread
From: Narsimhulu Musini @ 2015-04-09 11:49 UTC (permalink / raw)
To: JBottomley, linux-scsi, hare, hch; +Cc: Narsimhulu Musini, Sesidhar Baddela
snic_fwint.h contains firmware interface structures.
snic_res.h contains firmware request initialization
snic_res.c contains retrieval of resource configuration, and allocation,
and initialization of HW Queues.
snic_isr.c contains interrupt request, release, and handling
Signed-off-by: Narsimhulu Musini <nmusini@cisco.com>
Signed-off-by: Sesidhar Baddela <sebaddel@cisco.com>
---
* v4
- Updated comment on future changes when hardware supports multiple queues.
* v3
- Cleaned up unused structure definitions and functions.
drivers/scsi/snic/snic_fwint.h | 525 +++++++++++++++++++++++++++++++++++++++++
drivers/scsi/snic/snic_isr.c | 209 ++++++++++++++++
drivers/scsi/snic/snic_res.c | 297 +++++++++++++++++++++++
drivers/scsi/snic/snic_res.h | 96 ++++++++
4 files changed, 1127 insertions(+)
create mode 100644 drivers/scsi/snic/snic_fwint.h
create mode 100644 drivers/scsi/snic/snic_isr.c
create mode 100644 drivers/scsi/snic/snic_res.c
create mode 100644 drivers/scsi/snic/snic_res.h
diff --git a/drivers/scsi/snic/snic_fwint.h b/drivers/scsi/snic/snic_fwint.h
new file mode 100644
index 0000000..1142e52
--- /dev/null
+++ b/drivers/scsi/snic/snic_fwint.h
@@ -0,0 +1,525 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __SNIC_FWINT_H
+#define __SNIC_FWINT_H
+
+#define SNIC_CDB_LEN 32 /* SCSI CDB size 32, can be used for 16 bytes */
+#define LUN_ADDR_LEN 8
+
+/*
+ * Command entry type
+ */
+enum snic_io_type {
+ /*
+ * Initiator request types
+ */
+ SNIC_REQ_REPORT_TGTS = 0x2, /* Report Targets */
+ SNIC_REQ_ICMND, /* Initiator command for SCSI IO */
+ SNIC_REQ_ITMF, /* Initiator command for Task Mgmt */
+ SNIC_REQ_HBA_RESET, /* SNIC Reset */
+ SNIC_REQ_EXCH_VER, /* Exchange Version Information */
+ SNIC_REQ_TGT_INFO, /* Backend/Target Information */
+ SNIC_REQ_BOOT_LUNS,
+
+ /*
+ * Response type
+ */
+ SNIC_RSP_REPORT_TGTS_CMPL = 0x12,/* Report Targets Completion */
+ SNIC_RSP_ICMND_CMPL, /* SCSI IO Completion */
+ SNIC_RSP_ITMF_CMPL, /* Task Management Completion */
+ SNIC_RSP_HBA_RESET_CMPL, /* SNIC Reset Completion */
+ SNIC_RSP_EXCH_VER_CMPL, /* Exchange Version Completion*/
+ SNIC_RSP_BOOT_LUNS_CMPL,
+
+ /*
+ * Misc Request types
+ */
+ SNIC_MSG_ACK = 0x80, /* Ack: snic_notify_msg */
+ SNIC_MSG_ASYNC_EVNOTIFY, /* Asynchronous Event Notification */
+}; /* end of enum snic_io_type */
+
+
+/*
+ * Header status codes from firmware
+ */
+enum snic_io_status {
+ SNIC_STAT_IO_SUCCESS = 0, /* request was successful */
+
+ /*
+ * If a request to the fw is rejected, the original request header
+ * will be returned with the status set to one of the following:
+ */
+ SNIC_STAT_INVALID_HDR, /* header contains invalid data */
+ SNIC_STAT_OUT_OF_RES, /* out of resources to complete request */
+ SNIC_STAT_INVALID_PARM, /* some parameter in request is not valid */
+ SNIC_STAT_REQ_NOT_SUP, /* req type is not supported */
+ SNIC_STAT_IO_NOT_FOUND, /* requested IO was not found */
+
+ /*
+ * Once a request is processed, the fw will usually return
+ * a cmpl message type. In cases where errors occurred,
+ * the header status would be filled in with one of the following:
+ */
+ SNIC_STAT_ABORTED, /* req was aborted */
+ SNIC_STAT_TIMEOUT, /* req was timed out */
+ SNIC_STAT_SGL_INVALID, /* req was aborted due to sgl error */
+ SNIC_STAT_DATA_CNT_MISMATCH, /*recv/sent more/less data than expec */
+ SNIC_STAT_FW_ERR, /* req was terminated due to fw error */
+ SNIC_STAT_ITMF_REJECT, /* itmf req was rejected by target */
+ SNIC_STAT_ITMF_FAIL, /* itmf req was failed */
+ SNIC_STAT_ITMF_INCORRECT_LUN, /* itmf req has incorrect LUN id*/
+ SNIC_STAT_CMND_REJECT, /* req was invalid and rejected */
+ SNIC_STAT_DEV_OFFLINE, /* req sent to offline device */
+ SNIC_STAT_NO_BOOTLUN,
+ SNIC_STAT_SCSI_ERR, /* SCSI error returned by Target. */
+ SNIC_STAT_NOT_READY, /* sNIC Subsystem is not ready */
+ SNIC_STAT_FATAL_ERROR, /* sNIC is in unrecoverable state */
+}; /* end of enum snic_io_status */
+
+/*
+ * snic_io_hdr : host <--> firmare
+ *
+ * for any other message that will be queued to firmware should
+ * have the following request header
+ */
+struct snic_io_hdr {
+ u32 hid;
+ u32 cmnd_id; /* tag here */
+ u64 init_ctx; /* initiator context */
+ u8 type; /* request/response type */
+ u8 status; /* header status entry */
+ u8 protocol; /* Protocol specific, may needed for RoCE*/
+ u8 flags;
+ u16 sg_cnt;
+ u16 resvd;
+};
+
+/* auxillary funciton for encoding the snic_io_hdr */
+static inline void
+snic_io_hdr_enc(struct snic_io_hdr *hdr, u8 typ, u8 status, u32 id, u32 hid,
+ u16 sg_cnt, u64 ctx)
+{
+ hdr->type = typ;
+ hdr->status = status;
+ hdr->protocol = 0;
+ hdr->hid = hid;
+ hdr->cmnd_id = id;
+ hdr->sg_cnt = sg_cnt;
+ hdr->init_ctx = ctx;
+ hdr->flags = 0;
+}
+
+/* auxillary funciton for decoding the snic_io_hdr */
+static inline void
+snic_io_hdr_dec(struct snic_io_hdr *hdr, u8 *typ, u8 *stat, u32 *cmnd_id,
+ u32 *hid, u64 *ctx)
+{
+ *typ = hdr->type;
+ *stat = hdr->status;
+ *hid = hdr->hid;
+ *cmnd_id = hdr->cmnd_id;
+ *ctx = hdr->init_ctx;
+}
+
+/*
+ * snic_host_info: host -> firmware
+ *
+ * Used for sending host information to firmware, and request fw version
+ */
+struct snic_exch_ver_req {
+ u32 drvr_ver; /* for debugging, when fw dump captured */
+ u32 os_type; /* for OS specific features */
+};
+
+/*
+ * os_type flags
+ * Bit 0-7 : OS information
+ * Bit 8-31: Feature/Capability Information
+ */
+#define SNIC_OS_LINUX 0x1
+#define SNIC_OS_WIN 0x2
+#define SNIC_OS_ESX 0x3
+
+/*
+ * HBA Capabilities
+ * Bit 1: Reserved.
+ * Bit 2: Dynamic Discovery of LUNs.
+ * Bit 3: Async event notifications on on tgt online/offline events.
+ * Bit 4: IO timeout support in FW.
+ * Bit 5-31: Reserved.
+ */
+#define SNIC_HBA_CAP_DDL 0x02 /* Supports Dynamic Discovery of LUNs */
+#define SNIC_HBA_CAP_AEN 0x04 /* Supports Async Event Noitifcation */
+#define SNIC_HBA_CAP_TMO 0x08 /* Supports IO timeout in FW */
+
+/*
+ * snic_exch_ver_rsp : firmware -> host
+ *
+ * Used by firmware to send response to version request
+ */
+struct snic_exch_ver_rsp {
+ u32 version;
+ u32 hid;
+ u32 max_concur_ios; /* max concurrent ios */
+ u32 max_sgs_per_cmd; /* max sgls per IO */
+ u32 max_io_sz; /* max io size supported */
+ u32 hba_cap; /* hba capabilities */
+ u32 max_tgts; /* max tgts supported */
+ u16 io_timeout; /* FW extended timeout */
+ u16 rsvd;
+};
+
+
+/*
+ * snic_report_tgts : host -> firmware request
+ *
+ * Used by the host to request list of targets
+ */
+struct snic_report_tgts {
+ u16 sg_cnt;
+ u16 flags; /* specific flags from fw */
+ u8 _resvd[4];
+ u64 sg_addr; /* Points to SGL */
+ u64 sense_addr;
+};
+
+enum snic_type {
+ SNIC_NONE = 0x0,
+ SNIC_DAS,
+ SNIC_SAN,
+};
+
+
+/* Report Target Response */
+enum snic_tgt_type {
+ SNIC_TGT_NONE = 0x0,
+ SNIC_TGT_DAS, /* DAS Target */
+ SNIC_TGT_SAN, /* SAN Target */
+};
+
+/* target id format */
+struct snic_tgt_id {
+ u32 tgt_id; /* target id */
+ u16 tgt_type; /* tgt type */
+ u16 vnic_id; /* corresponding vnic id */
+};
+
+/*
+ * snic_report_tgts_cmpl : firmware -> host response
+ *
+ * Used by firmware to send response to Report Targets request
+ */
+struct snic_report_tgts_cmpl {
+ u32 tgt_cnt; /* Number of Targets accessible */
+ u32 _resvd;
+};
+
+/*
+ * Command flags
+ *
+ * Bit 0: Read flags
+ * Bit 1: Write flag
+ * Bit 2: ESGL - sg/esg array contains extended sg
+ * ESGE - is a host buffer contains sg elements
+ * Bit 3-4: Task Attributes
+ * 00b - simple
+ * 01b - head of queue
+ * 10b - ordered
+ * Bit 5-7: Priority - future use
+ * Bit 8-15: Reserved
+ */
+
+#define SNIC_ICMND_WR 0x01 /* write command */
+#define SNIC_ICMND_RD 0x02 /* read command */
+#define SNIC_ICMND_ESGL 0x04 /* SGE/ESGE array contains valid data*/
+
+/*
+ * Priority/Task Attribute settings
+ */
+#define SNIC_ICMND_TSK_SHIFT 2 /* task attr starts at bit 2 */
+#define SNIC_ICMND_TSK_MASK(x) ((x>>SNIC_ICMND_TSK_SHIFT) & ~(0xffff))
+#define SNIC_ICMND_TSK_SIMPLE 0 /* simple task attr */
+#define SNIC_ICMND_TSK_HEAD_OF_QUEUE 1 /* head of qeuue task attr */
+#define SNIC_ICMND_TSK_ORDERED 2 /* ordered task attr */
+
+#define SNIC_ICMND_PRI_SHIFT 5 /* prio val starts at bit 5 */
+
+/*
+ * snic_icmnd : host-> firmware request
+ *
+ * used for sending out an initiator SCSI 16/32-byte command
+ */
+struct snic_icmnd {
+ u16 sg_cnt; /* Number of SG Elements */
+ u16 flags; /* flags */
+ u32 sense_len; /* Sense buffer length */
+ u64 tgt_id; /* Destination Target ID */
+ u64 lun_id; /* Destination LUN ID */
+ u8 cdb_len;
+ u8 _resvd;
+ u16 time_out; /* ms time for Res allocations fw to handle io*/
+ u32 data_len; /* Total number of bytes to be transferred */
+ u8 cdb[SNIC_CDB_LEN];
+ u64 sg_addr; /* Points to SG List */
+ u64 sense_addr; /* Sense buffer address */
+};
+
+
+/* Response flags */
+/* Bit 0: Under run
+ * Bit 1: Over Run
+ * Bit 2-7: Reserved
+ */
+#define SNIC_ICMND_CMPL_UNDR_RUN 0x01 /* resid under and valid */
+#define SNIC_ICMND_CMPL_OVER_RUN 0x02 /* resid over and valid */
+
+/*
+ * snic_icmnd_cmpl: firmware -> host response
+ *
+ * Used for sending the host a response to an icmnd (initiator command)
+ */
+struct snic_icmnd_cmpl {
+ u8 scsi_status; /* value as per SAM */
+ u8 flags;
+ u16 sense_len; /* Sense Length */
+ u32 resid; /* Residue : # bytes under or over run */
+};
+
+/*
+ * snic_itmf: host->firmware request
+ *
+ * used for requesting the firmware to abort a request and/or send out
+ * a task management function
+ *
+ * the req_id field is valid in case of abort task and clear task
+ */
+struct snic_itmf {
+ u8 tm_type; /* SCSI Task Management request */
+ u8 resvd;
+ u16 flags; /* flags */
+ u32 req_id; /* Command id of snic req to be aborted */
+ u64 tgt_id; /* Target ID */
+ u64 lun_id; /* Destination LUN ID */
+ u16 timeout; /* in sec */
+};
+
+/*
+ * Task Management Request
+ */
+enum snic_itmf_tm_type {
+ SNIC_ITMF_ABTS_TASK = 0x01, /* Abort Task */
+ SNIC_ITMF_ABTS_TASK_SET, /* Abort Task Set */
+ SNIC_ITMF_CLR_TASK, /* Clear Task */
+ SNIC_ITMF_CLR_TASKSET, /* Clear Task Set */
+ SNIC_ITMF_LUN_RESET, /* Lun Reset */
+ SNIC_ITMF_ABTS_TASK_TERM, /* Supported for SAN Targets */
+};
+
+/*
+ * snic_itmf_cmpl: firmware -> host resposne
+ *
+ * used for sending the host a response for a itmf request
+ */
+struct snic_itmf_cmpl {
+ u32 nterminated; /* # IOs terminated as a result of tmf */
+ u8 flags; /* flags */
+ u8 _resvd[3];
+};
+
+/*
+ * itmfl_cmpl flags
+ * Bit 0 : 1 - Num terminated field valid
+ * Bit 1 - 7 : Reserved
+ */
+#define SNIC_NUM_TERM_VALID 0x01 /* Number of IOs terminated */
+
+/*
+ * snic_hba_reset: host -> firmware request
+ *
+ * used for requesting firmware to reset snic
+ */
+struct snic_hba_reset {
+ u16 flags; /* flags */
+ u8 _resvd[6];
+};
+
+/*
+ * snic_hba_reset_cmpl: firmware -> host response
+ *
+ * Used by firmware to respond to the host's hba reset request
+ */
+struct snic_hba_reset_cmpl {
+ u8 flags; /* flags : more info needs to be added*/
+ u8 _resvd[7];
+};
+
+/*
+ * snic_notify_msg: firmware -> host response
+ *
+ * Used by firmware to notify host of the last work queue entry received
+ */
+struct snic_notify_msg {
+ u32 wqe_num; /* wq entry number */
+ u8 flags; /* flags, macros */
+ u8 _resvd[4];
+};
+
+
+#define SNIC_EVDATA_LEN 24 /* in bytes */
+/* snic_async_evnotify: firmware -> host notification
+ *
+ * Used by firmware to notify the host about configuration/state changes
+ */
+struct snic_async_evnotify {
+ u8 FLS_EVENT_DESC; /* TODO: ?? */
+ u8 vnic; /* vnic id */
+ u8 _resvd[2];
+ u32 ev_id; /* Event ID */
+ u8 ev_data[SNIC_EVDATA_LEN]; /* Event Data */
+ u8 _resvd2[4];
+};
+
+/* async event flags */
+enum snic_ev_type {
+ SNIC_EV_TGT_OFFLINE = 0x01, /* Target Offline, PL contains TGT ID */
+ SNIC_EV_TGT_ONLINE, /* Target Online, PL contains TGT ID */
+ SNIC_EV_LUN_OFFLINE, /* LUN Offline, PL contains LUN ID */
+ SNIC_EV_LUN_ONLINE, /* LUN Online, PL contains LUN ID */
+ SNIC_EV_CONF_CHG, /* Dev Config/Attr Change Event */
+ SNIC_EV_TGT_ADDED, /* TODO:Target Added, PL contains ?? */
+ SNIC_EV_TGT_DELTD, /* Target Del'd, PL contains TGT ID */
+ SNIC_EV_LUN_ADDED, /* TODO:LUN Added, PL contains ?? */
+ SNIC_EV_LUN_DELTD, /* LUN Del'd, PL cont. TGT & LUN ID */
+
+ SNIC_EV_DISC_CMPL = 0x10, /* Discovery Completed Event */
+};
+
+
+#define SNIC_HOST_REQ_LEN 128 /*Exp length of host req, wq desc sz*/
+/* Payload 88 bytes = 128 - 24 - 16 */
+#define SNIC_HOST_REQ_PAYLOAD ((int)(SNIC_HOST_REQ_LEN - \
+ sizeof(struct snic_io_hdr) - \
+ (2 * sizeof(u64))))
+
+/*
+ * snic_host_req: host -> firmware request
+ *
+ * Basic structure for all snic requests that are sent from the host to
+ * firmware. They are 128 bytes in size.
+ */
+struct snic_host_req {
+ u64 ctrl_data[2]; /*16 bytes - Control Data */
+ struct snic_io_hdr hdr;
+ union {
+ /*
+ * Entry specific space, last byte contains color
+ */
+ u8 buf[SNIC_HOST_REQ_PAYLOAD];
+
+ /*
+ * Exchange firmware version
+ */
+ struct snic_exch_ver_req exch_ver;
+
+ /* report targets */
+ struct snic_report_tgts rpt_tgts;
+
+ /* io request */
+ struct snic_icmnd icmnd;
+
+ /* task management request */
+ struct snic_itmf itmf;
+
+ /* hba reset */
+ struct snic_hba_reset reset;
+ } u;
+}; /* end of snic_host_req structure */
+
+
+#define SNIC_FW_REQ_LEN 64 /* Expected length of fw req */
+struct snic_fw_req {
+ struct snic_io_hdr hdr;
+ union {
+ /*
+ * Entry specific space, last byte contains color
+ */
+ u8 buf[SNIC_FW_REQ_LEN - sizeof(struct snic_io_hdr)];
+
+ /* Exchange Version Response */
+ struct snic_exch_ver_rsp exch_ver_cmpl;
+
+ /* Report Targets Response */
+ struct snic_report_tgts_cmpl rpt_tgts_cmpl;
+
+ /* scsi response */
+ struct snic_icmnd_cmpl icmnd_cmpl;
+
+ /* task management response */
+ struct snic_itmf_cmpl itmf_cmpl;
+
+ /* hba reset response */
+ struct snic_hba_reset_cmpl reset_cmpl;
+
+ /* notify message */
+ struct snic_notify_msg ack;
+
+ /* async notification event */
+ struct snic_async_evnotify async_ev;
+
+ } u;
+}; /* end of snic_fw_req structure */
+
+/*
+ * Auxillary macro to verify specific snic req/cmpl structures
+ * to ensure that it will be aligned to 64 bit, and not using
+ * color bit field
+ */
+#define VERIFY_REQ_SZ(x)
+#define VERIFY_CMPL_SZ(x)
+
+/*
+ * Access routines to encode and decode the color bit, which is the most
+ * significant bit of the structure.
+ */
+static inline void
+snic_color_enc(struct snic_fw_req *req, u8 color)
+{
+ u8 *c = ((u8 *) req) + sizeof(struct snic_fw_req) - 1;
+
+ if (color)
+ *c |= 0x80;
+ else
+ *c &= ~0x80;
+}
+
+static inline void
+snic_color_dec(struct snic_fw_req *req, u8 *color)
+{
+ u8 *c = ((u8 *) req) + sizeof(struct snic_fw_req) - 1;
+
+ *color = *c >> 7;
+
+ /* Make sure color bit is read from desc *before* other fields
+ * are read from desc. Hardware guarantees color bit is last
+ * bit (byte) written. Adding the rmb() prevents the compiler
+ * and/or CPU from reordering the reads which would potentially
+ * result in reading stale values.
+ */
+ rmb();
+}
+#endif /* end of __SNIC_FWINT_H */
diff --git a/drivers/scsi/snic/snic_isr.c b/drivers/scsi/snic/snic_isr.c
new file mode 100644
index 0000000..5d5cb0c
--- /dev/null
+++ b/drivers/scsi/snic/snic_isr.c
@@ -0,0 +1,209 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ * [Insert appropriate license here when releasing outside of Cisco]
+ *
+ */
+
+#include <linux/string.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/interrupt.h>
+
+#include "vnic_dev.h"
+#include "vnic_intr.h"
+#include "vnic_stats.h"
+#include "snic_io.h"
+#include "snic.h"
+
+
+/*
+ * snic_isr_msix_wq : MSIx ISR for work queue.
+ */
+
+static irqreturn_t
+snic_isr_msix_wq(int irq, void *data)
+{
+ struct snic *snic = data;
+ unsigned long wq_work_done = 0;
+
+ snic->s_stats.misc.last_isr_time = jiffies;
+ atomic64_inc(&snic->s_stats.misc.isr_cnt);
+
+ wq_work_done = snic_wq_cmpl_handler(snic, -1);
+ vnic_intr_return_credits(&snic->intr[SNIC_MSIX_WQ],
+ wq_work_done,
+ 1 /* unmask intr */,
+ 1 /* reset intr timer */);
+
+ return IRQ_HANDLED;
+} /* end of snic_isr_msix_wq */
+
+static irqreturn_t
+snic_isr_msix_io_cmpl(int irq, void *data)
+{
+ struct snic *snic = data;
+ unsigned long iocmpl_work_done = 0;
+
+ snic->s_stats.misc.last_isr_time = jiffies;
+ atomic64_inc(&snic->s_stats.misc.isr_cnt);
+
+ iocmpl_work_done = snic_fwcq_cmpl_handler(snic, -1);
+ vnic_intr_return_credits(&snic->intr[SNIC_MSIX_IO_CMPL],
+ iocmpl_work_done,
+ 1 /* unmask intr */,
+ 1 /* reset intr timer */);
+
+ return IRQ_HANDLED;
+} /* end of snic_isr_msix_io_cmpl */
+
+static irqreturn_t
+snic_isr_msix_err_notify(int irq, void *data)
+{
+ struct snic *snic = data;
+
+ snic->s_stats.misc.last_isr_time = jiffies;
+ atomic64_inc(&snic->s_stats.misc.isr_cnt);
+
+ vnic_intr_return_all_credits(&snic->intr[SNIC_MSIX_ERR_NOTIFY]);
+ snic_log_q_error(snic);
+
+ /*Handling link events */
+ snic_handle_link_event(snic);
+
+ return IRQ_HANDLED;
+} /* end of snic_isr_msix_err_notify */
+
+
+void
+snic_free_intr(struct snic *snic)
+{
+ int i;
+
+ /* ONLY interrupt mode MSIX is supported */
+ for (i = 0; i < ARRAY_SIZE(snic->msix); i++) {
+ if (snic->msix[i].requested) {
+ free_irq(snic->msix_entry[i].vector,
+ snic->msix[i].devid);
+ }
+ }
+} /* end of snic_free_intr */
+
+int
+snic_request_intr(struct snic *snic)
+{
+ int ret = 0, i;
+
+#ifdef SNIC_DEBUG
+ enum vnic_dev_intr_mode intr_mode;
+
+ intr_mode = vnic_dev_get_intr_mode(snic->vdev);
+ SNIC_BUG_ON(intr_mode != VNIC_DEV_INTR_MODE_MSIX);
+#endif
+
+ /*
+ * Currently HW supports single WQ and CQ. So passing devid as snic.
+ * When hardware supports multiple WQs and CQs, one idea is
+ * to pass devid as corresponding WQ or CQ ptr and retrieve snic
+ * from queue ptr.
+ * Except for err_notify, which is always one.
+ */
+ sprintf(snic->msix[SNIC_MSIX_WQ].devname,
+ "%.11s-scsi-wq",
+ snic->name);
+ snic->msix[SNIC_MSIX_WQ].isr = snic_isr_msix_wq;
+ snic->msix[SNIC_MSIX_WQ].devid = snic;
+
+ sprintf(snic->msix[SNIC_MSIX_IO_CMPL].devname,
+ "%.11s-io-cmpl",
+ snic->name);
+ snic->msix[SNIC_MSIX_IO_CMPL].isr = snic_isr_msix_io_cmpl;
+ snic->msix[SNIC_MSIX_IO_CMPL].devid = snic;
+
+ sprintf(snic->msix[SNIC_MSIX_ERR_NOTIFY].devname,
+ "%.11s-err-notify",
+ snic->name);
+ snic->msix[SNIC_MSIX_ERR_NOTIFY].isr = snic_isr_msix_err_notify;
+ snic->msix[SNIC_MSIX_ERR_NOTIFY].devid = snic;
+
+ for (i = 0; i < ARRAY_SIZE(snic->msix); i++) {
+ ret = request_irq(snic->msix_entry[i].vector,
+ snic->msix[i].isr,
+ 0,
+ snic->msix[i].devname,
+ snic->msix[i].devid);
+ if (ret) {
+ SNIC_HOST_ERR(snic->shost,
+ "MSI-X: requrest_irq(%d) failed %d\n",
+ i,
+ ret);
+ snic_free_intr(snic);
+ break;
+ }
+ snic->msix[i].requested = 1;
+ }
+
+ return ret;
+} /* end of snic_requrest_intr */
+
+int
+snic_set_intr_mode(struct snic *snic)
+{
+ unsigned int n = ARRAY_SIZE(snic->wq);
+ unsigned int m = SNIC_CQ_IO_CMPL_MAX;
+ unsigned int i;
+
+ /*
+ * We need n WQs, m CQs, and n+m+1 INTRs
+ * (last INTR is used for WQ/CQ errors and notification area
+ */
+
+ BUILD_BUG_ON((ARRAY_SIZE(snic->wq) + SNIC_CQ_IO_CMPL_MAX) >
+ ARRAY_SIZE(snic->intr));
+ SNIC_BUG_ON(ARRAY_SIZE(snic->msix_entry) < (n + m + 1));
+
+ for (i = 0; i < (n + m + 1); i++)
+ snic->msix_entry[i].entry = i;
+
+ if (snic->wq_count >= n && snic->cq_count >= (n + m)) {
+ if (!pci_enable_msix(snic->pdev,
+ snic->msix_entry,
+ (n + m + 1))) {
+ snic->wq_count = n;
+ snic->cq_count = n + m;
+ snic->intr_count = n + m + 1;
+ snic->err_intr_offset = SNIC_MSIX_ERR_NOTIFY;
+
+ SNIC_ISR_DBG(snic->shost,
+ "Using MSI-X Interrupts\n");
+ vnic_dev_set_intr_mode(snic->vdev,
+ VNIC_DEV_INTR_MODE_MSIX);
+
+ return 0;
+ }
+ }
+
+ vnic_dev_set_intr_mode(snic->vdev, VNIC_DEV_INTR_MODE_UNKNOWN);
+
+ return -EINVAL;
+} /* end of snic_set_intr_mode */
+
+void
+snic_clear_intr_mode(struct snic *snic)
+{
+ pci_disable_msix(snic->pdev);
+
+ vnic_dev_set_intr_mode(snic->vdev, VNIC_DEV_INTR_MODE_INTX);
+}
diff --git a/drivers/scsi/snic/snic_res.c b/drivers/scsi/snic/snic_res.c
new file mode 100644
index 0000000..4b2e4b1
--- /dev/null
+++ b/drivers/scsi/snic/snic_res.c
@@ -0,0 +1,297 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/errno.h>
+#include <linux/types.h>
+#include <linux/pci.h>
+
+#include "wq_enet_desc.h"
+#include "cq_enet_desc.h"
+#include "vnic_resource.h"
+#include "vnic_dev.h"
+#include "vnic_wq.h"
+#include "vnic_cq.h"
+#include "vnic_intr.h"
+#include "vnic_stats.h"
+#include "snic.h"
+
+int
+snic_get_vnic_config(struct snic *snic)
+{
+ struct vnic_snic_config *c = &snic->config;
+ int ret;
+
+#define GET_CONFIG(m) \
+ do { \
+ ret = vnic_dev_spec(snic->vdev, \
+ offsetof(struct vnic_snic_config, m), \
+ sizeof(c->m), \
+ &c->m); \
+ if (ret) { \
+ SNIC_HOST_ERR(snic->shost, \
+ "Error getting %s, %d\n", #m, ret); \
+ return ret; \
+ } \
+ } while (0)
+
+ GET_CONFIG(wq_enet_desc_count);
+ GET_CONFIG(maxdatafieldsize);
+ GET_CONFIG(intr_timer);
+ GET_CONFIG(intr_timer_type);
+ GET_CONFIG(flags);
+ GET_CONFIG(io_throttle_count);
+ GET_CONFIG(port_down_timeout);
+ GET_CONFIG(port_down_io_retries);
+ GET_CONFIG(luns_per_tgt);
+ GET_CONFIG(xpt_type);
+ GET_CONFIG(hid);
+
+ c->wq_enet_desc_count = min_t(u32,
+ VNIC_SNIC_WQ_DESCS_MAX,
+ max_t(u32,
+ VNIC_SNIC_WQ_DESCS_MIN,
+ c->wq_enet_desc_count));
+
+ c->wq_enet_desc_count = ALIGN(c->wq_enet_desc_count, 16);
+
+ c->maxdatafieldsize = min_t(u32,
+ VNIC_SNIC_MAXDATAFIELDSIZE_MAX,
+ max_t(u32,
+ VNIC_SNIC_MAXDATAFIELDSIZE_MIN,
+ c->maxdatafieldsize));
+
+ c->io_throttle_count = min_t(u32,
+ VNIC_SNIC_IO_THROTTLE_COUNT_MAX,
+ max_t(u32,
+ VNIC_SNIC_IO_THROTTLE_COUNT_MIN,
+ c->io_throttle_count));
+
+ c->port_down_timeout = min_t(u32,
+ VNIC_SNIC_PORT_DOWN_TIMEOUT_MAX,
+ c->port_down_timeout);
+
+ c->port_down_io_retries = min_t(u32,
+ VNIC_SNIC_PORT_DOWN_IO_RETRIES_MAX,
+ c->port_down_io_retries);
+
+ c->luns_per_tgt = min_t(u32,
+ VNIC_SNIC_LUNS_PER_TARGET_MAX,
+ max_t(u32,
+ VNIC_SNIC_LUNS_PER_TARGET_MIN,
+ c->luns_per_tgt));
+
+ c->intr_timer = min_t(u32, VNIC_INTR_TIMER_MAX, c->intr_timer);
+
+ SNIC_INFO("vNIC resources wq %d\n", c->wq_enet_desc_count);
+ SNIC_INFO("vNIC mtu %d intr timer %d\n",
+ c->maxdatafieldsize,
+ c->intr_timer);
+
+ SNIC_INFO("vNIC flags 0x%x luns per tgt %d\n",
+ c->flags,
+ c->luns_per_tgt);
+
+ SNIC_INFO("vNIC io throttle count %d\n", c->io_throttle_count);
+ SNIC_INFO("vNIC port down timeout %d port down io retries %d\n",
+ c->port_down_timeout,
+ c->port_down_io_retries);
+
+ SNIC_INFO("vNIC back end type = %d\n", c->xpt_type);
+ SNIC_INFO("vNIC hid = %d\n", c->hid);
+
+ return 0;
+}
+
+void
+snic_get_res_counts(struct snic *snic)
+{
+ snic->wq_count = vnic_dev_get_res_count(snic->vdev, RES_TYPE_WQ);
+ SNIC_BUG_ON(snic->wq_count == 0);
+ snic->cq_count = vnic_dev_get_res_count(snic->vdev, RES_TYPE_CQ);
+ SNIC_BUG_ON(snic->cq_count == 0);
+ snic->intr_count = vnic_dev_get_res_count(snic->vdev,
+ RES_TYPE_INTR_CTRL);
+ SNIC_BUG_ON(snic->intr_count == 0);
+}
+
+void
+snic_free_vnic_res(struct snic *snic)
+{
+ unsigned int i;
+
+ for (i = 0; i < snic->wq_count; i++)
+ vnic_wq_free(&snic->wq[i]);
+
+ for (i = 0; i < snic->cq_count; i++)
+ vnic_cq_free(&snic->cq[i]);
+
+ for (i = 0; i < snic->intr_count; i++)
+ vnic_intr_free(&snic->intr[i]);
+}
+
+int
+snic_alloc_vnic_res(struct snic *snic)
+{
+ enum vnic_dev_intr_mode intr_mode;
+ unsigned int mask_on_assertion;
+ unsigned int intr_offset;
+ unsigned int err_intr_enable;
+ unsigned int err_intr_offset;
+ unsigned int i;
+ int ret;
+
+ intr_mode = vnic_dev_get_intr_mode(snic->vdev);
+
+ SNIC_INFO("vNIC interrupt mode: %s\n",
+ ((intr_mode == VNIC_DEV_INTR_MODE_INTX) ?
+ "Legacy PCI INTx" :
+ ((intr_mode == VNIC_DEV_INTR_MODE_MSI) ?
+ "MSI" :
+ ((intr_mode == VNIC_DEV_INTR_MODE_MSIX) ?
+ "MSI-X" : "Unknown"))));
+
+ /* only MSI-X is supported */
+ SNIC_BUG_ON(intr_mode != VNIC_DEV_INTR_MODE_MSIX);
+
+ SNIC_INFO("wq %d cq %d intr %d\n", snic->wq_count,
+ snic->cq_count,
+ snic->intr_count);
+
+
+ /* Allocate WQs used for SCSI IOs */
+ for (i = 0; i < snic->wq_count; i++) {
+ ret = vnic_wq_alloc(snic->vdev,
+ &snic->wq[i],
+ i,
+ snic->config.wq_enet_desc_count,
+ sizeof(struct wq_enet_desc));
+ if (ret)
+ goto error_cleanup;
+ }
+
+ /* CQ for each WQ */
+ for (i = 0; i < snic->wq_count; i++) {
+ ret = vnic_cq_alloc(snic->vdev,
+ &snic->cq[i],
+ i,
+ snic->config.wq_enet_desc_count,
+ sizeof(struct cq_enet_wq_desc));
+ if (ret)
+ goto error_cleanup;
+ }
+
+#ifdef SNIC_DEBUG
+ SNIC_BUG_ON(snic->cq_count != 2 * snic->wq_count);
+#endif
+ /* CQ for FW TO host */
+ for (i = snic->wq_count; i < snic->cq_count; i++) {
+ ret = vnic_cq_alloc(snic->vdev,
+ &snic->cq[i],
+ i,
+ (snic->config.wq_enet_desc_count * 3),
+ sizeof(struct snic_fw_req));
+ if (ret)
+ goto error_cleanup;
+ }
+
+ for (i = 0; i < snic->intr_count; i++) {
+ ret = vnic_intr_alloc(snic->vdev, &snic->intr[i], i);
+ if (ret)
+ goto error_cleanup;
+ }
+
+ /*
+ * Init WQ Resources.
+ * WQ[0 to n] points to CQ[0 to n-1]
+ * firmware to host comm points to CQ[n to m+1]
+ */
+ err_intr_enable = 1;
+ err_intr_offset = snic->err_intr_offset;
+
+ for (i = 0; i < snic->wq_count; i++) {
+ vnic_wq_init(&snic->wq[i],
+ i,
+ err_intr_enable,
+ err_intr_offset);
+ }
+
+ for (i = 0; i < snic->cq_count; i++) {
+ intr_offset = i;
+
+ vnic_cq_init(&snic->cq[i],
+ 0 /* flow_control_enable */,
+ 1 /* color_enable */,
+ 0 /* cq_head */,
+ 0 /* cq_tail */,
+ 1 /* cq_tail_color */,
+ 1 /* interrupt_enable */,
+ 1 /* cq_entry_enable */,
+ 0 /* cq_message_enable */,
+ intr_offset,
+ 0 /* cq_message_addr */);
+ }
+
+ /*
+ * Init INTR resources
+ * Assumption : snic is always in MSI-X mode
+ */
+ SNIC_BUG_ON(intr_mode != VNIC_DEV_INTR_MODE_MSIX);
+ mask_on_assertion = 1;
+
+ for (i = 0; i < snic->intr_count; i++) {
+ vnic_intr_init(&snic->intr[i],
+ snic->config.intr_timer,
+ snic->config.intr_timer_type,
+ mask_on_assertion);
+ }
+
+ /* init the stats memory by making the first call here */
+ ret = vnic_dev_stats_dump(snic->vdev, &snic->stats);
+ if (ret) {
+ SNIC_HOST_ERR(snic->shost,
+ "vnic_dev_stats_dump failed - x%x\n",
+ ret);
+ goto error_cleanup;
+ }
+
+ /* Clear LIF stats */
+ vnic_dev_stats_clear(snic->vdev);
+ ret = 0;
+
+ return ret;
+
+error_cleanup:
+ snic_free_vnic_res(snic);
+
+ return ret;
+}
+
+void
+snic_log_q_error(struct snic *snic)
+{
+ unsigned int i;
+ u32 err_status;
+
+ for (i = 0; i < snic->wq_count; i++) {
+ err_status = ioread32(&snic->wq[i].ctrl->error_status);
+ if (err_status)
+ SNIC_HOST_ERR(snic->shost,
+ "WQ[%d] error status %d\n",
+ i,
+ err_status);
+ }
+} /* end of snic_log_q_error */
diff --git a/drivers/scsi/snic/snic_res.h b/drivers/scsi/snic/snic_res.h
new file mode 100644
index 0000000..f8f4e29
--- /dev/null
+++ b/drivers/scsi/snic/snic_res.h
@@ -0,0 +1,96 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __SNIC_RES_H
+#define __SNIC_RES_H
+
+#include "snic_io.h"
+#include "wq_enet_desc.h"
+#include "vnic_wq.h"
+#include "snic_fwint.h"
+#include "vnic_cq_fw.h"
+
+static inline void
+snic_icmnd_init(struct snic_host_req *req, u32 cmnd_id, u32 host_id, u64 ctx,
+ u16 flags, u64 tgt_id, u8 *lun, u8 *scsi_cdb, u8 cdb_len,
+ u32 data_len, u16 sg_cnt, u64 sgl_addr, u64 sns_addr_pa,
+ u32 sense_len)
+{
+ snic_io_hdr_enc(&req->hdr, SNIC_REQ_ICMND, 0, cmnd_id, host_id, sg_cnt,
+ ctx);
+
+ req->u.icmnd.flags = flags;
+ req->u.icmnd.tgt_id = tgt_id;
+ memcpy(&req->u.icmnd.lun_id, lun, LUN_ADDR_LEN);
+ req->u.icmnd.cdb_len = cdb_len;
+ memset(req->u.icmnd.cdb, 0, SNIC_CDB_LEN);
+ memcpy(req->u.icmnd.cdb, scsi_cdb, cdb_len);
+ req->u.icmnd.data_len = data_len;
+ req->u.icmnd.sg_addr = sgl_addr;
+ req->u.icmnd.sense_len = sense_len;
+ req->u.icmnd.sense_addr = sns_addr_pa;
+}
+
+static inline void
+snic_itmf_init(struct snic_host_req *req, u32 cmnd_id, u32 host_id, u64 ctx,
+ u16 flags, u32 req_id, u64 tgt_id, u8 *lun, u8 tm_type)
+{
+ snic_io_hdr_enc(&req->hdr, SNIC_REQ_ITMF, 0, cmnd_id, host_id, 0, ctx);
+
+ req->u.itmf.tm_type = tm_type;
+ req->u.itmf.flags = flags;
+ req->u.itmf.req_id = req_id; /* valid only in abort, clear task */
+ req->u.itmf.tgt_id = tgt_id;
+ memcpy(&req->u.itmf.lun_id, lun, LUN_ADDR_LEN);
+}
+
+static inline void
+snic_queue_wq_eth_desc(struct vnic_wq *wq,
+ void *os_buf,
+ dma_addr_t dma_addr,
+ unsigned int len,
+ int vlan_tag_insert,
+ unsigned int vlan_tag,
+ int cq_entry)
+{
+ struct wq_enet_desc *desc = vnic_wq_next_desc(wq);
+
+ wq_enet_desc_enc(desc,
+ (u64)dma_addr | VNIC_PADDR_TARGET,
+ (u16)len,
+ 0, /* mss_or_csum_offset */
+ 0, /* fc_eof */
+ 0, /* offload mode */
+ 1, /* eop */
+ (u8)cq_entry,
+ 0, /* fcoe_encap */
+ (u8)vlan_tag_insert,
+ (u16)vlan_tag,
+ 0 /* loopback */);
+
+ vnic_wq_post(wq, os_buf, dma_addr, len, 1, 1);
+}
+
+struct snic;
+
+int snic_get_vnic_config(struct snic *);
+int snic_alloc_vnic_res(struct snic *);
+void snic_free_vnic_res(struct snic *);
+void snic_get_res_counts(struct snic *);
+void snic_log_q_error(struct snic *);
+int snic_get_vnic_resources_size(struct snic *);
+#endif /* __SNIC_RES_H */
--
1.8.5.4
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v4 3/9] snic:Add meta request, handling of meta requests.
2015-04-09 11:49 [PATCH v4 0/9] snic:initial submission of snic driver for Cisco SCSI HBA Narsimhulu Musini
2015-04-09 11:49 ` [PATCH v4 1/9] snic: snic module infrastructure Narsimhulu Musini
2015-04-09 11:49 ` [PATCH v4 2/9] snic:Add interrupt, resource firmware interfaces Narsimhulu Musini
@ 2015-04-09 11:49 ` Narsimhulu Musini
2015-04-09 12:50 ` Hannes Reinecke
2015-04-09 11:49 ` [PATCH v4 4/9] snic:Add snic target discovery Narsimhulu Musini
` (5 subsequent siblings)
8 siblings, 1 reply; 24+ messages in thread
From: Narsimhulu Musini @ 2015-04-09 11:49 UTC (permalink / raw)
To: JBottomley, linux-scsi, hare, hch; +Cc: Narsimhulu Musini, Sesidhar Baddela
snic_io.h contains meta request structure definition
meta request contains high level information about firmware requests.
such as request information, size, SGLs.
snic_io.c contains interfaces to handle meta request, firmware acknowledgment,
and high level generic queueing interface.
Signed-off-by: Narsimhulu Musini <nmusini@cisco.com>
Signed-off-by: Sesidhar Baddela <sebaddel@cisco.com>
---
* v3
- Removed request alignment functionality.
drivers/scsi/snic/snic_io.c | 519 ++++++++++++++++++++++++++++++++++++++++++++
drivers/scsi/snic/snic_io.h | 118 ++++++++++
2 files changed, 637 insertions(+)
create mode 100644 drivers/scsi/snic/snic_io.c
create mode 100644 drivers/scsi/snic/snic_io.h
diff --git a/drivers/scsi/snic/snic_io.c b/drivers/scsi/snic/snic_io.c
new file mode 100644
index 0000000..75235ea
--- /dev/null
+++ b/drivers/scsi/snic/snic_io.c
@@ -0,0 +1,519 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+
+#include <linux/interrupt.h>
+#include <linux/workqueue.h>
+#include <linux/spinlock.h>
+#include <linux/mempool.h>
+#include <scsi/scsi_tcq.h>
+
+#include "snic_io.h"
+#include "snic.h"
+#include "cq_enet_desc.h"
+#include "snic_fwint.h"
+
+static void
+snic_wq_cmpl_frame_send(struct vnic_wq *wq,
+ struct cq_desc *cq_desc,
+ struct vnic_wq_buf *buf,
+ void *opaque)
+{
+ struct snic *snic = vnic_dev_priv(wq->vdev);
+
+ SNIC_BUG_ON(buf->os_buf == NULL);
+
+ if (snic_log_level & SNIC_DESC_LOGGING)
+ SNIC_HOST_INFO(snic->shost,
+ "Ack received for snic_host_req %p.\n",
+ buf->os_buf);
+
+ SNIC_TRC(snic->shost->host_no, 0, 0,
+ ((u64)(buf->os_buf) - sizeof(struct snic_req_info)), 0, 0, 0);
+ pci_unmap_single(snic->pdev, buf->dma_addr, buf->len, PCI_DMA_TODEVICE);
+ buf->os_buf = NULL;
+}
+
+static int
+snic_wq_cmpl_handler_cont(struct vnic_dev *vdev,
+ struct cq_desc *cq_desc,
+ u8 type,
+ u16 q_num,
+ u16 cmpl_idx,
+ void *opaque)
+{
+ struct snic *snic = vnic_dev_priv(vdev);
+ unsigned long flags;
+
+ SNIC_BUG_ON(q_num != 0);
+
+ spin_lock_irqsave(&snic->wq_lock[q_num], flags);
+ vnic_wq_service(&snic->wq[q_num],
+ cq_desc,
+ cmpl_idx,
+ snic_wq_cmpl_frame_send,
+ NULL);
+ spin_unlock_irqrestore(&snic->wq_lock[q_num], flags);
+
+ return 0;
+} /* end of snic_cmpl_handler_cont */
+
+int
+snic_wq_cmpl_handler(struct snic *snic, int work_to_do)
+{
+ unsigned int work_done = 0;
+ unsigned int i;
+
+ snic->s_stats.misc.last_ack_time = jiffies;
+ for (i = 0; i < snic->wq_count; i++) {
+ work_done += vnic_cq_service(&snic->cq[i],
+ work_to_do,
+ snic_wq_cmpl_handler_cont,
+ NULL);
+ }
+
+ return work_done;
+} /* end of snic_wq_cmpl_handler */
+
+void
+snic_free_wq_buf(struct vnic_wq *wq, struct vnic_wq_buf *buf)
+{
+
+ struct snic_host_req *req = buf->os_buf;
+ struct snic *snic = vnic_dev_priv(wq->vdev);
+ struct snic_req_info *rqi = NULL;
+ unsigned long flags;
+
+ pci_unmap_single(snic->pdev, buf->dma_addr, buf->len, PCI_DMA_TODEVICE);
+
+ rqi = req_to_rqi(req);
+ spin_lock_irqsave(&snic->spl_cmd_lock, flags);
+ if (list_empty(&rqi->list)) {
+ spin_unlock_irqrestore(&snic->spl_cmd_lock, flags);
+ goto end;
+ }
+
+ SNIC_BUG_ON(rqi->list.next == NULL); /* if not added to spl_cmd_list */
+ list_del_init(&rqi->list);
+ spin_unlock_irqrestore(&snic->spl_cmd_lock, flags);
+
+ if (rqi->sge_va) {
+ snic_pci_unmap_rsp_buf(snic, rqi);
+ kfree((void *)rqi->sge_va);
+ rqi->sge_va = 0;
+ }
+ snic_req_free(snic, rqi);
+ SNIC_HOST_INFO(snic->shost, "snic_free_wq_buf .. freed.\n");
+
+end:
+ return;
+}
+
+/* Criteria to select work queue in multi queue mode */
+static int
+snic_select_wq(struct snic *snic)
+{
+ /* No multi queue support for now */
+ BUILD_BUG_ON(SNIC_WQ_MAX > 1);
+
+ return 0;
+}
+
+int
+snic_queue_wq_desc(struct snic *snic, void *os_buf, u16 len)
+{
+ dma_addr_t pa = 0;
+ unsigned long flags;
+ struct snic_fw_stats *fwstats = &snic->s_stats.fw;
+ long act_reqs;
+ int q_num = 0;
+
+ snic_print_desc(__func__, os_buf, len);
+
+ /* Map request buffer */
+ pa = pci_map_single(snic->pdev, os_buf, len, PCI_DMA_TODEVICE);
+ if (pci_dma_mapping_error(snic->pdev, pa)) {
+ SNIC_HOST_ERR(snic->shost, "qdesc: PCI DMA Mapping Fail.\n");
+
+ return -ENOMEM;
+ }
+
+ q_num = snic_select_wq(snic);
+
+ spin_lock_irqsave(&snic->wq_lock[q_num], flags);
+ if (!vnic_wq_desc_avail(snic->wq)) {
+ pci_unmap_single(snic->pdev, pa, len, PCI_DMA_TODEVICE);
+ spin_unlock_irqrestore(&snic->wq_lock[q_num], flags);
+ atomic64_inc(&snic->s_stats.misc.wq_alloc_fail);
+ SNIC_DBG("host = %d, WQ is Full\n", snic->shost->host_no);
+
+ return -ENOMEM;
+ }
+
+ snic_queue_wq_eth_desc(&snic->wq[q_num], os_buf, pa, len, 0, 0, 1);
+ spin_unlock_irqrestore(&snic->wq_lock[q_num], flags);
+
+ /* Update stats */
+ act_reqs = atomic64_inc_return(&fwstats->actv_reqs);
+ if (act_reqs > atomic64_read(&fwstats->max_actv_reqs))
+ atomic64_set(&fwstats->max_actv_reqs, act_reqs);
+
+ return 0;
+} /* end of snic_queue_wq_desc() */
+
+/*
+ * snic_handle_untagged_req: Adds snic specific requests to spl_cmd_list.
+ * Purpose : Used during driver unload to clean up the requests.
+ */
+void
+snic_handle_untagged_req(struct snic *snic, struct snic_req_info *rqi)
+{
+ unsigned long flags;
+
+ INIT_LIST_HEAD(&rqi->list);
+
+ spin_lock_irqsave(&snic->spl_cmd_lock, flags);
+ list_add_tail(&rqi->list, &snic->spl_cmd_list);
+ spin_unlock_irqrestore(&snic->spl_cmd_lock, flags);
+}
+
+/*
+ * snic_req_init:
+ * Allocates snic_req_info + snic_host_req + sgl data, and initializes.
+ */
+struct snic_req_info *
+snic_req_init(struct snic *snic, int sg_cnt)
+{
+ u8 typ;
+ struct snic_req_info *rqi = NULL;
+
+ typ = (sg_cnt <= SNIC_REQ_CACHE_DFLT_SGL) ?
+ SNIC_REQ_CACHE_DFLT_SGL : SNIC_REQ_CACHE_MAX_SGL;
+
+ rqi = mempool_alloc(snic->req_pool[typ], GFP_ATOMIC);
+ if (!rqi) {
+ atomic64_inc(&snic->s_stats.io.alloc_fail);
+ SNIC_HOST_ERR(snic->shost,
+ "Failed to allocate memory from snic req pool id = %d\n",
+ typ);
+ return rqi;
+ }
+
+ memset(rqi, 0, sizeof(*rqi));
+ rqi->rq_pool_type = typ;
+ rqi->start_time = jiffies;
+ rqi->req = (struct snic_host_req *) (rqi + 1);
+ rqi->req_len = sizeof(struct snic_host_req);
+ rqi->snic = snic;
+
+ BUILD_BUG_ON((sizeof(struct snic_req_info) % SNIC_SG_DESC_ALIGN) != 0);
+
+ rqi->req = (struct snic_host_req *)(rqi + 1);
+
+ if (sg_cnt == 0)
+ goto end;
+
+ rqi->req_len += (sg_cnt * sizeof(struct snic_sg_desc));
+
+ if (sg_cnt > atomic64_read(&snic->s_stats.io.max_sgl))
+ atomic64_set(&snic->s_stats.io.max_sgl, sg_cnt);
+
+ SNIC_BUG_ON(sg_cnt > SNIC_MAX_SG_DESC_CNT);
+ atomic64_inc(&snic->s_stats.io.sgl_cnt[sg_cnt - 1]);
+
+end:
+ memset(rqi->req, 0, rqi->req_len);
+
+ /* pre initialization of init_ctx to support req_to_rqi */
+ rqi->req->hdr.init_ctx = (u64) rqi;
+
+ SNIC_SCSI_DBG(snic->shost, "Req_alloc:rqi = %p allocatd.\n", rqi);
+
+ return rqi;
+} /* end of snic_req_init */
+
+/*
+ * snic_abort_req_init : Inits abort request.
+ */
+struct snic_host_req *
+snic_abort_req_init(struct snic *snic, struct snic_req_info *rqi)
+{
+ struct snic_host_req *req = NULL;
+
+ SNIC_BUG_ON(!rqi);
+
+ /* If abort to be issued second time, then reuse */
+ if (rqi->abort_req)
+ return rqi->abort_req;
+
+
+ req = mempool_alloc(snic->req_pool[SNIC_REQ_TM_CACHE], GFP_ATOMIC);
+ if (!req) {
+ SNIC_HOST_ERR(snic->shost, "abts:Failed to alloc tm req.\n");
+ WARN_ON_ONCE(1);
+
+ return NULL;
+ }
+
+ rqi->abort_req = req;
+ memset(req, 0, sizeof(struct snic_host_req));
+ /* pre initialization of init_ctx to support req_to_rqi */
+ req->hdr.init_ctx = (u64) rqi;
+
+ return req;
+} /* end of snic_abort_req_init */
+
+/*
+ * snic_dr_req_init : Inits device reset req
+ */
+struct snic_host_req *
+snic_dr_req_init(struct snic *snic, struct snic_req_info *rqi)
+{
+ struct snic_host_req *req = NULL;
+
+ SNIC_BUG_ON(!rqi);
+
+ req = mempool_alloc(snic->req_pool[SNIC_REQ_TM_CACHE], GFP_ATOMIC);
+ if (!req) {
+ SNIC_HOST_ERR(snic->shost, "dr:Failed to alloc tm req.\n");
+ WARN_ON_ONCE(1);
+
+ return NULL;
+ }
+
+ SNIC_BUG_ON(rqi->dr_req != NULL);
+ rqi->dr_req = req;
+ memset(req, 0, sizeof(struct snic_host_req));
+ /* pre initialization of init_ctx to support req_to_rqi */
+ req->hdr.init_ctx = (u64) rqi;
+
+ return req;
+} /* end of snic_dr_req_init */
+
+/* frees snic_req_info and snic_host_req */
+void
+snic_req_free(struct snic *snic, struct snic_req_info *rqi)
+{
+ SNIC_BUG_ON(rqi->req == rqi->abort_req);
+ SNIC_BUG_ON(rqi->req == rqi->dr_req);
+ SNIC_BUG_ON(rqi->sge_va != 0);
+
+ SNIC_SCSI_DBG(snic->shost,
+ "Req_free:rqi %p:ioreq %p:abt %p:dr %p\n",
+ rqi, rqi->req, rqi->abort_req, rqi->dr_req);
+
+ if (rqi->abort_req)
+ mempool_free(rqi->abort_req, snic->req_pool[SNIC_REQ_TM_CACHE]);
+
+ if (rqi->dr_req)
+ mempool_free(rqi->dr_req, snic->req_pool[SNIC_REQ_TM_CACHE]);
+
+ mempool_free(rqi, snic->req_pool[rqi->rq_pool_type]);
+}
+
+void
+snic_pci_unmap_rsp_buf(struct snic *snic, struct snic_req_info *rqi)
+{
+ struct snic_sg_desc *sgd;
+
+ sgd = req_to_sgl(rqi_to_req(rqi));
+ SNIC_BUG_ON(sgd[0].addr == 0);
+ pci_unmap_single(snic->pdev,
+ sgd[0].addr,
+ sgd[0].len,
+ PCI_DMA_FROMDEVICE);
+}
+
+/*
+ * snic_free_all_untagged_reqs: Walks through untagged reqs and frees them.
+ */
+void
+snic_free_all_untagged_reqs(struct snic *snic)
+{
+ struct snic_req_info *rqi;
+ struct list_head *cur, *nxt;
+ unsigned long flags;
+
+ spin_lock_irqsave(&snic->spl_cmd_lock, flags);
+ list_for_each_safe(cur, nxt, &snic->spl_cmd_list) {
+ rqi = list_entry(cur, struct snic_req_info, list);
+ list_del_init(&rqi->list);
+ if (rqi->sge_va) {
+ snic_pci_unmap_rsp_buf(snic, rqi);
+ kfree((void *)rqi->sge_va);
+ rqi->sge_va = 0;
+ }
+
+ snic_req_free(snic, rqi);
+ }
+ spin_unlock_irqrestore(&snic->spl_cmd_lock, flags);
+}
+
+/*
+ * snic_release_untagged_req : Unlinks the untagged req and frees it.
+ */
+void
+snic_release_untagged_req(struct snic *snic, struct snic_req_info *rqi)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&snic->snic_lock, flags);
+ if (snic->in_remove) {
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+ goto end;
+ }
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+
+ spin_lock_irqsave(&snic->spl_cmd_lock, flags);
+ if (list_empty(&rqi->list)) {
+ spin_unlock_irqrestore(&snic->spl_cmd_lock, flags);
+ goto end;
+ }
+ list_del_init(&rqi->list);
+ spin_unlock_irqrestore(&snic->spl_cmd_lock, flags);
+ snic_req_free(snic, rqi);
+
+end:
+ return;
+}
+
+/* dump buf in hex fmt */
+void
+snic_hex_dump(char *pfx, char *data, int len)
+{
+ SNIC_INFO("%s Dumping Data of Len = %d\n", pfx, len);
+ print_hex_dump_bytes(pfx, DUMP_PREFIX_NONE, data, len);
+}
+
+#define LINE_BUFSZ 128 /* for snic_print_desc fn */
+static void
+snic_dump_desc(const char *fn, char *os_buf, int len)
+{
+ struct snic_host_req *req = (struct snic_host_req *) os_buf;
+ struct snic_fw_req *fwreq = (struct snic_fw_req *) os_buf;
+ struct snic_req_info *rqi = NULL;
+ char line[LINE_BUFSZ] = { '\0' };
+ char *cmd_str = NULL;
+
+ if (req->hdr.type >= SNIC_RSP_REPORT_TGTS_CMPL)
+ rqi = (struct snic_req_info *) fwreq->hdr.init_ctx;
+ else
+ rqi = (struct snic_req_info *) req->hdr.init_ctx;
+
+ SNIC_BUG_ON(rqi == NULL || rqi->req == NULL);
+ switch (req->hdr.type) {
+ case SNIC_REQ_REPORT_TGTS:
+ cmd_str = "report-tgt : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_REQ_REPORT_TGTS :");
+ break;
+
+ case SNIC_REQ_ICMND:
+ cmd_str = "icmnd : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_REQ_ICMND : 0x%x :",
+ req->u.icmnd.cdb[0]);
+ break;
+
+ case SNIC_REQ_ITMF:
+ cmd_str = "itmf : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_REQ_ITMF :");
+ break;
+
+ case SNIC_REQ_HBA_RESET:
+ cmd_str = "hba reset :";
+ snprintf(line, LINE_BUFSZ, "SNIC_REQ_HBA_RESET :");
+ break;
+
+ case SNIC_REQ_EXCH_VER:
+ cmd_str = "exch ver : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_REQ_EXCH_VER :");
+ break;
+
+ case SNIC_REQ_TGT_INFO:
+ cmd_str = "tgt info : ";
+ break;
+
+ case SNIC_RSP_REPORT_TGTS_CMPL:
+ cmd_str = "report tgt cmpl : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_RSP_REPORT_TGTS_CMPL :");
+ break;
+
+ case SNIC_RSP_ICMND_CMPL:
+ cmd_str = "icmnd_cmpl : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_RSP_ICMND_CMPL : 0x%x :",
+ rqi->req->u.icmnd.cdb[0]);
+ break;
+
+ case SNIC_RSP_ITMF_CMPL:
+ cmd_str = "itmf_cmpl : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_RSP_ITMF_CMPL :");
+ break;
+
+ case SNIC_RSP_HBA_RESET_CMPL:
+ cmd_str = "hba_reset_cmpl : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_RSP_HBA_RESET_CMPL :");
+ break;
+
+ case SNIC_RSP_EXCH_VER_CMPL:
+ cmd_str = "exch_ver_cmpl : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_RSP_EXCH_VER_CMPL :");
+ break;
+
+ case SNIC_MSG_ACK:
+ cmd_str = "msg ack : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_MSG_ACK :");
+ break;
+
+ case SNIC_MSG_ASYNC_EVNOTIFY:
+ cmd_str = "async notify : ";
+ snprintf(line, LINE_BUFSZ, "SNIC_MSG_ASYNC_EVNOTIFY :");
+ break;
+
+ default:
+ cmd_str = "unknown : ";
+ SNIC_BUG_ON(1);
+ break;
+ }
+
+ SNIC_INFO("%s:%s >>cmndid=%x:sg_cnt = %x:status = %x:ctx = %llx.\n",
+ fn, line, req->hdr.cmnd_id, req->hdr.sg_cnt, req->hdr.status,
+ req->hdr.init_ctx);
+
+ /* Enable it, to dump byte stream */
+ if (snic_log_level & 0x20)
+ snic_hex_dump(cmd_str, os_buf, len);
+} /* end of __snic_print_desc */
+
+void
+snic_print_desc(const char *fn, char *os_buf, int len)
+{
+ if (snic_log_level & SNIC_DESC_LOGGING)
+ snic_dump_desc(fn, os_buf, len);
+}
+
+void
+snic_calc_io_process_time(struct snic *snic, struct snic_req_info *rqi)
+{
+ u64 duration;
+
+ duration = jiffies - rqi->start_time;
+
+ if (duration > atomic64_read(&snic->s_stats.io.max_time))
+ atomic64_set(&snic->s_stats.io.max_time, duration);
+}
diff --git a/drivers/scsi/snic/snic_io.h b/drivers/scsi/snic/snic_io.h
new file mode 100644
index 0000000..eca16e4
--- /dev/null
+++ b/drivers/scsi/snic/snic_io.h
@@ -0,0 +1,118 @@
+/*
+ * Copyright 2014 Cisco Systems, Inc. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _SNIC_IO_H
+#define _SNIC_IO_H
+
+#define SNIC_DFLT_SG_DESC_CNT 32 /* Default descriptors for sgl */
+#define SNIC_MAX_SG_DESC_CNT 60 /* Max descriptor for sgl */
+#define SNIC_SG_DESC_ALIGN 16 /* Descriptor address alignment */
+
+/* SG descriptor for snic */
+struct snic_sg_desc {
+ __le64 addr;
+ __le