* Re: [PATCH][RFC 0/12/1/5] SCST core
[not found] <4BC44A49.7070307@vlnb.net>
@ 2010-04-13 13:04 ` Vladislav Bolkhovitin
[not found] ` <4BC44D08.4060907@vlnb.net>
` (5 subsequent siblings)
6 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:04 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patchset contains SCST core together with documentation. Where
possible, it was divided based on functionality grouping, in other cases
it was divided by per file basis.
Vlad
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 1/12/1/5] SCST core's Makefile and Kconfig
[not found] ` <4BC44D08.4060907@vlnb.net>
@ 2010-04-13 13:04 ` Vladislav Bolkhovitin
2010-04-13 13:04 ` [PATCH][RFC 2/12/1/5] SCST core's external headers Vladislav Bolkhovitin
` (8 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:04 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Bart Van Assche,
James Smart, Joe Eykholt, Andy Yan, linux-driver, Vu Pham,
Linus Torvalds
This patch contains SCST core's Makefile and Kconfig.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Kconfig | 246 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Makefile | 11 ++
2 files changed, 257 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/Kconfig linux-2.6.33/drivers/scst/Kconfig
--- orig/linux-2.6.33/drivers/scst/Kconfig
+++ linux-2.6.33/drivers/scst/Kconfig
@@ -0,0 +1,246 @@
+menu "SCSI target (SCST) support"
+
+config SCST
+ tristate "SCSI target (SCST) support"
+ depends on SCSI
+ help
+ SCSI target (SCST) is designed to provide unified, consistent
+ interface between SCSI target drivers and Linux kernel and
+ simplify target drivers development as much as possible. Visit
+ http://scst.sourceforge.net for more info about it.
+
+config SCST_DISK
+ tristate "SCSI target disk support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for disk device.
+
+config SCST_TAPE
+ tristate "SCSI target tape support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for tape device.
+
+config SCST_CDROM
+ tristate "SCSI target CDROM support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for CDROM device.
+
+config SCST_MODISK
+ tristate "SCSI target MO disk support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for MO disk device.
+
+config SCST_CHANGER
+ tristate "SCSI target changer support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for changer device.
+
+config SCST_PROCESSOR
+ tristate "SCSI target processor support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for processor device.
+
+config SCST_RAID
+ tristate "SCSI target storage array controller (RAID) support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST pass-through device handler for raid storage array controller (RAID) device.
+
+config SCST_VDISK
+ tristate "SCSI target virtual disk and/or CDROM support"
+ default SCST
+ depends on SCSI && SCST
+ help
+ SCST device handler for virtual disk and/or CDROM device.
+
+config SCST_STRICT_SERIALIZING
+ bool "Strict serialization"
+ depends on SCST
+ help
+ Enable strict SCSI command serialization. When enabled, SCST sends
+ all SCSI commands to the underlying SCSI device synchronously, one
+ after one. This makes task management more reliable, at the cost of
+ a performance penalty. This is most useful for stateful SCSI devices
+ like tapes, where the result of the execution of a command
+ depends on the device settings configured by previous commands. Disk
+ and RAID devices are stateless in most cases. The current SCSI core
+ in Linux doesn't allow to abort all commands reliably if they have
+ been sent asynchronously to a stateful device.
+ Enable this option if you use stateful device(s) and need as much
+ error recovery reliability as possible.
+
+ If unsure, say "N".
+
+config SCST_STRICT_SECURITY
+ bool "Strict security"
+ depends on SCST
+ help
+ Makes SCST clear (zero-fill) allocated data buffers. Note: this has a
+ significant performance penalty.
+
+ If unsure, say "N".
+
+config SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ
+ bool "Allow pass-through commands to be sent from soft-IRQ context"
+ depends on SCST
+ help
+ Allows SCST to submit SCSI pass-through commands to real SCSI devices
+ via the SCSI middle layer using scsi_execute_async() function from
+ soft-IRQ context (tasklets). This used to be the default, but
+ currently it seems the SCSI middle layer starts expecting only thread
+ context on the IO submit path, so it is disabled now by default.
+ Enabling it will decrease amount of context switches and improve
+ performance. It is more or less safe. In the worst case, if in your
+ configuration the SCSI middle layer really doesn't expect SIRQ
+ context in scsi_execute_async() function, you will get a warning
+ message in the kernel log.
+
+ If unsure, say "N".
+
+config SCST_ABORT_CONSIDER_FINISHED_TASKS_AS_NOT_EXISTING
+ bool "Send back UNKNOWN TASK when an already finished task is aborted"
+ depends on SCST
+ help
+ Controls which response is sent by SCST to the initiator in case
+ the initiator attempts to abort (ABORT TASK) an already finished
+ request. If this option is enabled, the response UNKNOWN TASK is
+ sent back to the initiator. However, some initiators, particularly
+ the VMware iSCSI initiator, interpret the UNKNOWN TASK response as
+ if the target got crazy and try to RESET it. Then sometimes the
+ initiator gets crazy itself.
+
+ If unsure, say "N".
+
+config SCST_USE_EXPECTED_VALUES
+ bool "Prefer initiator-supplied SCSI command attributes"
+ depends on SCST
+ help
+ When SCST receives a SCSI command from an initiator, such a SCSI
+ command has both data transfer length and direction attributes.
+ There are two possible sources for these attributes: either the
+ values computed by SCST from its internal command translation table
+ or the values supplied by the initiator. The former are used by
+ default because of security reasons. Invalid initiator-supplied
+ attributes can crash the target, especially in pass-through mode.
+ Only consider enabling this option when SCST logs the following
+ message: "Unknown opcode XX for YY. Should you update
+ scst_scsi_op_table?" and when the initiator complains. Please
+ report any unrecognized commands to scst-devel@lists.sourceforge.net.
+
+ If unsure, say "N".
+
+config SCST_EXTRACHECKS
+ bool "Extra consistency checks"
+ depends on SCST
+ help
+ Enable additional consistency checks in the SCSI middle level target
+ code. This may be helpful for SCST developers. Enable it if you have
+ any problems.
+
+ If unsure, say "N".
+
+config SCST_TRACING
+ bool "Tracing support"
+ depends on SCST
+ default y
+ help
+ Enable SCSI middle level tracing support. Tracing can be controlled
+ dynamically via sysfs interface. The traced information
+ is sent to the kernel log and may be very helpful when analyzing
+ the cause of a communication problem between initiator and target.
+
+ If unsure, say "Y".
+
+config SCST_DEBUG
+ bool "Debugging support"
+ depends on SCST
+ select DEBUG_BUGVERBOSE
+ help
+ Enables support for debugging SCST. This may be helpful for SCST
+ developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_OOM
+ bool "Out-of-memory debugging support"
+ depends on SCST
+ help
+ Let SCST's internal memory allocation function
+ (scst_alloc_sg_entries()) fail about once in every 10000 calls, at
+ least if the flag __GFP_NOFAIL has not been set. This allows SCST
+ developers to test the behavior of SCST in out-of-memory conditions.
+ This may be helpful for SCST developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_RETRY
+ bool "SCSI command retry debugging support"
+ depends on SCST
+ help
+ Let SCST's internal SCSI command transfer function
+ (scst_rdy_to_xfer()) fail about once in every 100 calls. This allows
+ SCST developers to test the behavior of SCST when SCSI queues fill
+ up. This may be helpful for SCST developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_SN
+ bool "SCSI sequence number debugging support"
+ depends on SCST
+ help
+ Allows to test SCSI command ordering via sequence numbers by
+ randomly changing the type of SCSI commands into
+ SCST_CMD_QUEUE_ORDERED, SCST_CMD_QUEUE_HEAD_OF_QUEUE or
+ SCST_CMD_QUEUE_SIMPLE for about one in 300 SCSI commands.
+ This may be helpful for SCST developers.
+
+ If unsure, say "N".
+
+config SCST_DEBUG_TM
+ bool "Task management debugging support"
+ depends on SCST_DEBUG
+ help
+ Enables support for debugging of SCST's task management functions.
+ When enabled, some of the commands on LUN 0 in the default access
+ control group will be delayed for about 60 seconds. This will
+ cause the remote initiator send SCSI task management functions,
+ e.g. ABORT TASK and TARGET RESET.
+
+ If unsure, say "N".
+
+config SCST_TM_DBG_GO_OFFLINE
+ bool "Let devices become completely unresponsive"
+ depends on SCST_DEBUG_TM
+ help
+ Enable this option if you want that the device eventually becomes
+ completely unresponsive. When disabled, the device will receive
+ ABORT and RESET commands.
+
+config SCST_MEASURE_LATENCY
+ bool "Commands processing latency measurement facility"
+ depends on SCST
+ help
+ This option enables commands processing latency measurement
+ facility in SCST. It will provide in the sysfs interface
+ average commands processing latency statistics. You can clear
+ already measured results by writing 0 in the corresponding sysfs file.
+ Note, you need a non-preemtible kernel to have correct results.
+
+ If unsure, say "N".
+
+source "drivers/scst/iscsi-scst/Kconfig"
+source "drivers/scst/srpt/Kconfig"
+
+endmenu
diff -uprN orig/linux-2.6.33/drivers/scst/Makefile linux-2.6.33/drivers/scst/Makefile
--- orig/linux-2.6.33/drivers/scst/Makefile
+++ linux-2.6.33/drivers/scst/Makefile
@@ -0,0 +1,11 @@
+ccflags-y += -Iinclude/scst -Wno-unused-parameter
+
+scst-y += scst_main.o
+scst-y += scst_targ.o
+scst-y += scst_lib.o
+scst-y += scst_sysfs.o
+scst-y += scst_mem.o
+scst-y += scst_debug.o
+
+obj-$(CONFIG_SCST) += scst.o dev_handlers/ iscsi-scst/ srpt/
+
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 2/12/1/5] SCST core's external headers
[not found] ` <4BC44D08.4060907@vlnb.net>
2010-04-13 13:04 ` [PATCH][RFC 1/12/1/5] SCST core's Makefile and Kconfig Vladislav Bolkhovitin
@ 2010-04-13 13:04 ` Vladislav Bolkhovitin
2010-04-13 13:04 ` [PATCH][RFC 3/12/1/5] SCST core's scst_main.c Vladislav Bolkhovitin
` (7 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:04 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains declarations of all externally visible SCST constants,
types and functions prototypes.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst.h | 3169 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
scst_const.h | 330 ++++++
2 files changed, 3499 insertions(+)
diff -uprN orig/linux-2.6.33/include/scst/scst_const.h linux-2.6.33/include/scst/scst_const.h
--- orig/linux-2.6.33/include/scst/scst_const.h
+++ linux-2.6.33/include/scst/scst_const.h
@@ -0,0 +1,330 @@
+/*
+ * include/scst_const.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Contains common SCST constants.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_CONST_H
+#define __SCST_CONST_H
+
+#include <scsi/scsi.h>
+
+#define SCST_CONST_VERSION "$Revision: 1585 $"
+
+/*** Shared constants between user and kernel spaces ***/
+
+/* Max size of CDB */
+#define SCST_MAX_CDB_SIZE 16
+
+/* Max size of various names */
+#define SCST_MAX_NAME 50
+
+/* Max size of external names, like initiator name */
+#define SCST_MAX_EXTERNAL_NAME 256
+
+/*
+ * Size of sense sufficient to carry standard sense data.
+ * Warning! It's allocated on stack!
+ */
+#define SCST_STANDARD_SENSE_LEN 18
+
+/* Max size of sense */
+#define SCST_SENSE_BUFFERSIZE 96
+
+/*************************************************************
+ ** Allowed delivery statuses for cmd's delivery_status
+ *************************************************************/
+
+#define SCST_CMD_DELIVERY_SUCCESS 0
+#define SCST_CMD_DELIVERY_FAILED -1
+#define SCST_CMD_DELIVERY_ABORTED -2
+
+/*************************************************************
+ ** Values for task management functions
+ *************************************************************/
+#define SCST_ABORT_TASK 0
+#define SCST_ABORT_TASK_SET 1
+#define SCST_CLEAR_ACA 2
+#define SCST_CLEAR_TASK_SET 3
+#define SCST_LUN_RESET 4
+#define SCST_TARGET_RESET 5
+
+/** SCST extensions **/
+
+/*
+ * Notifies about I_T nexus loss event in the corresponding session.
+ * Aborts all tasks there, resets the reservation, if any, and sets
+ * up the I_T Nexus loss UA.
+ */
+#define SCST_NEXUS_LOSS_SESS 6
+
+/* Aborts all tasks in the corresponding session */
+#define SCST_ABORT_ALL_TASKS_SESS 7
+
+/*
+ * Notifies about I_T nexus loss event. Aborts all tasks in all sessions
+ * of the tgt, resets the reservations, if any, and sets up the I_T Nexus
+ * loss UA.
+ */
+#define SCST_NEXUS_LOSS 8
+
+/* Aborts all tasks in all sessions of the tgt */
+#define SCST_ABORT_ALL_TASKS 9
+
+/*
+ * Internal TM command issued by SCST in scst_unregister_session(). It is the
+ * same as SCST_NEXUS_LOSS_SESS, except:
+ * - it doesn't call task_mgmt_affected_cmds_done()
+ * - it doesn't call task_mgmt_fn_done()
+ * - it doesn't queue NEXUS LOSS UA.
+ *
+ * Target driver shall NEVER use it!!
+ */
+#define SCST_UNREG_SESS_TM 10
+
+/*************************************************************
+ ** Values for mgmt cmd's status field. Codes taken from iSCSI
+ *************************************************************/
+#define SCST_MGMT_STATUS_SUCCESS 0
+#define SCST_MGMT_STATUS_TASK_NOT_EXIST -1
+#define SCST_MGMT_STATUS_LUN_NOT_EXIST -2
+#define SCST_MGMT_STATUS_FN_NOT_SUPPORTED -5
+#define SCST_MGMT_STATUS_REJECTED -255
+#define SCST_MGMT_STATUS_FAILED -129
+
+/*************************************************************
+ ** SCSI task attribute queue types
+ *************************************************************/
+enum scst_cmd_queue_type {
+ SCST_CMD_QUEUE_UNTAGGED = 0,
+ SCST_CMD_QUEUE_SIMPLE,
+ SCST_CMD_QUEUE_ORDERED,
+ SCST_CMD_QUEUE_HEAD_OF_QUEUE,
+ SCST_CMD_QUEUE_ACA
+};
+
+/*************************************************************
+ ** CDB flags
+ *************************************************************/
+enum scst_cdb_flags {
+ SCST_TRANSFER_LEN_TYPE_FIXED = 0x001,
+ SCST_SMALL_TIMEOUT = 0x002,
+ SCST_LONG_TIMEOUT = 0x004,
+ SCST_UNKNOWN_LENGTH = 0x008,
+ SCST_INFO_VALID = 0x010, /* must be single bit */
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED = 0x020,
+ SCST_IMPLICIT_HQ = 0x040,
+ SCST_SKIP_UA = 0x080,
+ SCST_WRITE_MEDIUM = 0x100,
+ SCST_LOCAL_CMD = 0x200,
+ SCST_FULLY_LOCAL_CMD = 0x400,
+ SCST_REG_RESERVE_ALLOWED = 0x800,
+};
+
+/*************************************************************
+ ** Data direction aliases. Changing it don't forget to change
+ ** scst_to_tgt_dma_dir as well!!
+ *************************************************************/
+#define SCST_DATA_UNKNOWN 0
+#define SCST_DATA_WRITE 1
+#define SCST_DATA_READ 2
+#define SCST_DATA_BIDI (SCST_DATA_WRITE | SCST_DATA_READ)
+#define SCST_DATA_NONE 4
+
+/*************************************************************
+ ** Default suffix for targets with NULL names
+ *************************************************************/
+#define SCST_DEFAULT_TGT_NAME_SUFFIX "_target_"
+
+/*************************************************************
+ ** Sense manipulation and examination
+ *************************************************************/
+#define SCST_LOAD_SENSE(key_asc_ascq) key_asc_ascq
+
+#define SCST_SENSE_VALID(sense) ((sense != NULL) && \
+ ((((const uint8_t *)(sense))[0] & 0x70) == 0x70))
+
+#define SCST_NO_SENSE(sense) ((sense != NULL) && \
+ (((const uint8_t *)(sense))[2] == 0))
+
+/*************************************************************
+ ** Sense data for the appropriate errors. Can be used with
+ ** scst_set_cmd_error()
+ *************************************************************/
+#define scst_sense_no_sense NO_SENSE, 0x00, 0
+#define scst_sense_hardw_error HARDWARE_ERROR, 0x44, 0
+#define scst_sense_aborted_command ABORTED_COMMAND, 0x00, 0
+#define scst_sense_invalid_opcode ILLEGAL_REQUEST, 0x20, 0
+#define scst_sense_invalid_field_in_cdb ILLEGAL_REQUEST, 0x24, 0
+#define scst_sense_invalid_field_in_parm_list ILLEGAL_REQUEST, 0x26, 0
+#define scst_sense_parameter_value_invalid ILLEGAL_REQUEST, 0x26, 2
+#define scst_sense_reset_UA UNIT_ATTENTION, 0x29, 0
+#define scst_sense_nexus_loss_UA UNIT_ATTENTION, 0x29, 0x7
+#define scst_sense_saving_params_unsup ILLEGAL_REQUEST, 0x39, 0
+#define scst_sense_lun_not_supported ILLEGAL_REQUEST, 0x25, 0
+#define scst_sense_data_protect DATA_PROTECT, 0x00, 0
+#define scst_sense_miscompare_error MISCOMPARE, 0x1D, 0
+#define scst_sense_block_out_range_error ILLEGAL_REQUEST, 0x21, 0
+#define scst_sense_medium_changed_UA UNIT_ATTENTION, 0x28, 0
+#define scst_sense_read_error MEDIUM_ERROR, 0x11, 0
+#define scst_sense_write_error MEDIUM_ERROR, 0x03, 0
+#define scst_sense_not_ready NOT_READY, 0x04, 0x10
+#define scst_sense_invalid_message ILLEGAL_REQUEST, 0x49, 0
+#define scst_sense_cleared_by_another_ini_UA UNIT_ATTENTION, 0x2F, 0
+#define scst_sense_capacity_data_changed UNIT_ATTENTION, 0x2A, 0x9
+#define scst_sense_reported_luns_data_changed UNIT_ATTENTION, 0x3F, 0xE
+#define scst_sense_inquery_data_changed UNIT_ATTENTION, 0x3F, 0x3
+
+/*************************************************************
+ * SCSI opcodes not listed anywhere else
+ *************************************************************/
+#define REPORT_DEVICE_IDENTIFIER 0xA3
+#define INIT_ELEMENT_STATUS 0x07
+#define INIT_ELEMENT_STATUS_RANGE 0x37
+#define PREVENT_ALLOW_MEDIUM 0x1E
+#define READ_ATTRIBUTE 0x8C
+#define REQUEST_VOLUME_ADDRESS 0xB5
+#define WRITE_ATTRIBUTE 0x8D
+#define WRITE_VERIFY_16 0x8E
+#define VERIFY_6 0x13
+#define VERIFY_12 0xAF
+
+/*************************************************************
+ ** SCSI Architecture Model (SAM) Status codes. Taken from SAM-3 draft
+ ** T10/1561-D Revision 4 Draft dated 7th November 2002.
+ *************************************************************/
+#define SAM_STAT_GOOD 0x00
+#define SAM_STAT_CHECK_CONDITION 0x02
+#define SAM_STAT_CONDITION_MET 0x04
+#define SAM_STAT_BUSY 0x08
+#define SAM_STAT_INTERMEDIATE 0x10
+#define SAM_STAT_INTERMEDIATE_CONDITION_MET 0x14
+#define SAM_STAT_RESERVATION_CONFLICT 0x18
+#define SAM_STAT_COMMAND_TERMINATED 0x22 /* obsolete in SAM-3 */
+#define SAM_STAT_TASK_SET_FULL 0x28
+#define SAM_STAT_ACA_ACTIVE 0x30
+#define SAM_STAT_TASK_ABORTED 0x40
+
+/*************************************************************
+ ** Control byte field in CDB
+ *************************************************************/
+#define CONTROL_BYTE_LINK_BIT 0x01
+#define CONTROL_BYTE_NACA_BIT 0x04
+
+/*************************************************************
+ ** Byte 1 in INQUIRY CDB
+ *************************************************************/
+#define SCST_INQ_EVPD 0x01
+
+/*************************************************************
+ ** Byte 3 in Standard INQUIRY data
+ *************************************************************/
+#define SCST_INQ_BYTE3 3
+
+#define SCST_INQ_NORMACA_BIT 0x20
+
+/*************************************************************
+ ** Byte 2 in RESERVE_10 CDB
+ *************************************************************/
+#define SCST_RES_3RDPTY 0x10
+#define SCST_RES_LONGID 0x02
+
+/*************************************************************
+ ** Values for the control mode page TST field
+ *************************************************************/
+#define SCST_CONTR_MODE_ONE_TASK_SET 0
+#define SCST_CONTR_MODE_SEP_TASK_SETS 1
+
+/*******************************************************************
+ ** Values for the control mode page QUEUE ALGORITHM MODIFIER field
+ *******************************************************************/
+#define SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER 0
+#define SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER 1
+
+/*************************************************************
+ ** Values for the control mode page D_SENSE field
+ *************************************************************/
+#define SCST_CONTR_MODE_FIXED_SENSE 0
+#define SCST_CONTR_MODE_DESCR_SENSE 1
+
+/*************************************************************
+ ** Misc SCSI constants
+ *************************************************************/
+#define SCST_SENSE_ASC_UA_RESET 0x29
+#define READ_CAP_LEN 8
+#define READ_CAP16_LEN 32
+#define BYTCHK 0x02
+#define POSITION_LEN_SHORT 20
+#define POSITION_LEN_LONG 32
+
+/*************************************************************
+ ** Various timeouts
+ *************************************************************/
+#define SCST_DEFAULT_TIMEOUT (60 * HZ)
+
+#define SCST_GENERIC_CHANGER_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_CHANGER_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_PROCESSOR_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_PROCESSOR_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_TAPE_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_TAPE_REG_TIMEOUT (900 * HZ)
+#define SCST_GENERIC_TAPE_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_MODISK_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_MODISK_REG_TIMEOUT (900 * HZ)
+#define SCST_GENERIC_MODISK_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_DISK_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_DISK_REG_TIMEOUT (60 * HZ)
+#define SCST_GENERIC_DISK_LONG_TIMEOUT (3600 * HZ)
+
+#define SCST_GENERIC_RAID_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_RAID_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_GENERIC_CDROM_SMALL_TIMEOUT (3 * HZ)
+#define SCST_GENERIC_CDROM_REG_TIMEOUT (900 * HZ)
+#define SCST_GENERIC_CDROM_LONG_TIMEOUT (14000 * HZ)
+
+#define SCST_MAX_OTHER_TIMEOUT (14000 * HZ)
+
+/*************************************************************
+ ** I/O grouping attribute string values. Must match constants
+ ** w/o '_STR' suffix!
+ *************************************************************/
+#define SCST_IO_GROUPING_AUTO_STR "auto"
+#define SCST_IO_GROUPING_THIS_GROUP_ONLY_STR "this_group_only"
+#define SCST_IO_GROUPING_NEVER_STR "never"
+
+/*************************************************************
+ ** Threads pool type attribute string values.
+ ** Must match scst_dev_type_threads_pool_type!
+ *************************************************************/
+#define SCST_THREADS_POOL_PER_INITIATOR_STR "per_initiator"
+#define SCST_THREADS_POOL_SHARED_STR "shared"
+
+/*************************************************************
+ ** Misc constants
+ *************************************************************/
+#define SCST_SYSFS_BLOCK_SIZE PAGE_SIZE
+
+#define SCST_SYSFS_KEY_MARK "[key]"
+
+#define SCST_MIN_REL_TGT_ID 1
+#define SCST_MAX_REL_TGT_ID 65535
+
+#endif /* __SCST_CONST_H */
diff -uprN orig/linux-2.6.33/include/scst/scst.h linux-2.6.33/include/scst/scst.h
--- orig/linux-2.6.33/include/scst/scst.h
+++ linux-2.6.33/include/scst/scst.h
@@ -0,0 +1,3169 @@
+/*
+ * include/scst.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Main SCSI target mid-level include file.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_H
+#define __SCST_H
+
+#include <linux/types.h>
+#include <linux/version.h>
+#include <linux/blkdev.h>
+#include <linux/interrupt.h>
+#include <linux/wait.h>
+
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_eh.h>
+#include <scsi/scsi.h>
+
+#include <scst_const.h>
+
+#include "scst_sgv.h"
+
+/*
+ * Version numbers, the same as for the kernel.
+ *
+ * Changing it don't forget to change SCST_FIO_REV in scst_vdisk.c
+ * and FIO_REV in usr/fileio/common.h as well.
+ */
+#define SCST_VERSION(a, b, c, d) (((a) << 24) + ((b) << 16) + ((c) << 8) + d)
+#define SCST_VERSION_CODE SCST_VERSION(2, 0, 0, 0)
+#define SCST_VERSION_STRING_SUFFIX
+#define SCST_VERSION_STRING "2.0.0-pre1" SCST_VERSION_STRING_SUFFIX
+#define SCST_INTERFACE_VERSION \
+ SCST_VERSION_STRING "$Revision: 1603 $" SCST_CONST_VERSION
+
+#define SCST_LOCAL_NAME "scst_lcl_drvr"
+
+/*************************************************************
+ ** States of command processing state machine. At first,
+ ** "active" states, then - "passive" ones. This is to have
+ ** more efficient generated code of the corresponding
+ ** "switch" statements.
+ *************************************************************/
+
+/* Internal parsing */
+#define SCST_CMD_STATE_PRE_PARSE 0
+
+/* Dev handler's parse() is going to be called */
+#define SCST_CMD_STATE_DEV_PARSE 1
+
+/* Allocation of the cmd's data buffer */
+#define SCST_CMD_STATE_PREPARE_SPACE 2
+
+/* Calling preprocessing_done() */
+#define SCST_CMD_STATE_PREPROCESSING_DONE 3
+
+/* Target driver's rdy_to_xfer() is going to be called */
+#define SCST_CMD_STATE_RDY_TO_XFER 4
+
+/* Target driver's pre_exec() is going to be called */
+#define SCST_CMD_STATE_TGT_PRE_EXEC 5
+
+/* Cmd is going to be sent for execution */
+#define SCST_CMD_STATE_SEND_FOR_EXEC 6
+
+/* Cmd is being checked if it should be executed locally */
+#define SCST_CMD_STATE_LOCAL_EXEC 7
+
+/* Cmd is ready for execution */
+#define SCST_CMD_STATE_REAL_EXEC 8
+
+/* Internal post-exec checks */
+#define SCST_CMD_STATE_PRE_DEV_DONE 9
+
+/* Internal MODE SELECT pages related checks */
+#define SCST_CMD_STATE_MODE_SELECT_CHECKS 10
+
+/* Dev handler's dev_done() is going to be called */
+#define SCST_CMD_STATE_DEV_DONE 11
+
+/* Target driver's xmit_response() is going to be called */
+#define SCST_CMD_STATE_PRE_XMIT_RESP 12
+
+/* Target driver's xmit_response() is going to be called */
+#define SCST_CMD_STATE_XMIT_RESP 13
+
+/* Cmd finished */
+#define SCST_CMD_STATE_FINISHED 14
+
+/* Internal cmd finished */
+#define SCST_CMD_STATE_FINISHED_INTERNAL 15
+
+#define SCST_CMD_STATE_LAST_ACTIVE (SCST_CMD_STATE_FINISHED_INTERNAL+100)
+
+/* A cmd is created, but scst_cmd_init_done() not called */
+#define SCST_CMD_STATE_INIT_WAIT (SCST_CMD_STATE_LAST_ACTIVE+1)
+
+/* LUN translation (cmd->tgt_dev assignment) */
+#define SCST_CMD_STATE_INIT (SCST_CMD_STATE_LAST_ACTIVE+2)
+
+/* Waiting for scst_restart_cmd() */
+#define SCST_CMD_STATE_PREPROCESSING_DONE_CALLED (SCST_CMD_STATE_LAST_ACTIVE+3)
+
+/* Waiting for data from the initiator (until scst_rx_data() called) */
+#define SCST_CMD_STATE_DATA_WAIT (SCST_CMD_STATE_LAST_ACTIVE+4)
+
+/* Waiting for CDB's execution finish */
+#define SCST_CMD_STATE_REAL_EXECUTING (SCST_CMD_STATE_LAST_ACTIVE+5)
+
+/* Waiting for response's transmission finish */
+#define SCST_CMD_STATE_XMIT_WAIT (SCST_CMD_STATE_LAST_ACTIVE+6)
+
+/*************************************************************
+ * Can be retuned instead of cmd's state by dev handlers'
+ * functions, if the command's state should be set by default
+ *************************************************************/
+#define SCST_CMD_STATE_DEFAULT 500
+
+/*************************************************************
+ * Can be retuned instead of cmd's state by dev handlers'
+ * functions, if it is impossible to complete requested
+ * task in atomic context. The cmd will be restarted in thread
+ * context.
+ *************************************************************/
+#define SCST_CMD_STATE_NEED_THREAD_CTX 1000
+
+/*************************************************************
+ * Can be retuned instead of cmd's state by dev handlers'
+ * parse function, if the cmd processing should be stopped
+ * for now. The cmd will be restarted by dev handlers itself.
+ *************************************************************/
+#define SCST_CMD_STATE_STOP 1001
+
+/*************************************************************
+ ** States of mgmt command processing state machine
+ *************************************************************/
+
+/* LUN translation (mcmd->tgt_dev assignment) */
+#define SCST_MCMD_STATE_INIT 0
+
+/* Mgmt cmd is ready for processing */
+#define SCST_MCMD_STATE_READY 1
+
+/* Mgmt cmd is being executing */
+#define SCST_MCMD_STATE_EXECUTING 2
+
+/* Post check when affected commands done */
+#define SCST_MCMD_STATE_POST_AFFECTED_CMDS_DONE 3
+
+/* Target driver's task_mgmt_fn_done() is going to be called */
+#define SCST_MCMD_STATE_DONE 4
+
+/* The mcmd finished */
+#define SCST_MCMD_STATE_FINISHED 5
+
+/*************************************************************
+ ** Constants for "atomic" parameter of SCST's functions
+ *************************************************************/
+#define SCST_NON_ATOMIC 0
+#define SCST_ATOMIC 1
+
+/*************************************************************
+ ** Values for pref_context parameter of scst_cmd_init_done(),
+ ** scst_rx_data(), scst_restart_cmd(), scst_tgt_cmd_done()
+ ** and scst_cmd_done()
+ *************************************************************/
+
+enum scst_exec_context {
+ /*
+ * Direct cmd's processing (i.e. regular function calls in the current
+ * context) sleeping is not allowed
+ */
+ SCST_CONTEXT_DIRECT_ATOMIC,
+
+ /*
+ * Direct cmd's processing (i.e. regular function calls in the current
+ * context), sleeping is allowed, no restrictions
+ */
+ SCST_CONTEXT_DIRECT,
+
+ /* Tasklet or thread context required for cmd's processing */
+ SCST_CONTEXT_TASKLET,
+
+ /* Thread context required for cmd's processing */
+ SCST_CONTEXT_THREAD,
+
+ /*
+ * Context is the same as it was in previous call of the corresponding
+ * callback. For example, if dev handler's exec() does sync. data
+ * reading this value should be used for scst_cmd_done(). The same is
+ * true if scst_tgt_cmd_done() called directly from target driver's
+ * xmit_response(). Not allowed in scst_cmd_init_done() and
+ * scst_cmd_init_stage1_done().
+ */
+ SCST_CONTEXT_SAME
+};
+
+/*************************************************************
+ ** Values for status parameter of scst_rx_data()
+ *************************************************************/
+
+/* Success */
+#define SCST_RX_STATUS_SUCCESS 0
+
+/*
+ * Data receiving finished with error, so set the sense and
+ * finish the command, including xmit_response() call
+ */
+#define SCST_RX_STATUS_ERROR 1
+
+/*
+ * Data receiving finished with error and the sense is set,
+ * so finish the command, including xmit_response() call
+ */
+#define SCST_RX_STATUS_ERROR_SENSE_SET 2
+
+/*
+ * Data receiving finished with fatal error, so finish the command,
+ * but don't call xmit_response()
+ */
+#define SCST_RX_STATUS_ERROR_FATAL 3
+
+/*************************************************************
+ ** Values for status parameter of scst_restart_cmd()
+ *************************************************************/
+
+/* Success */
+#define SCST_PREPROCESS_STATUS_SUCCESS 0
+
+/*
+ * Command's processing finished with error, so set the sense and
+ * finish the command, including xmit_response() call
+ */
+#define SCST_PREPROCESS_STATUS_ERROR 1
+
+/*
+ * Command's processing finished with error and the sense is set,
+ * so finish the command, including xmit_response() call
+ */
+#define SCST_PREPROCESS_STATUS_ERROR_SENSE_SET 2
+
+/*
+ * Command's processing finished with fatal error, so finish the command,
+ * but don't call xmit_response()
+ */
+#define SCST_PREPROCESS_STATUS_ERROR_FATAL 3
+
+/* Thread context requested */
+#define SCST_PREPROCESS_STATUS_NEED_THREAD 4
+
+/*************************************************************
+ ** Values for AEN functions
+ *************************************************************/
+
+/*
+ * SCSI Asynchronous Event. Parameter contains SCSI sense
+ * (Unit Attention). AENs generated only for 2 the following UAs:
+ * CAPACITY DATA HAS CHANGED and REPORTED LUNS DATA HAS CHANGED.
+ * Other UAs reported regularly as CHECK CONDITION status,
+ * because it doesn't look safe to report them using AENs, since
+ * reporting using AENs opens delivery race windows even in case of
+ * untagged commands.
+ */
+#define SCST_AEN_SCSI 0
+
+/*************************************************************
+ ** Allowed return/status codes for report_aen() callback and
+ ** scst_set_aen_delivery_status() function
+ *************************************************************/
+
+/* Success */
+#define SCST_AEN_RES_SUCCESS 0
+
+/* Not supported */
+#define SCST_AEN_RES_NOT_SUPPORTED -1
+
+/* Failure */
+#define SCST_AEN_RES_FAILED -2
+
+/*************************************************************
+ ** Allowed return codes for xmit_response(), rdy_to_xfer()
+ *************************************************************/
+
+/* Success */
+#define SCST_TGT_RES_SUCCESS 0
+
+/* Internal device queue is full, retry again later */
+#define SCST_TGT_RES_QUEUE_FULL -1
+
+/*
+ * It is impossible to complete requested task in atomic context.
+ * The cmd will be restarted in thread context.
+ */
+#define SCST_TGT_RES_NEED_THREAD_CTX -2
+
+/*
+ * Fatal error, if returned by xmit_response() the cmd will
+ * be destroyed, if by any other function, xmit_response()
+ * will be called with HARDWARE ERROR sense data
+ */
+#define SCST_TGT_RES_FATAL_ERROR -3
+
+/*************************************************************
+ ** Allowed return codes for dev handler's exec()
+ *************************************************************/
+
+/* The cmd is done, go to other ones */
+#define SCST_EXEC_COMPLETED 0
+
+/* The cmd should be sent to SCSI mid-level */
+#define SCST_EXEC_NOT_COMPLETED 1
+
+/*
+ * Thread context is required to execute the command.
+ * Exec() will be called again in the thread context.
+ */
+#define SCST_EXEC_NEED_THREAD 2
+
+/*
+ * Set if cmd is finished and there is status/sense to be sent.
+ * The status should be not sent (i.e. the flag not set) if the
+ * possibility to perform a command in "chunks" (i.e. with multiple
+ * xmit_response()/rdy_to_xfer()) is used (not implemented yet).
+ * Obsolete, use scst_cmd_get_is_send_status() instead.
+ */
+#define SCST_TSC_FLAG_STATUS 0x2
+
+/*************************************************************
+ ** Additional return code for dev handler's task_mgmt_fn()
+ *************************************************************/
+
+/* Regular standard actions for the command should be done */
+#define SCST_DEV_TM_NOT_COMPLETED 1
+
+/*************************************************************
+ ** Session initialization phases
+ *************************************************************/
+
+/* Set if session is being initialized */
+#define SCST_SESS_IPH_INITING 0
+
+/* Set if the session is successfully initialized */
+#define SCST_SESS_IPH_SUCCESS 1
+
+/* Set if the session initialization failed */
+#define SCST_SESS_IPH_FAILED 2
+
+/* Set if session is initialized and ready */
+#define SCST_SESS_IPH_READY 3
+
+/*************************************************************
+ ** Session shutdown phases
+ *************************************************************/
+
+/* Set if session is initialized and ready */
+#define SCST_SESS_SPH_READY 0
+
+/* Set if session is shutting down */
+#define SCST_SESS_SPH_SHUTDOWN 1
+
+/*************************************************************
+ ** Session's async (atomic) flags
+ *************************************************************/
+
+/* Set if the sess's hw pending work is scheduled */
+#define SCST_SESS_HW_PENDING_WORK_SCHEDULED 0
+
+/*************************************************************
+ ** Cmd's async (atomic) flags
+ *************************************************************/
+
+/* Set if the cmd is aborted and ABORTED sense will be sent as the result */
+#define SCST_CMD_ABORTED 0
+
+/* Set if the cmd is aborted by other initiator */
+#define SCST_CMD_ABORTED_OTHER 1
+
+/* Set if no response should be sent to the target about this cmd */
+#define SCST_CMD_NO_RESP 2
+
+/* Set if the cmd is dead and can be destroyed at any time */
+#define SCST_CMD_CAN_BE_DESTROYED 3
+
+/*
+ * Set if the cmd's device has TAS flag set. Used only when aborted by
+ * other initiator.
+ */
+#define SCST_CMD_DEVICE_TAS 4
+
+/*************************************************************
+ ** Tgt_dev's async. flags (tgt_dev_flags)
+ *************************************************************/
+
+/* Set if tgt_dev has Unit Attention sense */
+#define SCST_TGT_DEV_UA_PENDING 0
+
+/* Set if tgt_dev is RESERVED by another session */
+#define SCST_TGT_DEV_RESERVED 1
+
+/* Set if the corresponding context is atomic */
+#define SCST_TGT_DEV_AFTER_INIT_WR_ATOMIC 5
+#define SCST_TGT_DEV_AFTER_INIT_OTH_ATOMIC 6
+#define SCST_TGT_DEV_AFTER_RESTART_WR_ATOMIC 7
+#define SCST_TGT_DEV_AFTER_RESTART_OTH_ATOMIC 8
+#define SCST_TGT_DEV_AFTER_RX_DATA_ATOMIC 9
+#define SCST_TGT_DEV_AFTER_EXEC_ATOMIC 10
+
+#define SCST_TGT_DEV_CLUST_POOL 11
+
+/*************************************************************
+ ** I/O groupping types. Changing them don't forget to change
+ ** the corresponding *_STR values in scst_const.h!
+ *************************************************************/
+
+/*
+ * All initiators with the same name connected to this group will have
+ * shared IO context, for each name own context. All initiators with
+ * different names will have own IO context.
+ */
+#define SCST_IO_GROUPING_AUTO 0
+
+/* All initiators connected to this group will have shared IO context */
+#define SCST_IO_GROUPING_THIS_GROUP_ONLY -1
+
+/* Each initiator connected to this group will have own IO context */
+#define SCST_IO_GROUPING_NEVER -2
+
+/*************************************************************
+ ** Activities suspending timeout
+ *************************************************************/
+#define SCST_SUSPENDING_TIMEOUT (90 * HZ)
+
+/*************************************************************
+ ** Kernel cache creation helper
+ *************************************************************/
+#ifndef KMEM_CACHE
+#define KMEM_CACHE(__struct, __flags) kmem_cache_create(#__struct,\
+ sizeof(struct __struct), __alignof__(struct __struct),\
+ (__flags), NULL, NULL)
+#endif
+
+/*************************************************************
+ ** Vlaid_mask constants for scst_analyze_sense()
+ *************************************************************/
+
+#define SCST_SENSE_KEY_VALID 1
+#define SCST_SENSE_ASC_VALID 2
+#define SCST_SENSE_ASCQ_VALID 4
+
+#define SCST_SENSE_ASCx_VALID (SCST_SENSE_ASC_VALID | \
+ SCST_SENSE_ASCQ_VALID)
+
+#define SCST_SENSE_ALL_VALID (SCST_SENSE_KEY_VALID | \
+ SCST_SENSE_ASC_VALID | \
+ SCST_SENSE_ASCQ_VALID)
+
+/*************************************************************
+ * TYPES
+ *************************************************************/
+
+struct scst_tgt;
+struct scst_session;
+struct scst_cmd;
+struct scst_mgmt_cmd;
+struct scst_device;
+struct scst_tgt_dev;
+struct scst_dev_type;
+struct scst_acg;
+struct scst_acg_dev;
+struct scst_acn;
+struct scst_aen;
+
+/*
+ * SCST uses 64-bit numbers to represent LUN's internally. The value
+ * NO_SUCH_LUN is guaranteed to be different of every valid LUN.
+ */
+#define NO_SUCH_LUN ((uint64_t)-1)
+
+typedef enum dma_data_direction scst_data_direction;
+
+/*
+ * SCST target template: defines target driver's parameters and callback
+ * functions.
+ *
+ * MUST HAVEs define functions that are expected to be defined in order to
+ * work. OPTIONAL says that there is a choice.
+ */
+struct scst_tgt_template {
+ /* public: */
+
+ /*
+ * SG tablesize allows to check whether scatter/gather can be used
+ * or not.
+ */
+ int sg_tablesize;
+
+ /*
+ * True, if this target adapter uses unchecked DMA onto an ISA bus.
+ */
+ unsigned unchecked_isa_dma:1;
+
+ /*
+ * True, if this target adapter can benefit from using SG-vector
+ * clustering (i.e. smaller number of segments).
+ */
+ unsigned use_clustering:1;
+
+ /*
+ * True, if this target adapter doesn't support SG-vector clustering
+ */
+ unsigned no_clustering:1;
+
+ /*
+ * True, if corresponding function supports execution in
+ * the atomic (non-sleeping) context
+ */
+ unsigned xmit_response_atomic:1;
+ unsigned rdy_to_xfer_atomic:1;
+
+ /*
+ * The maximum time in seconds cmd can stay inside the target
+ * hardware, i.e. after rdy_to_xfer() and xmit_response(), before
+ * on_hw_pending_cmd_timeout() will be called, if defined.
+ *
+ * In the current implementation a cmd will be aborted in time t
+ * max_hw_pending_time <= t < 2*max_hw_pending_time.
+ */
+ int max_hw_pending_time;
+
+ /*
+ * This function is equivalent to the SCSI
+ * queuecommand. The target should transmit the response
+ * buffer and the status in the scst_cmd struct.
+ * The expectation is that this executing this command is NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * After the response is actually transmitted, the target
+ * should call the scst_tgt_cmd_done() function of the
+ * mid-level, which will allow it to free up the command.
+ * Returns one of the SCST_TGT_RES_* constants.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * MUST HAVE
+ */
+ int (*xmit_response) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that data
+ * buffer corresponding to the said command have now been
+ * allocated and it is OK to receive data for this command.
+ * This function is necessary because a SCSI target does not
+ * have any control over the commands it receives. Most lower
+ * level protocols have a corresponding function which informs
+ * the initiator that buffers have been allocated e.g., XFER_
+ * RDY in Fibre Channel. After the data is actually received
+ * the low-level driver needs to call scst_rx_data() in order to
+ * continue processing this command.
+ * Returns one of the SCST_TGT_RES_* constants.
+ *
+ * This command is expected to be NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * OPTIONAL
+ */
+ int (*rdy_to_xfer) (struct scst_cmd *cmd);
+
+ /*
+ * Called if cmd stays inside the target hardware, i.e. after
+ * rdy_to_xfer() and xmit_response(), more than max_hw_pending_time
+ * time. The target driver supposed to cleanup this command and
+ * resume cmd's processing.
+ *
+ * OPTIONAL
+ */
+ void (*on_hw_pending_cmd_timeout) (struct scst_cmd *cmd);
+
+ /*
+ * Called to notify the driver that the command is about to be freed.
+ * Necessary, because for aborted commands xmit_response() could not
+ * be called. Could be called on IRQ context.
+ *
+ * OPTIONAL
+ */
+ void (*on_free_cmd) (struct scst_cmd *cmd);
+
+ /*
+ * This function allows target driver to handle data buffer
+ * allocations on its own.
+ *
+ * Target driver doesn't have to always allocate buffer in this
+ * function, but if it decide to do it, it must check that
+ * scst_cmd_get_data_buff_alloced() returns 0, otherwise to avoid
+ * double buffer allocation and memory leaks alloc_data_buf() shall
+ * fail.
+ *
+ * Shall return 0 in case of success or < 0 (preferrably -ENOMEM)
+ * in case of error, or > 0 if the regular SCST allocation should be
+ * done. In case of returning successfully,
+ * scst_cmd->tgt_data_buf_alloced will be set by SCST.
+ *
+ * It is possible that both target driver and dev handler request own
+ * memory allocation. In this case, data will be memcpy() between
+ * buffers, where necessary.
+ *
+ * If allocation in atomic context - cf. scst_cmd_atomic() - is not
+ * desired or fails and consequently < 0 is returned, this function
+ * will be re-called in thread context.
+ *
+ * Please note that the driver will have to handle itself all relevant
+ * details such as scatterlist setup, highmem, freeing the allocated
+ * memory, etc.
+ *
+ * OPTIONAL.
+ */
+ int (*alloc_data_buf) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that data
+ * buffer corresponding to the said command have now been
+ * allocated and other preprocessing tasks have been done.
+ * A target driver could need to do some actions at this stage.
+ * After the target driver done the needed actions, it shall call
+ * scst_restart_cmd() in order to continue processing this command.
+ * In case of preliminary the command completion, this function will
+ * also be called before xmit_response().
+ *
+ * Called only if the cmd is queued using scst_cmd_init_stage1_done()
+ * instead of scst_cmd_init_done().
+ *
+ * Returns void, the result is expected to be returned using
+ * scst_restart_cmd().
+ *
+ * This command is expected to be NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * OPTIONAL.
+ */
+ void (*preprocessing_done) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that the said command is about
+ * to be executed.
+ *
+ * Returns one of the SCST_PREPROCESS_* constants.
+ *
+ * This command is expected to be NON-BLOCKING.
+ * If it is blocking, consider to set threads_num to some none 0 number.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * OPTIONAL
+ */
+ int (*pre_exec) (struct scst_cmd *cmd);
+
+ /*
+ * This function informs the driver that all affected by the
+ * corresponding task management function commands have beed completed.
+ * No return value expected.
+ *
+ * This function is expected to be NON-BLOCKING.
+ *
+ * Called without any locks held from a thread context.
+ *
+ * OPTIONAL
+ */
+ void (*task_mgmt_affected_cmds_done) (struct scst_mgmt_cmd *mgmt_cmd);
+
+ /*
+ * This function informs the driver that the corresponding task
+ * management function has been completed, i.e. all the corresponding
+ * commands completed and freed. No return value expected.
+ *
+ * This function is expected to be NON-BLOCKING.
+ *
+ * Called without any locks held from a thread context.
+ *
+ * MUST HAVE if the target supports task management.
+ */
+ void (*task_mgmt_fn_done) (struct scst_mgmt_cmd *mgmt_cmd);
+
+ /*
+ * This function should detect the target adapters that
+ * are present in the system. The function should return a value
+ * >= 0 to signify the number of detected target adapters.
+ * A negative value should be returned whenever there is
+ * an error.
+ *
+ * MUST HAVE
+ */
+ int (*detect) (struct scst_tgt_template *tgt_template);
+
+ /*
+ * This function should free up the resources allocated to the device.
+ * The function should return 0 to indicate successful release
+ * or a negative value if there are some issues with the release.
+ * In the current version the return value is ignored.
+ *
+ * MUST HAVE
+ */
+ int (*release) (struct scst_tgt *tgt);
+
+ /*
+ * This function is used for Asynchronous Event Notifications.
+ *
+ * Returns one of the SCST_AEN_RES_* constants.
+ * After AEN is sent, target driver must call scst_aen_done() and,
+ * optionally, scst_set_aen_delivery_status().
+ *
+ * This command is expected to be NON-BLOCKING, but can sleep.
+ *
+ * MUST HAVE, if low-level protocol supports AENs.
+ */
+ int (*report_aen) (struct scst_aen *aen);
+
+ /*
+ * This function allows to enable or disable particular target.
+ * A disabled target doesn't receive and process any SCSI commands.
+ *
+ * SHOULD HAVE to avoid race when there are connected initiators,
+ * while target not yet completed the initial configuration. In this
+ * case the too early connected initiators would see not those devices,
+ * which they intended to see.
+ */
+ int (*enable_target) (struct scst_tgt *tgt, bool enable);
+
+ /*
+ * This function shows if particular target is enabled or not.
+ *
+ * SHOULD HAVE, see above why.
+ */
+ bool (*is_target_enabled) (struct scst_tgt *tgt);
+
+ /*
+ * This function adds a virtual target.
+ *
+ * If both add_target and del_target callbacks defined, then this
+ * target driver supposed to support virtual targets. In this case
+ * an "mgmt" entry will be created in the sysfs root for this driver.
+ * The "mgmt" entry will support 2 commands: "add_target" and
+ * "del_target", for which the corresponding callbacks will be called.
+ * Also target driver can define own commands for the "mgmt" entry, see
+ * mgmt_cmd and mgmt_cmd_help below.
+ *
+ * This approach allows uniform targets management to simplify external
+ * management tools like scstadmin. See README for more details.
+ *
+ * Either both add_target and del_target must be defined, or none.
+ *
+ * MUST HAVE if virtual targets are supported.
+ */
+ ssize_t (*add_target) (const char *target_name, char *params);
+
+ /*
+ * This function deletes a virtual target. See comment for add_target
+ * above.
+ *
+ * MUST HAVE if virtual targets are supported.
+ */
+ ssize_t (*del_target) (const char *target_name);
+
+ /*
+ * This function called if not "add_target" or "del_target" command is
+ * sent to the mgmt entry (see comment for add_target above). In this
+ * case the command passed to this function as is in a string form.
+ *
+ * OPTIONAL.
+ */
+ ssize_t (*mgmt_cmd) (char *cmd);
+
+ /*
+ * Name of the template. Must be unique to identify
+ * the template. MUST HAVE
+ */
+ const char name[SCST_MAX_NAME];
+
+ /*
+ * Number of additional threads to the pool of dedicated threads.
+ * Used if xmit_response() or rdy_to_xfer() is blocking.
+ * It is the target driver's duty to ensure that not more, than that
+ * number of threads, are blocked in those functions at any time.
+ */
+ int threads_num;
+
+ /* Optional default log flags */
+ const unsigned long default_trace_flags;
+
+ /* Optional pointer to trace flags */
+ unsigned long *trace_flags;
+
+ /* Optional local trace table */
+ struct scst_trace_log *trace_tbl;
+
+ /* Optional local trace table help string */
+ const char *trace_tbl_help;
+
+ /* Optional sysfs attributes */
+ const struct attribute **tgtt_attrs;
+
+ /* Optional sysfs target attributes */
+ const struct attribute **tgt_attrs;
+
+ /* Optional sysfs session attributes */
+ const struct attribute **sess_attrs;
+
+ /* Optional help string for mgmt_cmd commands */
+ const char *mgmt_cmd_help;
+
+ /* Optional help string for add_target parameters */
+ const char *add_target_parameters_help;
+
+ /** Private, must be inited to 0 by memset() **/
+
+ /* List of targets per template, protected by scst_mutex */
+ struct list_head tgt_list;
+
+ /* List entry of global templates list */
+ struct list_head scst_template_list_entry;
+
+ /* Set if tgtt_kobj was initialized */
+ unsigned int tgtt_kobj_initialized:1;
+
+ struct kobject tgtt_kobj; /* kobject for this struct */
+
+ struct completion tgtt_kobj_release_cmpl;
+
+};
+
+/*
+ * Threads pool types. Changing them don't forget to change
+ * the corresponding *_STR values in scst_const.h!
+ */
+enum scst_dev_type_threads_pool_type {
+ /* Each initiator will have dedicated threads pool. */
+ SCST_THREADS_POOL_PER_INITIATOR = 0,
+
+ /* All connected initiators will use shared threads pool */
+ SCST_THREADS_POOL_SHARED,
+
+ /* Invalid value for scst_parse_threads_pool_type() */
+ SCST_THREADS_POOL_TYPE_INVALID,
+};
+
+/*
+ * SCST dev handler template: defines dev handler's parameters and callback
+ * functions.
+ *
+ * MUST HAVEs define functions that are expected to be defined in order to
+ * work. OPTIONAL says that there is a choice.
+ */
+struct scst_dev_type {
+ /* SCSI type of the supported device. MUST HAVE */
+ int type;
+
+ /* True, if this dev handler is a pass-through dev handler */
+ unsigned pass_through:1;
+
+ /*
+ * True, if corresponding function supports execution in
+ * the atomic (non-sleeping) context
+ */
+ unsigned parse_atomic:1;
+ unsigned exec_atomic:1;
+ unsigned dev_done_atomic:1;
+
+ /*
+ * Should be true, if exec() is synchronous. This is a hint to SCST core
+ * to optimize commands order management.
+ */
+ unsigned exec_sync:1;
+
+ /*
+ * Called to parse CDB from the cmd and initialize
+ * cmd->bufflen and cmd->data_direction (both - REQUIRED).
+ * Returns the command's next state or SCST_CMD_STATE_DEFAULT,
+ * if the next default state should be used, or
+ * SCST_CMD_STATE_NEED_THREAD_CTX if the function called in atomic
+ * context, but requires sleeping, or SCST_CMD_STATE_STOP if the
+ * command should not be further processed for now. In the
+ * SCST_CMD_STATE_NEED_THREAD_CTX case the function
+ * will be recalled in the thread context, where sleeping is allowed.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * MUST HAVE
+ */
+ int (*parse) (struct scst_cmd *cmd);
+
+ /*
+ * Called to execute CDB. Useful, for instance, to implement
+ * data caching. The result of CDB execution is reported via
+ * cmd->scst_cmd_done() callback.
+ * Returns:
+ * - SCST_EXEC_COMPLETED - the cmd is done, go to other ones
+ * - SCST_EXEC_NEED_THREAD - thread context is required to execute
+ * the command. Exec() will be called again in the thread context.
+ * - SCST_EXEC_NOT_COMPLETED - the cmd should be sent to SCSI
+ * mid-level.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ *
+ * If this function provides sync execution, you should set
+ * exec_sync flag and consider to setup dedicated threads by
+ * setting threads_num > 0.
+ *
+ * !! If this function is implemented, scst_check_local_events() !!
+ * !! shall be called inside it just before the actual command's !!
+ * !! execution. !!
+ *
+ * OPTIONAL, if not set, the commands will be sent directly to SCSI
+ * device.
+ */
+ int (*exec) (struct scst_cmd *cmd);
+
+ /*
+ * Called to notify dev handler about the result of cmd execution
+ * and perform some post processing. Cmd's fields is_send_status and
+ * resp_data_len should be set by this function, but SCST offers good
+ * defaults.
+ * Returns the command's next state or SCST_CMD_STATE_DEFAULT,
+ * if the next default state should be used, or
+ * SCST_CMD_STATE_NEED_THREAD_CTX if the function called in atomic
+ * context, but requires sleeping. In the last case, the function
+ * will be recalled in the thread context, where sleeping is allowed.
+ *
+ * Pay attention to "atomic" attribute of the cmd, which can be get
+ * by scst_cmd_atomic(): it is true if the function called in the
+ * atomic (non-sleeping) context.
+ */
+ int (*dev_done) (struct scst_cmd *cmd);
+
+ /*
+ * Called to notify dev hander that the command is about to be freed.
+ * Could be called on IRQ context.
+ */
+ void (*on_free_cmd) (struct scst_cmd *cmd);
+
+ /*
+ * Called to execute a task management command.
+ * Returns:
+ * - SCST_MGMT_STATUS_SUCCESS - the command is done with success,
+ * no firther actions required
+ * - The SCST_MGMT_STATUS_* error code if the command is failed and
+ * no further actions required
+ * - SCST_DEV_TM_NOT_COMPLETED - regular standard actions for the
+ * command should be done
+ *
+ * Called without any locks held from a thread context.
+ */
+ int (*task_mgmt_fn) (struct scst_mgmt_cmd *mgmt_cmd,
+ struct scst_tgt_dev *tgt_dev);
+
+ /*
+ * Called when new device is attaching to the dev handler
+ * Returns 0 on success, error code otherwise.
+ */
+ int (*attach) (struct scst_device *dev);
+
+ /* Called when new device is detaching from the dev handler */
+ void (*detach) (struct scst_device *dev);
+
+ /*
+ * Called when new tgt_dev (session) is attaching to the dev handler.
+ * Returns 0 on success, error code otherwise.
+ */
+ int (*attach_tgt) (struct scst_tgt_dev *tgt_dev);
+
+ /* Called when tgt_dev (session) is detaching from the dev handler */
+ void (*detach_tgt) (struct scst_tgt_dev *tgt_dev);
+
+ /*
+ * This function adds a virtual device.
+ *
+ * If both add_device and del_device callbacks defined, then this
+ * dev handler supposed to support adding/deleting virtual devices.
+ * In this case an "mgmt" entry will be created in the sysfs root for
+ * this handler. The "mgmt" entry will support 2 commands: "add_device"
+ * and "del_device", for which the corresponding callbacks will be called.
+ * Also dev handler can define own commands for the "mgmt" entry, see
+ * mgmt_cmd and mgmt_cmd_help below.
+ *
+ * This approach allows uniform devices management to simplify external
+ * management tools like scstadmin. See README for more details.
+ *
+ * Either both add_device and del_device must be defined, or none.
+ *
+ * MUST HAVE if virtual devices are supported.
+ */
+ ssize_t (*add_device) (const char *device_name, char *params);
+
+ /*
+ * This function deletes a virtual device. See comment for add_device
+ * above.
+ *
+ * MUST HAVE if virtual devices are supported.
+ */
+ ssize_t (*del_device) (const char *device_name);
+
+ /*
+ * This function called if not "add_device" or "del_device" command is
+ * sent to the mgmt entry (see comment for add_device above). In this
+ * case the command passed to this function as is in a string form.
+ *
+ * OPTIONAL.
+ */
+ ssize_t (*mgmt_cmd) (char *cmd);
+
+ /*
+ * Name of the dev handler. Must be unique. MUST HAVE.
+ *
+ * It's SCST_MAX_NAME + few more bytes to match scst_user expectations.
+ */
+ char name[SCST_MAX_NAME + 10];
+
+ /*
+ * Number of threads in this handler's devices' threads pools.
+ * If 0 - no threads will be created, if <0 - creation of the threads
+ * pools is prohibited. Also pay attention to threads_pool_type below.
+ */
+ int threads_num;
+
+ /* Threads pool type. Valid only if threads_num > 0. */
+ enum scst_dev_type_threads_pool_type threads_pool_type;
+
+ /* Optional default log flags */
+ const unsigned long default_trace_flags;
+
+ /* Optional pointer to trace flags */
+ unsigned long *trace_flags;
+
+ /* Optional local trace table */
+ struct scst_trace_log *trace_tbl;
+
+ /* Optional local trace table help string */
+ const char *trace_tbl_help;
+
+ /* Optional help string for mgmt_cmd commands */
+ const char *mgmt_cmd_help;
+
+ /* Optional help string for add_device parameters */
+ const char *add_device_parameters_help;
+
+ /* Optional sysfs attributes */
+ const struct attribute **devt_attrs;
+
+ /* Optional sysfs device attributes */
+ const struct attribute **dev_attrs;
+
+ /* Pointer to dev handler's private data */
+ void *devt_priv;
+
+ /* Pointer to parent dev type in the sysfs hierarchy */
+ struct scst_dev_type *parent;
+
+ struct module *module;
+
+ /** Private, must be inited to 0 by memset() **/
+
+ /* list entry in scst_dev_type_list */
+ struct list_head dev_type_list_entry;
+
+ unsigned int devt_kobj_initialized:1;
+
+ struct kobject devt_kobj; /* main handlers/driver */
+
+ /* To wait until devt_kobj released */
+ struct completion devt_kobj_release_compl;
+};
+
+/*
+ * An SCST target, analog of SCSI target port.
+ */
+struct scst_tgt {
+ /* List of remote sessions per target, protected by scst_mutex */
+ struct list_head sess_list;
+
+ /* List entry of targets per template (tgts_list) */
+ struct list_head tgt_list_entry;
+
+ struct scst_tgt_template *tgtt; /* corresponding target template */
+
+ struct scst_acg *default_acg; /* The default acg for this target. */
+
+ /*
+ * Device ACG groups
+ */
+ struct list_head tgt_acg_list;
+
+ /*
+ * Maximum SG table size. Needed here, since different cards on the
+ * same target template can have different SG table limitations.
+ */
+ int sg_tablesize;
+
+ /* Used for storage of target driver private stuff */
+ void *tgt_priv;
+
+ /*
+ * The following fields used to store and retry cmds if target's
+ * internal queue is full, so the target is unable to accept
+ * the cmd returning QUEUE FULL.
+ * They protected by tgt_lock, where necessary.
+ */
+ bool retry_timer_active;
+ struct timer_list retry_timer;
+ atomic_t finished_cmds;
+ int retry_cmds;
+ spinlock_t tgt_lock;
+ struct list_head retry_cmd_list;
+
+ /* Used to wait until session finished to unregister */
+ wait_queue_head_t unreg_waitQ;
+
+ /* Name of the target */
+ char *tgt_name;
+
+ uint16_t rel_tgt_id;
+
+ /* Set if tgt_kobj was initialized */
+ unsigned int tgt_kobj_initialized:1;
+
+ /* Set if scst_tgt_sysfs_prepare_put() was called for tgt_kobj */
+ unsigned int tgt_kobj_put_prepared:1;
+
+ /*
+ * Used to protect sysfs attributes to be called after this
+ * object was unregistered.
+ */
+ struct rw_semaphore tgt_attr_rwsem;
+
+ struct kobject tgt_kobj; /* main targets/target kobject */
+ struct kobject *tgt_sess_kobj; /* target/sessions/ */
+ struct kobject *tgt_luns_kobj; /* target/luns/ */
+ struct kobject *tgt_ini_grp_kobj; /* target/ini_groups/ */
+};
+
+/* Hash size and hash fn for hash based lun translation */
+#define TGT_DEV_HASH_SHIFT 5
+#define TGT_DEV_HASH_SIZE (1 << TGT_DEV_HASH_SHIFT)
+#define HASH_VAL(_val) (_val & (TGT_DEV_HASH_SIZE - 1))
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+/* Defines extended latency statistics */
+struct scst_ext_latency_stat {
+ uint64_t scst_time_rd, tgt_time_rd, dev_time_rd;
+ unsigned int processed_cmds_rd;
+ uint64_t min_scst_time_rd, min_tgt_time_rd, min_dev_time_rd;
+ uint64_t max_scst_time_rd, max_tgt_time_rd, max_dev_time_rd;
+
+ uint64_t scst_time_wr, tgt_time_wr, dev_time_wr;
+ unsigned int processed_cmds_wr;
+ uint64_t min_scst_time_wr, min_tgt_time_wr, min_dev_time_wr;
+ uint64_t max_scst_time_wr, max_tgt_time_wr, max_dev_time_wr;
+};
+
+#define SCST_IO_SIZE_THRESHOLD_SMALL (8*1024)
+#define SCST_IO_SIZE_THRESHOLD_MEDIUM (32*1024)
+#define SCST_IO_SIZE_THRESHOLD_LARGE (128*1024)
+#define SCST_IO_SIZE_THRESHOLD_VERY_LARGE (512*1024)
+
+#define SCST_LATENCY_STAT_INDEX_SMALL 0
+#define SCST_LATENCY_STAT_INDEX_MEDIUM 1
+#define SCST_LATENCY_STAT_INDEX_LARGE 2
+#define SCST_LATENCY_STAT_INDEX_VERY_LARGE 3
+#define SCST_LATENCY_STAT_INDEX_OTHER 4
+#define SCST_LATENCY_STATS_NUM (SCST_LATENCY_STAT_INDEX_OTHER + 1)
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
+
+/*
+ * SCST session, analog of SCSI I_T nexus
+ */
+struct scst_session {
+ /*
+ * Initialization phase, one of SCST_SESS_IPH_* constants, protected by
+ * sess_list_lock
+ */
+ int init_phase;
+
+ struct scst_tgt *tgt; /* corresponding target */
+
+ /* Used for storage of target driver private stuff */
+ void *tgt_priv;
+
+ unsigned long sess_aflags; /* session's async flags */
+
+ /*
+ * Hash list of tgt_dev's for this session, protected by scst_mutex
+ * and suspended activity
+ */
+ struct list_head sess_tgt_dev_list_hash[TGT_DEV_HASH_SIZE];
+
+ /*
+ * List of cmds in this session. Protected by sess_list_lock.
+ *
+ * We must always keep commands in the sess list from the
+ * very beginning, because otherwise they can be missed during
+ * TM processing.
+ */
+ struct list_head sess_cmd_list;
+
+ spinlock_t sess_list_lock; /* protects sess_cmd_list, etc */
+
+ atomic_t refcnt; /* get/put counter */
+
+ /*
+ * Alive commands for this session. ToDo: make it part of the common
+ * IO flow control.
+ */
+ atomic_t sess_cmd_count;
+
+ /* Access control for this session and list entry there */
+ struct scst_acg *acg;
+
+ /* List entry for the sessions list inside ACG */
+ struct list_head acg_sess_list_entry;
+
+ struct delayed_work hw_pending_work;
+
+ /* Name of attached initiator */
+ const char *initiator_name;
+
+ /* List entry of sessions per target */
+ struct list_head sess_list_entry;
+
+ /* List entry for the list that keeps session, waiting for the init */
+ struct list_head sess_init_list_entry;
+
+ /*
+ * List entry for the list that keeps session, waiting for the shutdown
+ */
+ struct list_head sess_shut_list_entry;
+
+ /*
+ * Lists of deferred during session initialization commands.
+ * Protected by sess_list_lock.
+ */
+ struct list_head init_deferred_cmd_list;
+ struct list_head init_deferred_mcmd_list;
+
+ /*
+ * Shutdown phase, one of SCST_SESS_SPH_* constants, unprotected.
+ * Async. relating to init_phase, must be a separate variable, because
+ * session could be unregistered before async. registration is finished.
+ */
+ unsigned long shut_phase;
+
+ /* Used if scst_unregister_session() called in wait mode */
+ struct completion *shutdown_compl;
+
+ /* Set if sess_kobj was initialized */
+ unsigned int sess_kobj_initialized:1;
+
+ /*
+ * Used to protect sysfs attributes to be called after this
+ * object was unregistered.
+ */
+ struct rw_semaphore sess_attr_rwsem;
+
+ struct kobject sess_kobj; /* kobject for this struct */
+
+ /*
+ * Functions and data for user callbacks from scst_register_session()
+ * and scst_unregister_session()
+ */
+ void *reg_sess_data;
+ void (*init_result_fn) (struct scst_session *sess, void *data,
+ int result);
+ void (*unreg_done_fn) (struct scst_session *sess);
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ /*
+ * Must be the last to allow to work with drivers who don't know
+ * about this config time option.
+ */
+ spinlock_t lat_lock;
+ uint64_t scst_time, tgt_time, dev_time;
+ unsigned int processed_cmds;
+ uint64_t min_scst_time, min_tgt_time, min_dev_time;
+ uint64_t max_scst_time, max_tgt_time, max_dev_time;
+ struct scst_ext_latency_stat sess_latency_stat[SCST_LATENCY_STATS_NUM];
+#endif
+};
+
+/*
+ * Structure to control commands' queuing and threads pool processing the queue
+ */
+struct scst_cmd_threads {
+ spinlock_t cmd_list_lock;
+ struct list_head active_cmd_list; /* commands queue */
+ wait_queue_head_t cmd_list_waitQ;
+
+ struct io_context *io_context; /* IO context of the threads pool */
+
+ int nr_threads; /* number of processing threads */
+ struct list_head threads_list; /* processing threads */
+
+ struct list_head lists_list_entry;
+};
+
+/*
+ * SCST command, analog of I_T_L_Q nexus or task
+ */
+struct scst_cmd {
+ /* List entry for below *_cmd_threads */
+ struct list_head cmd_list_entry;
+
+ /* Pointer to lists of commands with the lock */
+ struct scst_cmd_threads *cmd_threads;
+
+ atomic_t cmd_ref;
+
+ struct scst_session *sess; /* corresponding session */
+
+ /* Cmd state, one of SCST_CMD_STATE_* constants */
+ int state;
+
+ /*************************************************************
+ ** Cmd's flags
+ *************************************************************/
+
+ /*
+ * Set if expected_sn should be incremented, i.e. cmd was sent
+ * for execution
+ */
+ unsigned int sent_for_exec:1;
+
+ /* Set if the cmd's action is completed */
+ unsigned int completed:1;
+
+ /* Set if we should ignore Unit Attention in scst_check_sense() */
+ unsigned int ua_ignore:1;
+
+ /* Set if cmd is being processed in atomic context */
+ unsigned int atomic:1;
+
+ /* Set if this command was sent in double UA possible state */
+ unsigned int double_ua_possible:1;
+
+ /* Set if this command contains status */
+ unsigned int is_send_status:1;
+
+ /* Set if cmd is being retried */
+ unsigned int retry:1;
+
+ /* Set if cmd is internally generated */
+ unsigned int internal:1;
+
+ /* Set if the device was blocked by scst_inc_on_dev_cmd() (for debug) */
+ unsigned int inc_blocking:1;
+
+ /* Set if the device should be unblocked after cmd's finish */
+ unsigned int needs_unblocking:1;
+
+ /* Set if scst_dec_on_dev_cmd() call is needed on the cmd's finish */
+ unsigned int dec_on_dev_needed:1;
+
+ /* Set if cmd is queued as hw pending */
+ unsigned int cmd_hw_pending:1;
+
+ /*
+ * Set if the target driver wants to alloc data buffers on its own.
+ * In this case alloc_data_buf() must be provided in the target driver
+ * template.
+ */
+ unsigned int tgt_need_alloc_data_buf:1;
+
+ /*
+ * Set by SCST if the custom data buffer allocation by the target driver
+ * succeeded.
+ */
+ unsigned int tgt_data_buf_alloced:1;
+
+ /* Set if custom data buffer allocated by dev handler */
+ unsigned int dh_data_buf_alloced:1;
+
+ /* Set if the target driver called scst_set_expected() */
+ unsigned int expected_values_set:1;
+
+ /*
+ * Set if the SG buffer was modified by scst_set_resp_data_len()
+ */
+ unsigned int sg_buff_modified:1;
+
+ /*
+ * Set if scst_cmd_init_stage1_done() called and the target
+ * want that preprocessing_done() will be called
+ */
+ unsigned int preprocessing_only:1;
+
+ /* Set if cmd's SN was set */
+ unsigned int sn_set:1;
+
+ /* Set if hq_cmd_count was incremented */
+ unsigned int hq_cmd_inced:1;
+
+ /*
+ * Set if scst_cmd_init_stage1_done() called and the target wants
+ * that the SN for the cmd won't be assigned until scst_restart_cmd()
+ */
+ unsigned int set_sn_on_restart_cmd:1;
+
+ /* Set if the cmd's must not use sgv cache for data buffer */
+ unsigned int no_sgv:1;
+
+ /*
+ * Set if target driver may need to call dma_sync_sg() or similar
+ * function before transferring cmd' data to the target device
+ * via DMA.
+ */
+ unsigned int may_need_dma_sync:1;
+
+ /* Set if the cmd was done or aborted out of its SN */
+ unsigned int out_of_sn:1;
+
+ /* Set if increment expected_sn in cmd->scst_cmd_done() */
+ unsigned int inc_expected_sn_on_done:1;
+
+ /* Set if tgt_sn field is valid */
+ unsigned int tgt_sn_set:1;
+
+ /* Set if cmd is done */
+ unsigned int done:1;
+
+ /* Set if cmd is finished */
+ unsigned int finished:1;
+
+ /*
+ * Set if the cmd was delayed by task management debugging code.
+ * Used only if CONFIG_SCST_DEBUG_TM is on.
+ */
+ unsigned int tm_dbg_delayed:1;
+
+ /*
+ * Set if the cmd must be ignored by task management debugging code.
+ * Used only if CONFIG_SCST_DEBUG_TM is on.
+ */
+ unsigned int tm_dbg_immut:1;
+
+ /**************************************************************/
+
+ unsigned long cmd_flags; /* cmd's async flags */
+
+ /* Keeps status of cmd's status/data delivery to remote initiator */
+ int delivery_status;
+
+ struct scst_tgt_template *tgtt; /* to save extra dereferences */
+ struct scst_tgt *tgt; /* to save extra dereferences */
+ struct scst_device *dev; /* to save extra dereferences */
+
+ struct scst_tgt_dev *tgt_dev; /* corresponding device for this cmd */
+
+ uint64_t lun; /* LUN for this cmd */
+
+ unsigned long start_time;
+
+ /* List entry for tgt_dev's SN related lists */
+ struct list_head sn_cmd_list_entry;
+
+ /* Cmd's serial number, used to execute cmd's in order of arrival */
+ unsigned int sn;
+
+ /* The corresponding sn_slot in tgt_dev->sn_slots */
+ atomic_t *sn_slot;
+
+ /* List entry for sess's sess_cmd_list */
+ struct list_head sess_cmd_list_entry;
+
+ /*
+ * Used to found the cmd by scst_find_cmd_by_tag(). Set by the
+ * target driver on the cmd's initialization time
+ */
+ uint64_t tag;
+
+ uint32_t tgt_sn; /* SN set by target driver (for TM purposes) */
+
+ /* CDB and its len */
+ uint8_t cdb[SCST_MAX_CDB_SIZE];
+ short cdb_len; /* it might be -1 */
+ unsigned short ext_cdb_len;
+ uint8_t *ext_cdb;
+
+ enum scst_cdb_flags op_flags;
+ const char *op_name;
+
+ enum scst_cmd_queue_type queue_type;
+
+ int timeout; /* CDB execution timeout in seconds */
+ int retries; /* Amount of retries that will be done by SCSI mid-level */
+
+ /* SCSI data direction, one of SCST_DATA_* constants */
+ scst_data_direction data_direction;
+
+ /* Remote initiator supplied values, if any */
+ scst_data_direction expected_data_direction;
+ int expected_transfer_len;
+ int expected_in_transfer_len; /* for bidi writes */
+
+ /*
+ * Cmd data length. Could be different from bufflen for commands like
+ * VERIFY, which transfer different amount of data (if any), than
+ * processed.
+ */
+ int data_len;
+
+ /* Completition routine */
+ void (*scst_cmd_done) (struct scst_cmd *cmd, int next_state,
+ enum scst_exec_context pref_context);
+
+ struct sgv_pool_obj *sgv; /* sgv object */
+ int bufflen; /* cmd buffer length */
+ struct scatterlist *sg; /* cmd data buffer SG vector */
+ int sg_cnt; /* SG segments count */
+
+ /*
+ * Response data length in data buffer. This field must not be set
+ * directly, use scst_set_resp_data_len() for that
+ */
+ int resp_data_len;
+
+ /* scst_get_sg_buf_[first,next]() support */
+ int get_sg_buf_entry_num;
+
+ /* Bidirectional transfers support */
+ int in_bufflen; /* WRITE buffer length */
+ struct sgv_pool_obj *in_sgv; /* WRITE sgv object */
+ struct scatterlist *in_sg; /* WRITE data buffer SG vector */
+ int in_sg_cnt; /* WRITE SG segments count */
+
+ /*
+ * Used if both target driver and dev handler request own memory
+ * allocation. In other cases, both are equal to sg and sg_cnt
+ * correspondingly.
+ *
+ * If target driver requests own memory allocations, it MUST use
+ * functions scst_cmd_get_tgt_sg*() to get sg and sg_cnt! Otherwise,
+ * it may use functions scst_cmd_get_sg*().
+ */
+ struct scatterlist *tgt_sg;
+ int tgt_sg_cnt;
+ struct scatterlist *tgt_in_sg; /* bidirectional */
+ int tgt_in_sg_cnt; /* bidirectional */
+
+ /*
+ * The status fields in case of errors must be set using
+ * scst_set_cmd_error_status()!
+ */
+ uint8_t status; /* status byte from target device */
+ uint8_t msg_status; /* return status from host adapter itself */
+ uint8_t host_status; /* set by low-level driver to indicate status */
+ uint8_t driver_status; /* set by mid-level */
+
+ uint8_t *sense; /* pointer to sense buffer */
+ unsigned short sense_valid_len; /* length of valid sense data */
+ unsigned short sense_buflen; /* length of the sense buffer, if any */
+
+ /* Start time when cmd was sent to rdy_to_xfer() or xmit_response() */
+ unsigned long hw_pending_start;
+
+ /* Used for storage of target driver private stuff */
+ void *tgt_priv;
+
+ /* Used for storage of dev handler private stuff */
+ void *dh_priv;
+
+ /*
+ * Used to restore the SG vector if it was modified by
+ * scst_set_resp_data_len()
+ */
+ int orig_sg_cnt, orig_sg_entry, orig_entry_len;
+
+ /* Used to retry commands in case of double UA */
+ int dbl_ua_orig_resp_data_len, dbl_ua_orig_data_direction;
+
+ /* List corresponding mgmt cmd, if any, protected by sess_list_lock */
+ struct list_head mgmt_cmd_list;
+
+ /* List entry for dev's blocked_cmd_list */
+ struct list_head blocked_cmd_list_entry;
+
+ struct scst_cmd *orig_cmd; /* Used to issue REQUEST SENSE */
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ /*
+ * Must be the last to allow to work with drivers who don't know
+ * about this config time option.
+ */
+ uint64_t start, curr_start, parse_time, alloc_buf_time;
+ uint64_t restart_waiting_time, rdy_to_xfer_time;
+ uint64_t pre_exec_time, exec_time, dev_done_time;
+ uint64_t xmit_time, tgt_on_free_time, dev_on_free_time;
+#endif
+};
+
+/*
+ * Parameters for SCST management commands
+ */
+struct scst_rx_mgmt_params {
+ int fn;
+ uint64_t tag;
+ const uint8_t *lun;
+ int lun_len;
+ uint32_t cmd_sn;
+ int atomic;
+ void *tgt_priv;
+ unsigned char tag_set;
+ unsigned char lun_set;
+ unsigned char cmd_sn_set;
+};
+
+/*
+ * A stub structure to link an management command and affected regular commands
+ */
+struct scst_mgmt_cmd_stub {
+ struct scst_mgmt_cmd *mcmd;
+
+ /* List entry in cmd->mgmt_cmd_list */
+ struct list_head cmd_mgmt_cmd_list_entry;
+
+ /* set if the cmd was counted in mcmd->cmd_done_wait_count */
+ unsigned int done_counted:1;
+};
+
+/*
+ * SCST task management structure
+ */
+struct scst_mgmt_cmd {
+ /* List entry for *_mgmt_cmd_list */
+ struct list_head mgmt_cmd_list_entry;
+
+ struct scst_session *sess;
+
+ /* Mgmt cmd state, one of SCST_MCMD_STATE_* constants */
+ int state;
+
+ int fn; /* task management function */
+
+ unsigned int completed:1; /* set, if the mcmd is completed */
+ /* Set if device(s) should be unblocked after mcmd's finish */
+ unsigned int needs_unblocking:1;
+ unsigned int lun_set:1; /* set, if lun field is valid */
+ unsigned int cmd_sn_set:1; /* set, if cmd_sn field is valid */
+ /* set, if scst_mgmt_affected_cmds_done was called */
+ unsigned int affected_cmds_done_called:1;
+
+ /*
+ * Number of commands to finish before sending response,
+ * protected by scst_mcmd_lock
+ */
+ int cmd_finish_wait_count;
+
+ /*
+ * Number of commands to complete (done) before resetting reservation,
+ * protected by scst_mcmd_lock
+ */
+ int cmd_done_wait_count;
+
+ /* Number of completed commands, protected by scst_mcmd_lock */
+ int completed_cmd_count;
+
+ uint64_t lun; /* LUN for this mgmt cmd */
+ /* or (and for iSCSI) */
+ uint64_t tag; /* tag of the corresponding cmd */
+
+ uint32_t cmd_sn; /* affected command's highest SN */
+
+ /* corresponding cmd (to be aborted, found by tag) */
+ struct scst_cmd *cmd_to_abort;
+
+ /* corresponding device for this mgmt cmd (found by lun) */
+ struct scst_tgt_dev *mcmd_tgt_dev;
+
+ /* completition status, one of the SCST_MGMT_STATUS_* constants */
+ int status;
+
+ /* Used for storage of target driver private stuff */
+ void *tgt_priv;
+};
+
+/*
+ * SCST device
+ */
+struct scst_device {
+ struct scst_dev_type *handler; /* corresponding dev handler */
+
+ struct scst_mem_lim dev_mem_lim;
+
+ unsigned short type; /* SCSI type of the device */
+
+ /*************************************************************
+ ** Dev's flags. Updates serialized by dev_lock or suspended
+ ** activity
+ *************************************************************/
+
+ /* Set if dev is RESERVED */
+ unsigned short dev_reserved:1;
+
+ /* Set if double reset UA is possible */
+ unsigned short dev_double_ua_possible:1;
+
+ /* If set, dev is read only */
+ unsigned short rd_only:1;
+
+ /**************************************************************/
+
+ /*************************************************************
+ ** Dev's control mode page related values. Updates serialized
+ ** by scst_block_dev(). It's long to not interfere with the
+ ** above flags.
+ *************************************************************/
+
+ unsigned long queue_alg:4;
+ unsigned long tst:3;
+ unsigned long tas:1;
+ unsigned long swp:1;
+ unsigned long d_sense:1;
+
+ /*
+ * Set if device implements own ordered commands management. If not set
+ * and queue_alg is SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER,
+ * expected_sn will be incremented only after commands finished.
+ */
+ unsigned long has_own_order_mgmt:1;
+
+ /**************************************************************/
+
+ /* Used for storage of dev handler private stuff */
+ void *dh_priv;
+
+ /* Corresponding real SCSI device, could be NULL for virtual devices */
+ struct scsi_device *scsi_dev;
+
+ /* Lists of commands with lock, if dedicated threads are used */
+ struct scst_cmd_threads dev_cmd_threads;
+
+ /* How many cmds alive on this dev */
+ atomic_t dev_cmd_count;
+
+ /* How many write cmds alive on this dev. Temporary, ToDo */
+ atomic_t write_cmd_count;
+
+ spinlock_t dev_lock; /* device lock */
+
+ /*
+ * How many times device was blocked for new cmds execution.
+ * Protected by dev_lock
+ */
+ int block_count;
+
+ /*
+ * How many there are "on_dev" commands, i.e. ones those are being
+ * executed by the underlying SCSI/virtual device.
+ */
+ atomic_t on_dev_count;
+
+ struct list_head blocked_cmd_list; /* protected by dev_lock */
+
+ /* Used to wait for requested amount of "on_dev" commands */
+ wait_queue_head_t on_dev_waitQ;
+
+ /* A list entry used during TM, protected by scst_mutex */
+ struct list_head tm_dev_list_entry;
+
+ /* Virtual device internal ID */
+ int virt_id;
+
+ /* Pointer to virtual device name, for convenience only */
+ char *virt_name;
+
+ /* List entry in global devices list */
+ struct list_head dev_list_entry;
+
+ /*
+ * List of tgt_dev's, one per session, protected by scst_mutex or
+ * dev_lock for reads and both for writes
+ */
+ struct list_head dev_tgt_dev_list;
+
+ /* List of acg_dev's, one per acg, protected by scst_mutex */
+ struct list_head dev_acg_dev_list;
+
+ /* Number of threads in the device's threads pools */
+ int threads_num;
+
+ /* Threads pool type of the device. Valid only if threads_num > 0. */
+ enum scst_dev_type_threads_pool_type threads_pool_type;
+
+ /* Set if tgt_kobj was initialized */
+ unsigned int dev_kobj_initialized:1;
+
+ /*
+ * Used to protect sysfs attributes to be called after this
+ * object was unregistered.
+ */
+ struct rw_semaphore dev_attr_rwsem;
+
+ struct kobject dev_kobj; /* kobject for this struct */
+ struct kobject *dev_exp_kobj; /* exported groups */
+
+ /* Export number in the dev's sysfs list. Protected by scst_mutex */
+ int dev_exported_lun_num;
+};
+
+/*
+ * Used to store threads local tgt_dev specific data
+ */
+struct scst_thr_data_hdr {
+ /* List entry in tgt_dev->thr_data_list */
+ struct list_head thr_data_list_entry;
+ struct task_struct *owner_thr; /* the owner thread */
+ atomic_t ref;
+ /* Function that will be called on the tgt_dev destruction */
+ void (*free_fn) (struct scst_thr_data_hdr *data);
+};
+
+/*
+ * Used to clearly dispose async io_context
+ */
+struct scst_async_io_context_keeper {
+ struct kref aic_keeper_kref;
+ struct io_context *aic;
+ struct task_struct *aic_keeper_thr;
+ wait_queue_head_t aic_keeper_waitQ;
+};
+
+/*
+ * Used to store per-session specific device information, analog of
+ * SCSI I_T_L nexus.
+ */
+struct scst_tgt_dev {
+ /* List entry in sess->sess_tgt_dev_list_hash */
+ struct list_head sess_tgt_dev_list_entry;
+
+ struct scst_device *dev; /* to save extra dereferences */
+ uint64_t lun; /* to save extra dereferences */
+
+ gfp_t gfp_mask;
+ struct sgv_pool *pool;
+ int max_sg_cnt;
+
+ unsigned long tgt_dev_flags; /* tgt_dev's async flags */
+
+ /* Used for storage of dev handler private stuff */
+ void *dh_priv;
+
+ /* How many cmds alive on this dev in this session */
+ atomic_t tgt_dev_cmd_count;
+
+ /*
+ * Used to execute cmd's in order of arrival, honoring SCSI task
+ * attributes.
+ *
+ * Protected by sn_lock, except expected_sn, which is protected by
+ * itself. Curr_sn must have the same size as expected_sn to
+ * overflow simultaneously.
+ */
+ int def_cmd_count;
+ spinlock_t sn_lock;
+ unsigned int expected_sn;
+ unsigned int curr_sn;
+ int hq_cmd_count;
+ struct list_head deferred_cmd_list;
+ struct list_head skipped_sn_list;
+
+ /*
+ * Set if the prev cmd was ORDERED. Size must allow unprotected
+ * modifications independant to the neighbour fields.
+ */
+ unsigned long prev_cmd_ordered;
+
+ int num_free_sn_slots; /* if it's <0, then all slots are busy */
+ atomic_t *cur_sn_slot;
+ atomic_t sn_slots[15];
+
+ /* List of scst_thr_data_hdr and lock */
+ spinlock_t thr_data_lock;
+ struct list_head thr_data_list;
+
+ /* Pointer to lists of commands with the lock */
+ struct scst_cmd_threads *active_cmd_threads;
+
+ /* Union to save some CPU cache footprint */
+ union {
+ struct {
+ /* Copy to save fast path dereference */
+ struct io_context *async_io_context;
+
+ struct scst_async_io_context_keeper *aic_keeper;
+ };
+
+ /* Lists of commands with lock, if dedicated threads are used */
+ struct scst_cmd_threads tgt_dev_cmd_threads;
+ };
+
+ spinlock_t tgt_dev_lock; /* per-session device lock */
+
+ /* List of UA's for this device, protected by tgt_dev_lock */
+ struct list_head UA_list;
+
+ struct scst_session *sess; /* corresponding session */
+ struct scst_acg_dev *acg_dev; /* corresponding acg_dev */
+
+ /* List entry in dev->dev_tgt_dev_list */
+ struct list_head dev_tgt_dev_list_entry;
+
+ /* Internal tmp list entry */
+ struct list_head extra_tgt_dev_list_entry;
+
+ /* Set if INQUIRY DATA HAS CHANGED UA is needed */
+ unsigned int inq_changed_ua_needed:1;
+
+ /*
+ * Stored Unit Attention sense and its length for possible
+ * subsequent REQUEST SENSE. Both protected by tgt_dev_lock.
+ */
+ unsigned short tgt_dev_valid_sense_len;
+ uint8_t tgt_dev_sense[SCST_SENSE_BUFFERSIZE];
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ /*
+ * Must be the last to allow to work with drivers who don't know
+ * about this config time option.
+ *
+ * Protected by sess->lat_lock.
+ */
+ uint64_t scst_time, tgt_time, dev_time;
+ unsigned int processed_cmds;
+ struct scst_ext_latency_stat dev_latency_stat[SCST_LATENCY_STATS_NUM];
+#endif
+};
+
+/*
+ * Used to store ACG-specific device information, like LUN
+ */
+struct scst_acg_dev {
+ struct scst_device *dev; /* corresponding device */
+
+ uint64_t lun; /* device's LUN in this acg */
+
+ /* If set, the corresponding LU is read only */
+ unsigned int rd_only:1;
+
+ /* Set if acg_dev_kobj was initialized */
+ unsigned int acg_dev_kobj_initialized:1;
+
+ struct scst_acg *acg; /* parent acg */
+
+ /* List entry in dev->dev_acg_dev_list */
+ struct list_head dev_acg_dev_list_entry;
+
+ /* List entry in acg->acg_dev_list */
+ struct list_head acg_dev_list_entry;
+
+ /* kobject for this structure */
+ struct kobject acg_dev_kobj;
+};
+
+/*
+ * ACG - access control group. Used to store group related
+ * control information.
+ */
+struct scst_acg {
+ /* List of acg_dev's in this acg, protected by scst_mutex */
+ struct list_head acg_dev_list;
+
+ /* List of attached sessions, protected by scst_mutex */
+ struct list_head acg_sess_list;
+
+ /* List of attached acn's, protected by scst_mutex */
+ struct list_head acn_list;
+
+ /* List entry in acg_lists */
+ struct list_head acg_list_entry;
+
+ /* Name of this acg */
+ const char *acg_name;
+
+ /* Type of I/O initiators groupping */
+ int acg_io_grouping_type;
+
+ unsigned int acg_kobj_initialized:1;
+ unsigned int in_tgt_acg_list:1;
+
+ /* kobject for this structure */
+ struct kobject acg_kobj;
+
+ struct kobject *luns_kobj;
+ struct kobject *initiators_kobj;
+
+ unsigned int addr_method;
+};
+
+/*
+ * ACN - access control name. Used to store names, by which
+ * incoming sessions will be assigned to appropriate ACG.
+ */
+struct scst_acn {
+ /* Initiator's name */
+ const char *name;
+ /* List entry in acg->acn_list */
+ struct list_head acn_list_entry;
+
+ /* sysfs file attributes */
+ struct kobj_attribute *acn_attr;
+};
+
+/*
+ * Used to store per-session UNIT ATTENTIONs
+ */
+struct scst_tgt_dev_UA {
+ /* List entry in tgt_dev->UA_list */
+ struct list_head UA_list_entry;
+
+ /* Set if UA is global for session */
+ unsigned short global_UA:1;
+
+ /* Unit Attention valid sense len */
+ unsigned short UA_valid_sense_len;
+ /* Unit Attention sense buf */
+ uint8_t UA_sense_buffer[SCST_SENSE_BUFFERSIZE];
+};
+
+/* Used to deliver AENs */
+struct scst_aen {
+ int event_fn; /* AEN fn */
+
+ struct scst_session *sess; /* corresponding session */
+ uint64_t lun; /* corresponding LUN in SCSI form */
+
+ union {
+ /* SCSI AEN data */
+ struct {
+ int aen_sense_len;
+ uint8_t aen_sense[SCST_STANDARD_SENSE_LEN];
+ };
+ };
+
+ /* Keeps status of AEN's delivery to remote initiator */
+ int delivery_status;
+};
+
+#ifndef smp_mb__after_set_bit
+/* There is no smp_mb__after_set_bit() in the kernel */
+#define smp_mb__after_set_bit() smp_mb()
+#endif
+
+/*
+ * Registers target template.
+ * Returns 0 on success or appropriate error code otherwise.
+ *
+ * Note: *vtt must be static!
+ */
+int __scst_register_target_template(struct scst_tgt_template *vtt,
+ const char *version);
+static inline int scst_register_target_template(struct scst_tgt_template *vtt)
+{
+ return __scst_register_target_template(vtt, SCST_INTERFACE_VERSION);
+}
+
+/*
+ * Registers target template, non-GPL version.
+ * Returns 0 on success or appropriate error code otherwise.
+ *
+ * Note: *vtt must be static!
+ */
+int __scst_register_target_template_non_gpl(struct scst_tgt_template *vtt,
+ const char *version);
+static inline int scst_register_target_template_non_gpl(
+ struct scst_tgt_template *vtt)
+{
+ return __scst_register_target_template_non_gpl(vtt,
+ SCST_INTERFACE_VERSION);
+}
+
+void scst_unregister_target_template(struct scst_tgt_template *vtt);
+
+struct scst_tgt *scst_register_target(struct scst_tgt_template *vtt,
+ const char *target_name);
+void scst_unregister_target(struct scst_tgt *tgt);
+
+struct scst_session *scst_register_session(struct scst_tgt *tgt, int atomic,
+ const char *initiator_name, void *data,
+ void (*result_fn) (struct scst_session *sess, void *data, int result));
+struct scst_session *scst_register_session_simple(struct scst_tgt *tgt,
+ const char *initiator_name);
+void scst_unregister_session(struct scst_session *sess, int wait,
+ void (*unreg_done_fn) (struct scst_session *sess));
+void scst_unregister_session_simple(struct scst_session *sess);
+
+int __scst_register_dev_driver(struct scst_dev_type *dev_type,
+ const char *version);
+static inline int scst_register_dev_driver(struct scst_dev_type *dev_type)
+{
+ return __scst_register_dev_driver(dev_type, SCST_INTERFACE_VERSION);
+}
+void scst_unregister_dev_driver(struct scst_dev_type *dev_type);
+
+int __scst_register_virtual_dev_driver(struct scst_dev_type *dev_type,
+ const char *version);
+/*
+ * Registers dev handler driver for virtual devices (eg VDISK).
+ * Returns 0 on success or appropriate error code otherwise.
+ *
+ * Note: *dev_type must be static!
+ */
+static inline int scst_register_virtual_dev_driver(
+ struct scst_dev_type *dev_type)
+{
+ return __scst_register_virtual_dev_driver(dev_type,
+ SCST_INTERFACE_VERSION);
+}
+
+void scst_unregister_virtual_dev_driver(struct scst_dev_type *dev_type);
+
+struct scst_cmd *scst_rx_cmd(struct scst_session *sess,
+ const uint8_t *lun, int lun_len, const uint8_t *cdb,
+ int cdb_len, int atomic);
+void scst_cmd_init_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context);
+
+/*
+ * Notifies SCST that the driver finished the first stage of the command
+ * initialization, and the command is ready for execution, but after
+ * SCST done the command's preprocessing preprocessing_done() function
+ * should be called. The second argument sets preferred command execition
+ * context. See SCST_CONTEXT_* constants for details.
+ *
+ * See comment for scst_cmd_init_done() for the serialization requirements.
+ */
+static inline void scst_cmd_init_stage1_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context, int set_sn)
+{
+ cmd->preprocessing_only = 1;
+ cmd->set_sn_on_restart_cmd = !set_sn;
+ scst_cmd_init_done(cmd, pref_context);
+}
+
+void scst_restart_cmd(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context);
+
+void scst_rx_data(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context);
+
+void scst_tgt_cmd_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context);
+
+int scst_rx_mgmt_fn(struct scst_session *sess,
+ const struct scst_rx_mgmt_params *params);
+
+/*
+ * Creates new management command using tag and sends it for execution.
+ * Can be used for SCST_ABORT_TASK only.
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same sess. Returns 0 for success, error code otherwise.
+ *
+ * Obsolete in favor of scst_rx_mgmt_fn()
+ */
+static inline int scst_rx_mgmt_fn_tag(struct scst_session *sess, int fn,
+ uint64_t tag, int atomic, void *tgt_priv)
+{
+ struct scst_rx_mgmt_params params;
+
+ BUG_ON(fn != SCST_ABORT_TASK);
+
+ memset(¶ms, 0, sizeof(params));
+ params.fn = fn;
+ params.tag = tag;
+ params.tag_set = 1;
+ params.atomic = atomic;
+ params.tgt_priv = tgt_priv;
+ return scst_rx_mgmt_fn(sess, ¶ms);
+}
+
+/*
+ * Creates new management command using LUN and sends it for execution.
+ * Currently can be used for any fn, except SCST_ABORT_TASK.
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same sess. Returns 0 for success, error code otherwise.
+ *
+ * Obsolete in favor of scst_rx_mgmt_fn()
+ */
+static inline int scst_rx_mgmt_fn_lun(struct scst_session *sess, int fn,
+ const uint8_t *lun, int lun_len, int atomic, void *tgt_priv)
+{
+ struct scst_rx_mgmt_params params;
+
+ BUG_ON(fn == SCST_ABORT_TASK);
+
+ memset(¶ms, 0, sizeof(params));
+ params.fn = fn;
+ params.lun = lun;
+ params.lun_len = lun_len;
+ params.lun_set = 1;
+ params.atomic = atomic;
+ params.tgt_priv = tgt_priv;
+ return scst_rx_mgmt_fn(sess, ¶ms);
+}
+
+int scst_get_cdb_info(struct scst_cmd *cmd);
+
+int scst_set_cmd_error_status(struct scst_cmd *cmd, int status);
+int scst_set_cmd_error(struct scst_cmd *cmd, int key, int asc, int ascq);
+void scst_set_busy(struct scst_cmd *cmd);
+
+void scst_check_convert_sense(struct scst_cmd *cmd);
+
+void scst_set_initial_UA(struct scst_session *sess, int key, int asc, int ascq);
+
+void scst_capacity_data_changed(struct scst_device *dev);
+
+struct scst_cmd *scst_find_cmd_by_tag(struct scst_session *sess, uint64_t tag);
+struct scst_cmd *scst_find_cmd(struct scst_session *sess, void *data,
+ int (*cmp_fn) (struct scst_cmd *cmd,
+ void *data));
+
+enum dma_data_direction scst_to_dma_dir(int scst_dir);
+enum dma_data_direction scst_to_tgt_dma_dir(int scst_dir);
+
+/*
+ * Returns true, if cmd's CDB is fully locally handled by SCST and false
+ * otherwise. Dev handlers parse() and dev_done() not called for such commands.
+ */
+static inline bool scst_is_cmd_fully_local(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_FULLY_LOCAL_CMD) != 0;
+}
+
+/*
+ * Returns true, if cmd's CDB is locally handled by SCST and
+ * false otherwise.
+ */
+static inline bool scst_is_cmd_local(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_LOCAL_CMD) != 0;
+}
+
+/* Returns true, if cmd can deliver UA */
+static inline bool scst_is_ua_command(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_SKIP_UA) == 0;
+}
+
+int scst_register_virtual_device(struct scst_dev_type *dev_handler,
+ const char *dev_name);
+void scst_unregister_virtual_device(int id);
+
+/*
+ * Get/Set functions for tgt's sg_tablesize
+ */
+static inline int scst_tgt_get_sg_tablesize(struct scst_tgt *tgt)
+{
+ return tgt->sg_tablesize;
+}
+
+static inline void scst_tgt_set_sg_tablesize(struct scst_tgt *tgt, int val)
+{
+ tgt->sg_tablesize = val;
+}
+
+/*
+ * Get/Set functions for tgt's target private data
+ */
+static inline void *scst_tgt_get_tgt_priv(struct scst_tgt *tgt)
+{
+ return tgt->tgt_priv;
+}
+
+static inline void scst_tgt_set_tgt_priv(struct scst_tgt *tgt, void *val)
+{
+ tgt->tgt_priv = val;
+}
+
+/*
+ * Get/Set functions for session's target private data
+ */
+static inline void *scst_sess_get_tgt_priv(struct scst_session *sess)
+{
+ return sess->tgt_priv;
+}
+
+static inline void scst_sess_set_tgt_priv(struct scst_session *sess,
+ void *val)
+{
+ sess->tgt_priv = val;
+}
+
+/**
+ * Returns TRUE if cmd is being executed in atomic context.
+ *
+ * Note: checkpatch will complain on the use of in_atomic() below. You can
+ * safely ignore this warning since in_atomic() is used here only for debugging
+ * purposes.
+ */
+static inline bool scst_cmd_atomic(struct scst_cmd *cmd)
+{
+ int res = cmd->atomic;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely((in_atomic() || in_interrupt() || irqs_disabled()) &&
+ !res)) {
+ printk(KERN_ERR "ERROR: atomic context and non-atomic cmd\n");
+ dump_stack();
+ cmd->atomic = 1;
+ res = 1;
+ }
+#endif
+ return res;
+}
+
+/*
+ * Returns TRUE if cmd has been preliminary completed, i.e. completed or
+ * aborted.
+ */
+static inline bool scst_cmd_prelim_completed(struct scst_cmd *cmd)
+{
+ return cmd->completed || test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags);
+}
+
+static inline enum scst_exec_context __scst_estimate_context(bool direct)
+{
+ if (in_irq())
+ return SCST_CONTEXT_TASKLET;
+ else if (irqs_disabled())
+ return SCST_CONTEXT_THREAD;
+ else
+ return direct ? SCST_CONTEXT_DIRECT :
+ SCST_CONTEXT_DIRECT_ATOMIC;
+}
+
+static inline enum scst_exec_context scst_estimate_context(void)
+{
+ return __scst_estimate_context(0);
+}
+
+static inline enum scst_exec_context scst_estimate_context_direct(void)
+{
+ return __scst_estimate_context(1);
+}
+
+/* Returns cmd's CDB */
+static inline const uint8_t *scst_cmd_get_cdb(struct scst_cmd *cmd)
+{
+ return cmd->cdb;
+}
+
+/* Returns cmd's CDB length */
+static inline int scst_cmd_get_cdb_len(struct scst_cmd *cmd)
+{
+ return cmd->cdb_len;
+}
+
+/* Returns cmd's extended CDB */
+static inline const uint8_t *scst_cmd_get_ext_cdb(struct scst_cmd *cmd)
+{
+ return cmd->ext_cdb;
+}
+
+/* Returns cmd's extended CDB length */
+static inline int scst_cmd_get_ext_cdb_len(struct scst_cmd *cmd)
+{
+ return cmd->ext_cdb_len;
+}
+
+/* Sets cmd's extended CDB and its length */
+static inline void scst_cmd_set_ext_cdb(struct scst_cmd *cmd,
+ uint8_t *ext_cdb, unsigned int ext_cdb_len)
+{
+ cmd->ext_cdb = ext_cdb;
+ cmd->ext_cdb_len = ext_cdb_len;
+}
+
+/* Returns cmd's session */
+static inline struct scst_session *scst_cmd_get_session(struct scst_cmd *cmd)
+{
+ return cmd->sess;
+}
+
+/* Returns cmd's response data length */
+static inline int scst_cmd_get_resp_data_len(struct scst_cmd *cmd)
+{
+ return cmd->resp_data_len;
+}
+
+/* Returns if status should be sent for cmd */
+static inline int scst_cmd_get_is_send_status(struct scst_cmd *cmd)
+{
+ return cmd->is_send_status;
+}
+
+/*
+ * Returns pointer to cmd's SG data buffer.
+ *
+ * Usage of this function is not recommended, use scst_get_buf_*()
+ * family of functions instead.
+ */
+static inline struct scatterlist *scst_cmd_get_sg(struct scst_cmd *cmd)
+{
+ return cmd->sg;
+}
+
+/*
+ * Returns cmd's sg_cnt.
+ *
+ * Usage of this function is not recommended, use scst_get_buf_*()
+ * family of functions instead.
+ */
+static inline int scst_cmd_get_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->sg_cnt;
+}
+
+/*
+ * Returns cmd's data buffer length.
+ *
+ * In case if you need to iterate over data in the buffer, usage of
+ * this function is not recommended, use scst_get_buf_*()
+ * family of functions instead.
+ */
+static inline unsigned int scst_cmd_get_bufflen(struct scst_cmd *cmd)
+{
+ return cmd->bufflen;
+}
+
+/*
+ * Returns pointer to cmd's bidirectional in (WRITE) SG data buffer.
+ *
+ * Usage of this function is not recommended, use scst_get_in_buf_*()
+ * family of functions instead.
+ */
+static inline struct scatterlist *scst_cmd_get_in_sg(struct scst_cmd *cmd)
+{
+ return cmd->in_sg;
+}
+
+/*
+ * Returns cmd's bidirectional in (WRITE) sg_cnt.
+ *
+ * Usage of this function is not recommended, use scst_get_in_buf_*()
+ * family of functions instead.
+ */
+static inline int scst_cmd_get_in_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->in_sg_cnt;
+}
+
+/*
+ * Returns cmd's bidirectional in (WRITE) data buffer length.
+ *
+ * In case if you need to iterate over data in the buffer, usage of
+ * this function is not recommended, use scst_get_in_buf_*()
+ * family of functions instead.
+ */
+static inline unsigned int scst_cmd_get_in_bufflen(struct scst_cmd *cmd)
+{
+ return cmd->in_bufflen;
+}
+
+/* Returns pointer to cmd's target's SG data buffer */
+static inline struct scatterlist *scst_cmd_get_tgt_sg(struct scst_cmd *cmd)
+{
+ return cmd->tgt_sg;
+}
+
+/* Returns cmd's target's sg_cnt */
+static inline int scst_cmd_get_tgt_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->tgt_sg_cnt;
+}
+
+/* Sets cmd's target's SG data buffer */
+static inline void scst_cmd_set_tgt_sg(struct scst_cmd *cmd,
+ struct scatterlist *sg, int sg_cnt)
+{
+ cmd->tgt_sg = sg;
+ cmd->tgt_sg_cnt = sg_cnt;
+ cmd->tgt_data_buf_alloced = 1;
+}
+
+/* Returns pointer to cmd's target's IN SG data buffer */
+static inline struct scatterlist *scst_cmd_get_in_tgt_sg(struct scst_cmd *cmd)
+{
+ return cmd->tgt_in_sg;
+}
+
+/* Returns cmd's target's IN sg_cnt */
+static inline int scst_cmd_get_tgt_in_sg_cnt(struct scst_cmd *cmd)
+{
+ return cmd->tgt_in_sg_cnt;
+}
+
+/* Sets cmd's target's IN SG data buffer */
+static inline void scst_cmd_set_tgt_in_sg(struct scst_cmd *cmd,
+ struct scatterlist *sg, int sg_cnt)
+{
+ WARN_ON(!cmd->tgt_data_buf_alloced);
+
+ cmd->tgt_in_sg = sg;
+ cmd->tgt_in_sg_cnt = sg_cnt;
+}
+
+/* Returns cmd's data direction */
+static inline scst_data_direction scst_cmd_get_data_direction(
+ struct scst_cmd *cmd)
+{
+ return cmd->data_direction;
+}
+
+/* Returns cmd's status byte from host device */
+static inline uint8_t scst_cmd_get_status(struct scst_cmd *cmd)
+{
+ return cmd->status;
+}
+
+/* Returns cmd's status from host adapter itself */
+static inline uint8_t scst_cmd_get_msg_status(struct scst_cmd *cmd)
+{
+ return cmd->msg_status;
+}
+
+/* Returns cmd's status set by low-level driver to indicate its status */
+static inline uint8_t scst_cmd_get_host_status(struct scst_cmd *cmd)
+{
+ return cmd->host_status;
+}
+
+/* Returns cmd's status set by SCSI mid-level */
+static inline uint8_t scst_cmd_get_driver_status(struct scst_cmd *cmd)
+{
+ return cmd->driver_status;
+}
+
+/* Returns pointer to cmd's sense buffer */
+static inline uint8_t *scst_cmd_get_sense_buffer(struct scst_cmd *cmd)
+{
+ return cmd->sense;
+}
+
+/* Returns cmd's valid sense length */
+static inline int scst_cmd_get_sense_buffer_len(struct scst_cmd *cmd)
+{
+ return cmd->sense_valid_len;
+}
+
+/*
+ * Get/Set functions for cmd's queue_type
+ */
+static inline enum scst_cmd_queue_type scst_cmd_get_queue_type(
+ struct scst_cmd *cmd)
+{
+ return cmd->queue_type;
+}
+
+static inline void scst_cmd_set_queue_type(struct scst_cmd *cmd,
+ enum scst_cmd_queue_type queue_type)
+{
+ cmd->queue_type = queue_type;
+}
+
+/*
+ * Get/Set functions for cmd's target SN
+ */
+static inline uint64_t scst_cmd_get_tag(struct scst_cmd *cmd)
+{
+ return cmd->tag;
+}
+
+static inline void scst_cmd_set_tag(struct scst_cmd *cmd, uint64_t tag)
+{
+ cmd->tag = tag;
+}
+
+/*
+ * Get/Set functions for cmd's target private data.
+ * Variant with *_lock must be used if target driver uses
+ * scst_find_cmd() to avoid race with it, except inside scst_find_cmd()'s
+ * callback, where lock is already taken.
+ */
+static inline void *scst_cmd_get_tgt_priv(struct scst_cmd *cmd)
+{
+ return cmd->tgt_priv;
+}
+
+static inline void scst_cmd_set_tgt_priv(struct scst_cmd *cmd, void *val)
+{
+ cmd->tgt_priv = val;
+}
+
+/*
+ * Get/Set functions for tgt_need_alloc_data_buf flag
+ */
+static inline int scst_cmd_get_tgt_need_alloc_data_buf(struct scst_cmd *cmd)
+{
+ return cmd->tgt_need_alloc_data_buf;
+}
+
+static inline void scst_cmd_set_tgt_need_alloc_data_buf(struct scst_cmd *cmd)
+{
+ cmd->tgt_need_alloc_data_buf = 1;
+}
+
+/*
+ * Get/Set functions for tgt_data_buf_alloced flag
+ */
+static inline int scst_cmd_get_tgt_data_buff_alloced(struct scst_cmd *cmd)
+{
+ return cmd->tgt_data_buf_alloced;
+}
+
+static inline void scst_cmd_set_tgt_data_buff_alloced(struct scst_cmd *cmd)
+{
+ cmd->tgt_data_buf_alloced = 1;
+}
+
+/*
+ * Get/Set functions for dh_data_buf_alloced flag
+ */
+static inline int scst_cmd_get_dh_data_buff_alloced(struct scst_cmd *cmd)
+{
+ return cmd->dh_data_buf_alloced;
+}
+
+static inline void scst_cmd_set_dh_data_buff_alloced(struct scst_cmd *cmd)
+{
+ cmd->dh_data_buf_alloced = 1;
+}
+
+/*
+ * Get/Set functions for no_sgv flag
+ */
+static inline int scst_cmd_get_no_sgv(struct scst_cmd *cmd)
+{
+ return cmd->no_sgv;
+}
+
+static inline void scst_cmd_set_no_sgv(struct scst_cmd *cmd)
+{
+ cmd->no_sgv = 1;
+}
+
+/*
+ * Get/Set functions for tgt_sn
+ */
+static inline int scst_cmd_get_tgt_sn(struct scst_cmd *cmd)
+{
+ BUG_ON(!cmd->tgt_sn_set);
+ return cmd->tgt_sn;
+}
+
+static inline void scst_cmd_set_tgt_sn(struct scst_cmd *cmd, uint32_t tgt_sn)
+{
+ cmd->tgt_sn_set = 1;
+ cmd->tgt_sn = tgt_sn;
+}
+
+/*
+ * Returns 1 if the cmd was aborted, so its status is invalid and no
+ * reply shall be sent to the remote initiator. A target driver should
+ * only clear internal resources, associated with cmd.
+ */
+static inline int scst_cmd_aborted(struct scst_cmd *cmd)
+{
+ return test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags) &&
+ !test_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+}
+
+/* Returns sense data format for cmd's dev */
+static inline bool scst_get_cmd_dev_d_sense(struct scst_cmd *cmd)
+{
+ return (cmd->dev != NULL) ? cmd->dev->d_sense : 0;
+}
+
+/*
+ * Get/Set functions for expected data direction, transfer length
+ * and its validity flag
+ */
+static inline int scst_cmd_is_expected_set(struct scst_cmd *cmd)
+{
+ return cmd->expected_values_set;
+}
+
+static inline scst_data_direction scst_cmd_get_expected_data_direction(
+ struct scst_cmd *cmd)
+{
+ return cmd->expected_data_direction;
+}
+
+static inline int scst_cmd_get_expected_transfer_len(
+ struct scst_cmd *cmd)
+{
+ return cmd->expected_transfer_len;
+}
+
+static inline int scst_cmd_get_expected_in_transfer_len(
+ struct scst_cmd *cmd)
+{
+ return cmd->expected_in_transfer_len;
+}
+
+static inline void scst_cmd_set_expected(struct scst_cmd *cmd,
+ scst_data_direction expected_data_direction,
+ int expected_transfer_len)
+{
+ cmd->expected_data_direction = expected_data_direction;
+ cmd->expected_transfer_len = expected_transfer_len;
+ cmd->expected_values_set = 1;
+}
+
+static inline void scst_cmd_set_expected_in_transfer_len(struct scst_cmd *cmd,
+ int expected_in_transfer_len)
+{
+ WARN_ON(!cmd->expected_values_set);
+ cmd->expected_in_transfer_len = expected_in_transfer_len;
+}
+
+/*
+ * Get/clear functions for cmd's may_need_dma_sync
+ */
+static inline int scst_get_may_need_dma_sync(struct scst_cmd *cmd)
+{
+ return cmd->may_need_dma_sync;
+}
+
+static inline void scst_clear_may_need_dma_sync(struct scst_cmd *cmd)
+{
+ cmd->may_need_dma_sync = 0;
+}
+
+/*
+ * Get/set functions for cmd's delivery_status. It is one of
+ * SCST_CMD_DELIVERY_* constants. It specifies the status of the
+ * command's delivery to initiator.
+ */
+static inline int scst_get_delivery_status(struct scst_cmd *cmd)
+{
+ return cmd->delivery_status;
+}
+
+static inline void scst_set_delivery_status(struct scst_cmd *cmd,
+ int delivery_status)
+{
+ cmd->delivery_status = delivery_status;
+}
+
+/*
+ * Get/Set function for mgmt cmd's target private data
+ */
+static inline void *scst_mgmt_cmd_get_tgt_priv(struct scst_mgmt_cmd *mcmd)
+{
+ return mcmd->tgt_priv;
+}
+
+static inline void scst_mgmt_cmd_set_tgt_priv(struct scst_mgmt_cmd *mcmd,
+ void *val)
+{
+ mcmd->tgt_priv = val;
+}
+
+/* Returns mgmt cmd's completition status (SCST_MGMT_STATUS_* constants) */
+static inline int scst_mgmt_cmd_get_status(struct scst_mgmt_cmd *mcmd)
+{
+ return mcmd->status;
+}
+
+/* Returns mgmt cmd's TM fn */
+static inline int scst_mgmt_cmd_get_fn(struct scst_mgmt_cmd *mcmd)
+{
+ return mcmd->fn;
+}
+
+/*
+ * Called by dev handler's task_mgmt_fn() to notify SCST core that mcmd
+ * is going to complete asynchronously.
+ */
+void scst_prepare_async_mcmd(struct scst_mgmt_cmd *mcmd);
+
+/*
+ * Called by dev handler to notify SCST core that async. mcmd is completed
+ * with status "status".
+ */
+void scst_async_mcmd_completed(struct scst_mgmt_cmd *mcmd, int status);
+
+/* Returns AEN's fn */
+static inline int scst_aen_get_event_fn(struct scst_aen *aen)
+{
+ return aen->event_fn;
+}
+
+/* Returns AEN's session */
+static inline struct scst_session *scst_aen_get_sess(struct scst_aen *aen)
+{
+ return aen->sess;
+}
+
+/* Returns AEN's LUN */
+static inline uint64_t scst_aen_get_lun(struct scst_aen *aen)
+{
+ return aen->lun;
+}
+
+/* Returns SCSI AEN's sense */
+static inline const uint8_t *scst_aen_get_sense(struct scst_aen *aen)
+{
+ return aen->aen_sense;
+}
+
+/* Returns SCSI AEN's sense length */
+static inline int scst_aen_get_sense_len(struct scst_aen *aen)
+{
+ return aen->aen_sense_len;
+}
+
+/*
+ * Get/set functions for AEN's delivery_status. It is one of
+ * SCST_AEN_RES_* constants. It specifies the status of the
+ * command's delivery to initiator.
+ */
+static inline int scst_get_aen_delivery_status(struct scst_aen *aen)
+{
+ return aen->delivery_status;
+}
+
+static inline void scst_set_aen_delivery_status(struct scst_aen *aen,
+ int status)
+{
+ aen->delivery_status = status;
+}
+
+void scst_aen_done(struct scst_aen *aen);
+
+static inline void sg_clear(struct scatterlist *sg)
+{
+ memset(sg, 0, sizeof(*sg));
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+}
+
+enum scst_sg_copy_dir {
+ SCST_SG_COPY_FROM_TARGET,
+ SCST_SG_COPY_TO_TARGET
+};
+
+void scst_copy_sg(struct scst_cmd *cmd, enum scst_sg_copy_dir copy_dir);
+
+/*
+ * Functions for access to the commands data (SG) buffer,
+ * including HIGHMEM environment. Should be used instead of direct
+ * access. Returns the mapped buffer length for success, 0 for EOD,
+ * negative error code otherwise.
+ *
+ * "Buf" argument returns the mapped buffer
+ *
+ * The "put" function unmaps the buffer.
+ */
+static inline int __scst_get_buf(struct scst_cmd *cmd, struct scatterlist *sg,
+ int sg_cnt, uint8_t **buf)
+{
+ int res = 0;
+ int i = cmd->get_sg_buf_entry_num;
+
+ *buf = NULL;
+
+ if ((i >= sg_cnt) || unlikely(sg == NULL))
+ goto out;
+
+ *buf = page_address(sg_page(&sg[i]));
+ *buf += sg[i].offset;
+
+ res = sg[i].length;
+ cmd->get_sg_buf_entry_num++;
+
+out:
+ return res;
+}
+
+static inline int scst_get_buf_first(struct scst_cmd *cmd, uint8_t **buf)
+{
+ cmd->get_sg_buf_entry_num = 0;
+ cmd->may_need_dma_sync = 1;
+ return __scst_get_buf(cmd, cmd->sg, cmd->sg_cnt, buf);
+}
+
+static inline int scst_get_buf_next(struct scst_cmd *cmd, uint8_t **buf)
+{
+ return __scst_get_buf(cmd, cmd->sg, cmd->sg_cnt, buf);
+}
+
+static inline void scst_put_buf(struct scst_cmd *cmd, void *buf)
+{
+ /* Nothing to do */
+}
+
+static inline int scst_get_in_buf_first(struct scst_cmd *cmd, uint8_t **buf)
+{
+ cmd->get_sg_buf_entry_num = 0;
+ cmd->may_need_dma_sync = 1;
+ return __scst_get_buf(cmd, cmd->in_sg, cmd->in_sg_cnt, buf);
+}
+
+static inline int scst_get_in_buf_next(struct scst_cmd *cmd, uint8_t **buf)
+{
+ return __scst_get_buf(cmd, cmd->in_sg, cmd->in_sg_cnt, buf);
+}
+
+static inline void scst_put_in_buf(struct scst_cmd *cmd, void *buf)
+{
+ /* Nothing to do */
+}
+
+/*
+ * Returns approximate higher rounded buffers count that
+ * scst_get_buf_[first|next]() return.
+ */
+static inline int scst_get_buf_count(struct scst_cmd *cmd)
+{
+ return (cmd->sg_cnt == 0) ? 1 : cmd->sg_cnt;
+}
+
+/*
+ * Returns approximate higher rounded buffers count that
+ * scst_get_in_buf_[first|next]() return.
+ */
+static inline int scst_get_in_buf_count(struct scst_cmd *cmd)
+{
+ return (cmd->in_sg_cnt == 0) ? 1 : cmd->in_sg_cnt;
+}
+
+int scst_suspend_activity(bool interruptible);
+void scst_resume_activity(void);
+
+void scst_process_active_cmd(struct scst_cmd *cmd, bool atomic);
+void scst_post_parse_process_active_cmd(struct scst_cmd *cmd, bool atomic);
+
+int scst_check_local_events(struct scst_cmd *cmd);
+
+int scst_get_cmd_abnormal_done_state(const struct scst_cmd *cmd);
+void scst_set_cmd_abnormal_done_state(struct scst_cmd *cmd);
+
+struct scst_trace_log {
+ unsigned int val;
+ const char *token;
+};
+
+extern struct mutex scst_mutex;
+
+extern struct sysfs_ops scst_sysfs_ops;
+
+/*
+ * Returns target driver's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_tgtt_kobj(
+ struct scst_tgt_template *tgtt)
+{
+ return &tgtt->tgtt_kobj;
+}
+
+/*
+ * Returns target's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_tgt_kobj(
+ struct scst_tgt *tgt)
+{
+ return &tgt->tgt_kobj;
+}
+
+/*
+ * Returns device handler's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_devt_kobj(
+ struct scst_dev_type *devt)
+{
+ return &devt->devt_kobj;
+}
+
+/*
+ * Returns device's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_dev_kobj(
+ struct scst_device *dev)
+{
+ return &dev->dev_kobj;
+}
+
+/*
+ * Returns session's root sysfs kobject.
+ * The driver can create own files/directories/links here.
+ */
+static inline struct kobject *scst_sysfs_get_sess_kobj(
+ struct scst_session *sess)
+{
+ return &sess->sess_kobj;
+}
+
+/* Returns target name */
+static inline const char *scst_get_tgt_name(const struct scst_tgt *tgt)
+{
+ return tgt->tgt_name;
+}
+
+int scst_alloc_sense(struct scst_cmd *cmd, int atomic);
+int scst_alloc_set_sense(struct scst_cmd *cmd, int atomic,
+ const uint8_t *sense, unsigned int len);
+
+int scst_set_sense(uint8_t *buffer, int len, bool d_sense,
+ int key, int asc, int ascq);
+
+bool scst_is_ua_sense(const uint8_t *sense, int len);
+
+bool scst_analyze_sense(const uint8_t *sense, int len,
+ unsigned int valid_mask, int key, int asc, int ascq);
+
+unsigned long scst_random(void);
+
+void scst_set_resp_data_len(struct scst_cmd *cmd, int resp_data_len);
+
+void scst_get(void);
+void scst_put(void);
+
+void scst_cmd_get(struct scst_cmd *cmd);
+void scst_cmd_put(struct scst_cmd *cmd);
+
+struct scatterlist *scst_alloc(int size, gfp_t gfp_mask, int *count);
+void scst_free(struct scatterlist *sg, int count);
+
+void scst_add_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct scst_thr_data_hdr *data,
+ void (*free_fn) (struct scst_thr_data_hdr *data));
+void scst_del_all_thr_data(struct scst_tgt_dev *tgt_dev);
+void scst_dev_del_all_thr_data(struct scst_device *dev);
+struct scst_thr_data_hdr *__scst_find_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct task_struct *tsk);
+
+/* Finds local to the current thread data. Returns NULL, if they not found. */
+static inline struct scst_thr_data_hdr *scst_find_thr_data(
+ struct scst_tgt_dev *tgt_dev)
+{
+ return __scst_find_thr_data(tgt_dev, current);
+}
+
+/* Increase ref counter for the thread data */
+static inline void scst_thr_data_get(struct scst_thr_data_hdr *data)
+{
+ atomic_inc(&data->ref);
+}
+
+/* Decrease ref counter for the thread data */
+static inline void scst_thr_data_put(struct scst_thr_data_hdr *data)
+{
+ if (atomic_dec_and_test(&data->ref))
+ data->free_fn(data);
+}
+
+int scst_calc_block_shift(int sector_size);
+int scst_sbc_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd));
+int scst_cdrom_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd));
+int scst_modisk_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd));
+int scst_tape_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_size)(struct scst_cmd *cmd));
+int scst_changer_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd));
+int scst_processor_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd));
+int scst_raid_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd));
+
+int scst_block_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_shift)(struct scst_cmd *cmd, int block_shift));
+int scst_tape_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_size)(struct scst_cmd *cmd, int block_size));
+
+int scst_obtain_device_parameters(struct scst_device *dev);
+
+int scst_get_max_lun_commands(struct scst_session *sess, uint64_t lun);
+
+/*
+ * Has to be put here open coded, because Linux doesn't have equivalent, which
+ * allows exclusive wake ups of threads in LIFO order. We need it to let (yet)
+ * unneeded threads sleep and not pollute CPU cache by their stacks.
+ */
+static inline void add_wait_queue_exclusive_head(wait_queue_head_t *q,
+ wait_queue_t *wait)
+{
+ unsigned long flags;
+
+ wait->flags |= WQ_FLAG_EXCLUSIVE;
+ spin_lock_irqsave(&q->lock, flags);
+ __add_wait_queue(q, wait);
+ spin_unlock_irqrestore(&q->lock, flags);
+}
+
+/*
+ * Structure to match events to user space and replies on them
+ */
+struct scst_sysfs_user_info {
+ /* Unique cookie to identify request */
+ uint32_t info_cookie;
+
+ /* Entry in the global list */
+ struct list_head info_list_entry;
+
+ /* Set if reply from the user space is being executed */
+ unsigned int info_being_executed:1;
+
+ /* Set if this info is in the info_list */
+ unsigned int info_in_list:1;
+
+ /* Completion to wait on for the request completion */
+ struct completion info_completion;
+
+ /* Request completion status and optional data */
+ int info_status;
+ void *data;
+};
+
+int scst_sysfs_user_add_info(struct scst_sysfs_user_info **out_info);
+void scst_sysfs_user_del_info(struct scst_sysfs_user_info *info);
+struct scst_sysfs_user_info *scst_sysfs_user_get_info(uint32_t cookie);
+int scst_wait_info_completion(struct scst_sysfs_user_info *info,
+ unsigned long timeout);
+
+unsigned int scst_get_setup_id(void);
+
+char *scst_get_next_lexem(char **token_str);
+void scst_restore_token_str(char *prev_lexem, char *token_str);
+char *scst_get_next_token_str(char **input_str);
+
+void scst_init_threads(struct scst_cmd_threads *cmd_threads);
+void scst_deinit_threads(struct scst_cmd_threads *cmd_threads);
+
+#endif /* __SCST_H */
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 3/12/1/5] SCST core's scst_main.c
[not found] ` <4BC44D08.4060907@vlnb.net>
2010-04-13 13:04 ` [PATCH][RFC 1/12/1/5] SCST core's Makefile and Kconfig Vladislav Bolkhovitin
2010-04-13 13:04 ` [PATCH][RFC 2/12/1/5] SCST core's external headers Vladislav Bolkhovitin
@ 2010-04-13 13:04 ` Vladislav Bolkhovitin
2010-04-13 13:05 ` [PATCH][RFC 4/12/1/5] SCST core's scst_targ.c Vladislav Bolkhovitin
` (6 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:04 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains file scst_main.c.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_main.c | 2047 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 2047 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/scst_main.c linux-2.6.33/drivers/scst/scst_main.c
--- orig/linux-2.6.33/drivers/scst/scst_main.c
+++ linux-2.6.33/drivers/scst/scst_main.c
@@ -0,0 +1,2047 @@
+/*
+ * scst_main.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+#include <linux/kthread.h>
+
+#include "scst.h"
+#include "scst_priv.h"
+#include "scst_mem.h"
+
+#if defined(CONFIG_HIGHMEM4G) || defined(CONFIG_HIGHMEM64G)
+#warning "HIGHMEM kernel configurations are fully supported, but not\
+ recommended for performance reasons. Consider changing VMSPLIT\
+ option or use a 64-bit configuration instead. See README file for\
+ details."
+#endif
+
+/**
+ ** SCST global variables. They are all uninitialized to have their layout in
+ ** memory be exactly as specified. Otherwise compiler puts zero-initialized
+ ** variable separately from nonzero-initialized ones.
+ **/
+
+/*
+ * Main SCST mutex. All targets, devices and dev_types management is done
+ * under this mutex.
+ *
+ * It must NOT be used in any works (schedule_work(), etc.), because
+ * otherwise a deadlock (double lock, actually) is possible, e.g., with
+ * scst_user detach_tgt(), which is called under scst_mutex and calls
+ * flush_scheduled_work().
+ */
+struct mutex scst_mutex;
+EXPORT_SYMBOL_GPL(scst_mutex);
+
+ /* All 3 protected by scst_mutex */
+struct list_head scst_template_list;
+struct list_head scst_dev_list;
+struct list_head scst_dev_type_list;
+
+spinlock_t scst_main_lock;
+
+static struct kmem_cache *scst_mgmt_cachep;
+mempool_t *scst_mgmt_mempool;
+static struct kmem_cache *scst_mgmt_stub_cachep;
+mempool_t *scst_mgmt_stub_mempool;
+static struct kmem_cache *scst_ua_cachep;
+mempool_t *scst_ua_mempool;
+static struct kmem_cache *scst_sense_cachep;
+mempool_t *scst_sense_mempool;
+static struct kmem_cache *scst_aen_cachep;
+mempool_t *scst_aen_mempool;
+struct kmem_cache *scst_tgtd_cachep;
+struct kmem_cache *scst_sess_cachep;
+struct kmem_cache *scst_acgd_cachep;
+
+unsigned int scst_setup_id;
+
+spinlock_t scst_init_lock;
+wait_queue_head_t scst_init_cmd_list_waitQ;
+struct list_head scst_init_cmd_list;
+unsigned int scst_init_poll_cnt;
+
+struct kmem_cache *scst_cmd_cachep;
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+unsigned long scst_trace_flag;
+#endif
+
+unsigned long scst_flags;
+atomic_t scst_cmd_count;
+
+struct scst_cmd_threads scst_main_cmd_threads;
+
+struct scst_tasklet scst_tasklets[NR_CPUS];
+
+spinlock_t scst_mcmd_lock;
+struct list_head scst_active_mgmt_cmd_list;
+struct list_head scst_delayed_mgmt_cmd_list;
+wait_queue_head_t scst_mgmt_cmd_list_waitQ;
+
+wait_queue_head_t scst_mgmt_waitQ;
+spinlock_t scst_mgmt_lock;
+struct list_head scst_sess_init_list;
+struct list_head scst_sess_shut_list;
+
+wait_queue_head_t scst_dev_cmd_waitQ;
+
+static struct mutex scst_suspend_mutex;
+/* protected by scst_suspend_mutex */
+static struct list_head scst_cmd_threads_list;
+
+int scst_threads;
+static struct task_struct *scst_init_cmd_thread;
+static struct task_struct *scst_mgmt_thread;
+static struct task_struct *scst_mgmt_cmd_thread;
+
+static int suspend_count;
+
+static int scst_virt_dev_last_id; /* protected by scst_mutex */
+
+static unsigned int scst_max_cmd_mem;
+unsigned int scst_max_dev_cmd_mem;
+
+module_param_named(scst_threads, scst_threads, int, 0);
+MODULE_PARM_DESC(scst_threads, "SCSI target threads count");
+
+module_param_named(scst_max_cmd_mem, scst_max_cmd_mem, int, S_IRUGO);
+MODULE_PARM_DESC(scst_max_cmd_mem, "Maximum memory allowed to be consumed by "
+ "all SCSI commands of all devices at any given time in MB");
+
+module_param_named(scst_max_dev_cmd_mem, scst_max_dev_cmd_mem, int, S_IRUGO);
+MODULE_PARM_DESC(scst_max_dev_cmd_mem, "Maximum memory allowed to be consumed "
+ "by all SCSI commands of a device at any given time in MB");
+
+struct scst_dev_type scst_null_devtype = {
+ .name = "none",
+};
+
+static void __scst_resume_activity(void);
+
+/**
+ * __scst_register_target_template() - register target template.
+ * @vtt: target template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the target driver use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a target template and returns 0 on success or appropriate
+ * error code otherwise.
+ *
+ * Note: *vtt must be static!
+ */
+int __scst_register_target_template(struct scst_tgt_template *vtt,
+ const char *version)
+{
+ int res = 0;
+ struct scst_tgt_template *t;
+ static DEFINE_MUTEX(m);
+
+ INIT_LIST_HEAD(&vtt->tgt_list);
+
+ if (strcmp(version, SCST_INTERFACE_VERSION) != 0) {
+ PRINT_ERROR("Incorrect version of target %s", vtt->name);
+ res = -EINVAL;
+ goto out_err;
+ }
+
+ if (!vtt->detect) {
+ PRINT_ERROR("Target driver %s must have "
+ "detect() method.", vtt->name);
+ res = -EINVAL;
+ goto out_err;
+ }
+
+ if (!vtt->release) {
+ PRINT_ERROR("Target driver %s must have "
+ "release() method.", vtt->name);
+ res = -EINVAL;
+ goto out_err;
+ }
+
+ if (!vtt->xmit_response) {
+ PRINT_ERROR("Target driver %s must have "
+ "xmit_response() method.", vtt->name);
+ res = -EINVAL;
+ goto out_err;
+ }
+
+ if (vtt->threads_num < 0) {
+ PRINT_ERROR("Wrong threads_num value %d for "
+ "target \"%s\"", vtt->threads_num,
+ vtt->name);
+ res = -EINVAL;
+ goto out_err;
+ }
+
+ if (!vtt->enable_target || !vtt->is_target_enabled) {
+ PRINT_WARNING("Target driver %s doesn't have enable_target() "
+ "and/or is_target_enabled() method(s). This is unsafe "
+ "and can lead that initiators connected on the "
+ "initialization time can see an unexpected set of "
+ "devices or no devices at all!", vtt->name);
+ }
+
+ if (((vtt->add_target != NULL) && (vtt->del_target == NULL)) ||
+ ((vtt->add_target == NULL) && (vtt->del_target != NULL))) {
+ PRINT_ERROR("Target driver %s must either define both "
+ "add_target() and del_target(), or none.", vtt->name);
+ res = -EINVAL;
+ goto out_err;
+ }
+
+ res = scst_create_tgtt_sysfs(vtt);
+ if (res)
+ goto out_sysfs_err;
+
+ if (vtt->rdy_to_xfer == NULL)
+ vtt->rdy_to_xfer_atomic = 1;
+
+ if (mutex_lock_interruptible(&m) != 0)
+ goto out_sysfs_err;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0)
+ goto out_m_err;
+ list_for_each_entry(t, &scst_template_list, scst_template_list_entry) {
+ if (strcmp(t->name, vtt->name) == 0) {
+ PRINT_ERROR("Target driver %s already registered",
+ vtt->name);
+ mutex_unlock(&scst_mutex);
+ goto out_m_err;
+ }
+ }
+ mutex_unlock(&scst_mutex);
+
+ TRACE_DBG("%s", "Calling target driver's detect()");
+ res = vtt->detect(vtt);
+ TRACE_DBG("Target driver's detect() returned %d", res);
+ if (res < 0) {
+ PRINT_ERROR("%s", "The detect() routine failed");
+ res = -EINVAL;
+ goto out_m_err;
+ }
+
+ mutex_lock(&scst_mutex);
+ list_add_tail(&vtt->scst_template_list_entry, &scst_template_list);
+ mutex_unlock(&scst_mutex);
+
+ res = 0;
+
+ PRINT_INFO("Target template %s registered successfully", vtt->name);
+
+ mutex_unlock(&m);
+
+out:
+ return res;
+
+out_m_err:
+ mutex_unlock(&m);
+
+out_sysfs_err:
+ scst_tgtt_sysfs_put(vtt);
+
+out_err:
+ PRINT_ERROR("Failed to register target template %s", vtt->name);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(__scst_register_target_template);
+
+static int scst_check_non_gpl_target_template(struct scst_tgt_template *vtt)
+{
+ int res;
+
+ if (vtt->task_mgmt_affected_cmds_done || vtt->threads_num) {
+ PRINT_ERROR("Not allowed functionality in non-GPL version for "
+ "target template %s", vtt->name);
+ res = -EPERM;
+ goto out;
+ }
+
+ res = 0;
+
+out:
+ return res;
+}
+
+/**
+ * __scst_register_target_template_non_gpl() - register target template,
+ * non-GPL version
+ * @vtt: target template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the target driver use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a target template and returns 0 on success or appropriate
+ * error code otherwise.
+ *
+ * Note: *vtt must be static!
+ */
+int __scst_register_target_template_non_gpl(struct scst_tgt_template *vtt,
+ const char *version)
+{
+ int res;
+
+ res = scst_check_non_gpl_target_template(vtt);
+ if (res != 0)
+ goto out;
+
+ res = __scst_register_target_template(vtt, version);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(__scst_register_target_template_non_gpl);
+
+/**
+ * scst_unregister_target_template() - unregister target template
+ */
+void scst_unregister_target_template(struct scst_tgt_template *vtt)
+{
+ struct scst_tgt *tgt;
+ struct scst_tgt_template *t;
+ int found = 0;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(t, &scst_template_list, scst_template_list_entry) {
+ if (strcmp(t->name, vtt->name) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found) {
+ PRINT_ERROR("Target driver %s isn't registered", vtt->name);
+ goto out_err_up;
+ }
+
+restart:
+ list_for_each_entry(tgt, &vtt->tgt_list, tgt_list_entry) {
+ mutex_unlock(&scst_mutex);
+ scst_unregister_target(tgt);
+ mutex_lock(&scst_mutex);
+ goto restart;
+ }
+ list_del(&vtt->scst_template_list_entry);
+
+ mutex_unlock(&scst_mutex);
+
+ scst_tgtt_sysfs_put(vtt);
+
+ PRINT_INFO("Target template %s unregistered successfully", vtt->name);
+
+out:
+ return;
+
+out_err_up:
+ mutex_unlock(&scst_mutex);
+ goto out;
+}
+EXPORT_SYMBOL(scst_unregister_target_template);
+
+/**
+ * scst_register_target() - register target
+ *
+ * Registers a target for template vtt and returns new target structure on
+ * success or NULL otherwise.
+ */
+struct scst_tgt *scst_register_target(struct scst_tgt_template *vtt,
+ const char *target_name)
+{
+ struct scst_tgt *tgt;
+ int rc = 0;
+
+ rc = scst_alloc_tgt(vtt, &tgt);
+ if (rc != 0)
+ goto out_err;
+
+ rc = scst_suspend_activity(true);
+ if (rc != 0)
+ goto out_free_tgt_err;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ rc = -EINTR;
+ goto out_resume_free;
+ }
+
+ if (target_name != NULL) {
+
+ tgt->tgt_name = kmalloc(strlen(target_name) + 1, GFP_KERNEL);
+ if (tgt->tgt_name == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of tgt name %s failed",
+ target_name);
+ rc = -ENOMEM;
+ goto out_unlock_resume;
+ }
+ strcpy(tgt->tgt_name, target_name);
+ } else {
+ static int tgt_num; /* protected by scst_mutex */
+ int len = strlen(vtt->name) +
+ strlen(SCST_DEFAULT_TGT_NAME_SUFFIX) + 11 + 1;
+
+ tgt->tgt_name = kmalloc(len, GFP_KERNEL);
+ if (tgt->tgt_name == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of tgt name failed "
+ "(template name %s)", vtt->name);
+ rc = -ENOMEM;
+ goto out_unlock_resume;
+ }
+ sprintf(tgt->tgt_name, "%s%s%d", vtt->name,
+ SCST_DEFAULT_TGT_NAME_SUFFIX, tgt_num++);
+ }
+
+ tgt->default_acg = scst_alloc_add_acg(NULL, tgt->tgt_name);
+ if (tgt->default_acg == NULL)
+ goto out_free_tgt_name;
+
+ INIT_LIST_HEAD(&tgt->tgt_acg_list);
+
+ rc = scst_create_tgt_sysfs(tgt);
+ if (rc < 0)
+ goto out_clear_acg;
+
+ list_add_tail(&tgt->tgt_list_entry, &vtt->tgt_list);
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ PRINT_INFO("Target %s for template %s registered successfully",
+ tgt->tgt_name, vtt->name);
+
+ TRACE_DBG("tgt %p", tgt);
+
+out:
+ return tgt;
+
+out_clear_acg:
+ scst_clear_acg(tgt->default_acg);
+
+out_free_tgt_name:
+ kfree(tgt->tgt_name);
+
+out_unlock_resume:
+ mutex_unlock(&scst_mutex);
+
+out_resume_free:
+ scst_resume_activity();
+
+out_free_tgt_err:
+ scst_tgt_sysfs_put(tgt); /* must not be called under scst_mutex */
+ tgt = NULL;
+
+out_err:
+ PRINT_ERROR("Failed to register target %s for template %s (error %d)",
+ (tgt->tgt_name != NULL) ? tgt->tgt_name : target_name,
+ vtt->name, rc);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_register_target);
+
+/**
+ * scst_register_target_non_gpl() - register target, non-GPL version
+ *
+ * Registers a target for template vtt and returns new target structure on
+ * success or NULL otherwise.
+ */
+struct scst_tgt *scst_register_target_non_gpl(struct scst_tgt_template *vtt,
+ const char *target_name)
+{
+ struct scst_tgt *res;
+
+ if (scst_check_non_gpl_target_template(vtt)) {
+ res = NULL;
+ goto out;
+ }
+
+ res = scst_register_target(vtt, target_name);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_register_target_non_gpl);
+
+static inline int test_sess_list(struct scst_tgt *tgt)
+{
+ int res;
+ mutex_lock(&scst_mutex);
+ res = list_empty(&tgt->sess_list);
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+/**
+ * scst_unregister_target() - unregister target
+ */
+void scst_unregister_target(struct scst_tgt *tgt)
+{
+ struct scst_session *sess;
+ struct scst_tgt_template *vtt = tgt->tgtt;
+ struct scst_acg *acg, *acg_tmp;
+
+ scst_tgt_sysfs_prepare_put(tgt);
+
+ TRACE_DBG("%s", "Calling target driver's release()");
+ tgt->tgtt->release(tgt);
+ TRACE_DBG("%s", "Target driver's release() returned");
+
+ mutex_lock(&scst_mutex);
+again:
+ list_for_each_entry(sess, &tgt->sess_list, sess_list_entry) {
+ if (sess->shut_phase == SCST_SESS_SPH_READY) {
+ /*
+ * Sometimes it's hard for target driver to track all
+ * its sessions (see scst_local, for example), so let's
+ * help it.
+ */
+ mutex_unlock(&scst_mutex);
+ scst_unregister_session(sess, 0, NULL);
+ mutex_lock(&scst_mutex);
+ goto again;
+ }
+ }
+ mutex_unlock(&scst_mutex);
+
+ TRACE_DBG("%s", "Waiting for sessions shutdown");
+ wait_event(tgt->unreg_waitQ, test_sess_list(tgt));
+ TRACE_DBG("%s", "wait_event() returned");
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ list_del(&tgt->tgt_list_entry);
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ scst_clear_acg(tgt->default_acg);
+
+ list_for_each_entry_safe(acg, acg_tmp, &tgt->tgt_acg_list,
+ acg_list_entry) {
+ scst_acg_sysfs_put(acg);
+ }
+
+ del_timer_sync(&tgt->retry_timer);
+
+ PRINT_INFO("Target %s for template %s unregistered successfully",
+ tgt->tgt_name, vtt->name);
+
+ scst_tgt_sysfs_put(tgt); /* must not be called under scst_mutex */
+
+ TRACE_DBG("Unregistering tgt %p finished", tgt);
+ return;
+}
+EXPORT_SYMBOL(scst_unregister_target);
+
+static int scst_susp_wait(bool interruptible)
+{
+ int res = 0;
+
+ if (interruptible) {
+ res = wait_event_interruptible_timeout(scst_dev_cmd_waitQ,
+ (atomic_read(&scst_cmd_count) == 0),
+ SCST_SUSPENDING_TIMEOUT);
+ if (res <= 0) {
+ __scst_resume_activity();
+ if (res == 0)
+ res = -EBUSY;
+ } else
+ res = 0;
+ } else
+ wait_event(scst_dev_cmd_waitQ,
+ atomic_read(&scst_cmd_count) == 0);
+
+ TRACE_MGMT_DBG("wait_event() returned %d", res);
+ return res;
+}
+
+/**
+ * scst_suspend_activity() - globally suspend any activity
+ *
+ * Description:
+ * Globally suspends any activity and doesn't return, until there are any
+ * active commands (state after SCST_CMD_STATE_INIT). If "interruptible"
+ * is true, it returns after SCST_SUSPENDING_TIMEOUT or if it was interrupted
+ * by a signal with the corresponding error status < 0. If "interruptible"
+ * is false, it will wait virtually forever. On success returns 0.
+ *
+ * New arriving commands stay in the suspended state until
+ * scst_resume_activity() is called.
+ */
+int scst_suspend_activity(bool interruptible)
+{
+ int res = 0;
+ bool rep = false;
+
+ if (interruptible) {
+ if (mutex_lock_interruptible(&scst_suspend_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+ } else
+ mutex_lock(&scst_suspend_mutex);
+
+ TRACE_MGMT_DBG("suspend_count %d", suspend_count);
+ suspend_count++;
+ if (suspend_count > 1)
+ goto out_up;
+
+ set_bit(SCST_FLAG_SUSPENDING, &scst_flags);
+ set_bit(SCST_FLAG_SUSPENDED, &scst_flags);
+ /*
+ * Assignment of SCST_FLAG_SUSPENDING and SCST_FLAG_SUSPENDED must be
+ * ordered with scst_cmd_count. Otherwise lockless logic in
+ * scst_translate_lun() and scst_mgmt_translate_lun() won't work.
+ */
+ smp_mb__after_set_bit();
+
+ /*
+ * See comment in scst_user.c::dev_user_task_mgmt_fn() for more
+ * information about scst_user behavior.
+ *
+ * ToDo: make the global suspending unneeded (switch to per-device
+ * reference counting? That would mean to switch off from lockless
+ * implementation of scst_translate_lun().. )
+ */
+
+ if (atomic_read(&scst_cmd_count) != 0) {
+ PRINT_INFO("Waiting for %d active commands to complete... This "
+ "might take few minutes for disks or few hours for "
+ "tapes, if you use long executed commands, like "
+ "REWIND or FORMAT. In case, if you have a hung user "
+ "space device (i.e. made using scst_user module) not "
+ "responding to any commands, if might take virtually "
+ "forever until the corresponding user space "
+ "program recovers and starts responding or gets "
+ "killed.", atomic_read(&scst_cmd_count));
+ rep = true;
+ }
+
+ res = scst_susp_wait(interruptible);
+ if (res != 0)
+ goto out_clear;
+
+ clear_bit(SCST_FLAG_SUSPENDING, &scst_flags);
+ /* See comment about smp_mb() above */
+ smp_mb__after_clear_bit();
+
+ TRACE_MGMT_DBG("Waiting for %d active commands finally to complete",
+ atomic_read(&scst_cmd_count));
+
+ res = scst_susp_wait(interruptible);
+ if (res != 0)
+ goto out_clear;
+
+ if (rep)
+ PRINT_INFO("%s", "All active commands completed");
+
+out_up:
+ mutex_unlock(&scst_suspend_mutex);
+
+out:
+ return res;
+
+out_clear:
+ clear_bit(SCST_FLAG_SUSPENDING, &scst_flags);
+ /* See comment about smp_mb() above */
+ smp_mb__after_clear_bit();
+ goto out_up;
+}
+EXPORT_SYMBOL_GPL(scst_suspend_activity);
+
+static void __scst_resume_activity(void)
+{
+ struct scst_cmd_threads *l;
+
+ suspend_count--;
+ TRACE_MGMT_DBG("suspend_count %d left", suspend_count);
+ if (suspend_count > 0)
+ goto out;
+
+ clear_bit(SCST_FLAG_SUSPENDED, &scst_flags);
+ /*
+ * The barrier is needed to make sure all woken up threads see the
+ * cleared flag. Not sure if it's really needed, but let's be safe.
+ */
+ smp_mb__after_clear_bit();
+
+ list_for_each_entry(l, &scst_cmd_threads_list, lists_list_entry) {
+ wake_up_all(&l->cmd_list_waitQ);
+ }
+ wake_up_all(&scst_init_cmd_list_waitQ);
+
+ spin_lock_irq(&scst_mcmd_lock);
+ if (!list_empty(&scst_delayed_mgmt_cmd_list)) {
+ struct scst_mgmt_cmd *m;
+ m = list_entry(scst_delayed_mgmt_cmd_list.next, typeof(*m),
+ mgmt_cmd_list_entry);
+ TRACE_MGMT_DBG("Moving delayed mgmt cmd %p to head of active "
+ "mgmt cmd list", m);
+ list_move(&m->mgmt_cmd_list_entry, &scst_active_mgmt_cmd_list);
+ }
+ spin_unlock_irq(&scst_mcmd_lock);
+ wake_up_all(&scst_mgmt_cmd_list_waitQ);
+
+out:
+ return;
+}
+
+/**
+ * scst_resume_activity() - globally resume all activities
+ *
+ * Resumes suspended by scst_suspend_activity() activities.
+ */
+void scst_resume_activity(void)
+{
+
+ mutex_lock(&scst_suspend_mutex);
+ __scst_resume_activity();
+ mutex_unlock(&scst_suspend_mutex);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_resume_activity);
+
+static int scst_register_device(struct scsi_device *scsidp)
+{
+ int res = 0;
+ struct scst_device *dev, *d;
+ struct scst_dev_type *dt;
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out_err;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ res = scst_alloc_device(GFP_KERNEL, &dev);
+ if (res != 0)
+ goto out_up;
+
+ dev->type = scsidp->type;
+
+ dev->virt_name = kmalloc(50, GFP_KERNEL);
+ if (dev->virt_name == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc device name");
+ res = -ENOMEM;
+ goto out_free_dev;
+ }
+ snprintf(dev->virt_name, 50, "%d:%d:%d:%d", scsidp->host->host_no,
+ scsidp->channel, scsidp->id, scsidp->lun);
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (strcmp(d->virt_name, dev->virt_name) == 0) {
+ PRINT_ERROR("Device %s already exists", dev->virt_name);
+ res = -EEXIST;
+ goto out_free_dev;
+ }
+ }
+
+ dev->scsi_dev = scsidp;
+
+ list_add_tail(&dev->dev_list_entry, &scst_dev_list);
+
+ res = scst_create_device_sysfs(dev);
+ if (res != 0)
+ goto out_free;
+
+ list_for_each_entry(dt, &scst_dev_type_list, dev_type_list_entry) {
+ if (dt->type == scsidp->type) {
+ res = scst_assign_dev_handler(dev, dt);
+ if (res != 0)
+ goto out_free;
+ break;
+ }
+ }
+
+out_up:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out_err:
+ if (res == 0) {
+ PRINT_INFO("Attached to scsi%d, channel %d, id %d, lun %d, "
+ "type %d", scsidp->host->host_no, scsidp->channel,
+ scsidp->id, scsidp->lun, scsidp->type);
+ } else {
+ PRINT_ERROR("Failed to attach to scsi%d, channel %d, id %d, "
+ "lun %d, type %d", scsidp->host->host_no,
+ scsidp->channel, scsidp->id, scsidp->lun, scsidp->type);
+ }
+ return res;
+
+out_free:
+ list_del(&dev->dev_list_entry);
+
+out_free_dev:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+ scst_device_sysfs_put(dev); /* must not be called under scst_mutex */
+ goto out_err;
+}
+
+static void scst_unregister_device(struct scsi_device *scsidp)
+{
+ struct scst_device *d, *dev = NULL;
+ struct scst_acg_dev *acg_dev, *aa;
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (d->scsi_dev == scsidp) {
+ dev = d;
+ TRACE_DBG("Target device %p found", dev);
+ break;
+ }
+ }
+ if (dev == NULL) {
+ PRINT_ERROR("%s", "Target device not found");
+ goto out_resume;
+ }
+
+ list_del(&dev->dev_list_entry);
+
+ list_for_each_entry_safe(acg_dev, aa, &dev->dev_acg_dev_list,
+ dev_acg_dev_list_entry) {
+ scst_acg_remove_dev(acg_dev->acg, dev, true);
+ }
+
+ scst_assign_dev_handler(dev, &scst_null_devtype);
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ scst_device_sysfs_put(dev); /* must not be called under scst_mutex */
+
+ PRINT_INFO("Detached from scsi%d, channel %d, id %d, lun %d, type %d",
+ scsidp->host->host_no, scsidp->channel, scsidp->id,
+ scsidp->lun, scsidp->type);
+
+out:
+ return;
+
+out_resume:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+ goto out;
+}
+
+static int scst_dev_handler_check(struct scst_dev_type *dev_handler)
+{
+ int res = 0;
+
+ if (dev_handler->parse == NULL) {
+ PRINT_ERROR("scst dev handler %s must have "
+ "parse() method.", dev_handler->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (((dev_handler->add_device != NULL) &&
+ (dev_handler->del_device == NULL)) ||
+ ((dev_handler->add_device == NULL) &&
+ (dev_handler->del_device != NULL))) {
+ PRINT_ERROR("Dev handler %s must either define both "
+ "add_device() and del_device(), or none.",
+ dev_handler->name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (dev_handler->exec == NULL) {
+#ifdef CONFIG_SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ
+ dev_handler->exec_atomic = 1;
+#else
+ dev_handler->exec_atomic = 0;
+#endif
+ }
+
+ if (dev_handler->dev_done == NULL)
+ dev_handler->dev_done_atomic = 1;
+
+out:
+ return res;
+}
+
+/**
+ * scst_register_virtual_device() - register a virtual device.
+ * @dev_handler: the device's device handler
+ * @dev_name: the new device name, NULL-terminated string. Must be uniq
+ * among all virtual devices in the system.
+ *
+ * Registers a virtual device and returns assinged to the device ID on
+ * success, or negative value otherwise
+ */
+int scst_register_virtual_device(struct scst_dev_type *dev_handler,
+ const char *dev_name)
+{
+ int res, rc;
+ struct scst_device *dev;
+
+ if (dev_handler == NULL) {
+ PRINT_ERROR("%s: valid device handler must be supplied",
+ __func__);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (dev_name == NULL) {
+ PRINT_ERROR("%s: device name must be non-NULL", __func__);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_dev_handler_check(dev_handler);
+ if (res != 0)
+ goto out;
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ if (strcmp(dev->virt_name, dev_name) == 0) {
+ PRINT_ERROR("Device %s already exists", dev_name);
+ res = -EEXIST;
+ goto out;
+ }
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ res = scst_alloc_device(GFP_KERNEL, &dev);
+ if (res != 0)
+ goto out_up;
+
+ dev->type = dev_handler->type;
+ dev->scsi_dev = NULL;
+ dev->virt_name = kstrdup(dev_name, GFP_KERNEL);
+ if (dev->virt_name == NULL) {
+ PRINT_ERROR("Unable to allocate virt_name for dev %s",
+ dev_name);
+ res = -ENOMEM;
+ goto out_release;
+ }
+ dev->virt_id = scst_virt_dev_last_id++;
+
+ list_add_tail(&dev->dev_list_entry, &scst_dev_list);
+
+ res = dev->virt_id;
+
+ rc = scst_create_device_sysfs(dev);
+ if (rc != 0) {
+ res = rc;
+ goto out_free_del;
+ }
+
+ rc = scst_assign_dev_handler(dev, dev_handler);
+ if (rc != 0) {
+ res = rc;
+ goto out_free_del;
+ }
+
+out_up:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ if (res > 0)
+ PRINT_INFO("Attached to virtual device %s (id %d)",
+ dev_name, dev->virt_id);
+ else
+ PRINT_INFO("Failed to attach to virtual device %s", dev_name);
+ return res;
+
+out_free_del:
+ list_del(&dev->dev_list_entry);
+
+out_release:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+ scst_device_sysfs_put(dev); /* must not be called under scst_mutex */
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_register_virtual_device);
+
+/**
+ * scst_unregister_virtual_device() - unegister a virtual device.
+ * @id: the device's ID, returned by the registration function
+ */
+void scst_unregister_virtual_device(int id)
+{
+ struct scst_device *d, *dev = NULL;
+ struct scst_acg_dev *acg_dev, *aa;
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (d->virt_id == id) {
+ dev = d;
+ TRACE_DBG("Target device %p (id %d) found", dev, id);
+ break;
+ }
+ }
+ if (dev == NULL) {
+ PRINT_ERROR("Target virtual device (id %d) not found", id);
+ goto out_unblock;
+ }
+
+ list_del(&dev->dev_list_entry);
+
+ list_for_each_entry_safe(acg_dev, aa, &dev->dev_acg_dev_list,
+ dev_acg_dev_list_entry) {
+ scst_acg_remove_dev(acg_dev->acg, dev, true);
+ }
+
+ scst_assign_dev_handler(dev, &scst_null_devtype);
+
+ PRINT_INFO("Detached from virtual device %s (id %d)",
+ dev->virt_name, dev->virt_id);
+
+out_unblock:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ scst_device_sysfs_put(dev); /* must not be called under scst_mutex */
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_virtual_device);
+
+/**
+ * __scst_register_dev_driver() - register pass-through dev handler driver
+ * @dev_type: dev handler template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the dev handler use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a pass-through dev handler driver. Returns 0 on success
+ * or appropriate error code otherwise.
+ *
+ * Note: *dev_type must be static!
+ */
+int __scst_register_dev_driver(struct scst_dev_type *dev_type,
+ const char *version)
+{
+ struct scst_dev_type *dt;
+ struct scst_device *dev;
+ int res;
+ int exist;
+
+ if (strcmp(version, SCST_INTERFACE_VERSION) != 0) {
+ PRINT_ERROR("Incorrect version of dev handler %s",
+ dev_type->name);
+ res = -EINVAL;
+ goto out_error;
+ }
+
+ res = scst_dev_handler_check(dev_type);
+ if (res != 0)
+ goto out_error;
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out_error;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_err_res;
+ }
+
+ exist = 0;
+ list_for_each_entry(dt, &scst_dev_type_list, dev_type_list_entry) {
+ if (strcmp(dt->name, dev_type->name) == 0) {
+ PRINT_ERROR("Device type handler \"%s\" already "
+ "exist", dt->name);
+ exist = 1;
+ break;
+ }
+ }
+ if (exist)
+ goto out_up;
+
+ res = scst_create_devt_sysfs(dev_type);
+ if (res < 0)
+ goto out_free;
+
+ list_add_tail(&dev_type->dev_type_list_entry, &scst_dev_type_list);
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ if (dev->scsi_dev == NULL || dev->handler != &scst_null_devtype)
+ continue;
+ if (dev->scsi_dev->type == dev_type->type)
+ scst_assign_dev_handler(dev, dev_type);
+ }
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ if (res == 0) {
+ PRINT_INFO("Device handler \"%s\" for type %d registered "
+ "successfully", dev_type->name, dev_type->type);
+ }
+
+out:
+ return res;
+
+out_free:
+ scst_devt_sysfs_put(dev_type);
+
+out_up:
+ mutex_unlock(&scst_mutex);
+
+out_err_res:
+ scst_resume_activity();
+
+out_error:
+ PRINT_ERROR("Failed to register device handler \"%s\" for type %d",
+ dev_type->name, dev_type->type);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(__scst_register_dev_driver);
+
+/**
+ * scst_unregister_dev_driver() - unregister pass-through dev handler driver
+ */
+void scst_unregister_dev_driver(struct scst_dev_type *dev_type)
+{
+ struct scst_device *dev;
+ struct scst_dev_type *dt;
+ int found = 0;
+
+ scst_suspend_activity(false);
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(dt, &scst_dev_type_list, dev_type_list_entry) {
+ if (strcmp(dt->name, dev_type->name) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found) {
+ PRINT_ERROR("Dev handler \"%s\" isn't registered",
+ dev_type->name);
+ goto out_up;
+ }
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ if (dev->handler == dev_type) {
+ scst_assign_dev_handler(dev, &scst_null_devtype);
+ TRACE_DBG("Dev handler removed from device %p", dev);
+ }
+ }
+
+ list_del(&dev_type->dev_type_list_entry);
+
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+
+ scst_devt_sysfs_put(dev_type);
+
+ PRINT_INFO("Device handler \"%s\" for type %d unloaded",
+ dev_type->name, dev_type->type);
+
+out:
+ return;
+
+out_up:
+ mutex_unlock(&scst_mutex);
+ scst_resume_activity();
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_dev_driver);
+
+/**
+ * __scst_register_virtual_dev_driver() - register virtual dev handler driver
+ * @dev_type: dev handler template
+ * @version: SCST_INTERFACE_VERSION version string to ensure that
+ * SCST core and the dev handler use the same version of
+ * the SCST interface
+ *
+ * Description:
+ * Registers a virtual dev handler driver. Returns 0 on success or
+ * appropriate error code otherwise.
+ *
+ * Note: *dev_type must be static!
+ */
+int __scst_register_virtual_dev_driver(struct scst_dev_type *dev_type,
+ const char *version)
+{
+ int res;
+
+ if (strcmp(version, SCST_INTERFACE_VERSION) != 0) {
+ PRINT_ERROR("Incorrect version of virtual dev handler %s",
+ dev_type->name);
+ res = -EINVAL;
+ goto out_err;
+ }
+
+ res = scst_dev_handler_check(dev_type);
+ if (res != 0)
+ goto out_err;
+
+ res = scst_create_devt_sysfs(dev_type);
+ if (res < 0)
+ goto out_free;
+
+ if (dev_type->type != -1) {
+ PRINT_INFO("Virtual device handler %s for type %d "
+ "registered successfully", dev_type->name,
+ dev_type->type);
+ } else {
+ PRINT_INFO("Virtual device handler \"%s\" registered "
+ "successfully", dev_type->name);
+ }
+
+out:
+ return res;
+
+out_free:
+
+ scst_devt_sysfs_put(dev_type);
+
+out_err:
+ PRINT_ERROR("Failed to register virtual device handler \"%s\"",
+ dev_type->name);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(__scst_register_virtual_dev_driver);
+
+/**
+ * scst_unregister_virtual_dev_driver() - unregister virtual dev driver
+ */
+void scst_unregister_virtual_dev_driver(struct scst_dev_type *dev_type)
+{
+
+ scst_devt_sysfs_put(dev_type);
+
+ PRINT_INFO("Device handler \"%s\" unloaded", dev_type->name);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_virtual_dev_driver);
+
+/* scst_mutex supposed to be held */
+int scst_add_threads(struct scst_cmd_threads *cmd_threads,
+ struct scst_device *dev, struct scst_tgt_dev *tgt_dev, int num)
+{
+ int res, i;
+ struct scst_cmd_thread_t *thr;
+ int n = 0, tgt_dev_num = 0;
+
+ list_for_each_entry(thr, &cmd_threads->threads_list, thread_list_entry) {
+ n++;
+ }
+
+ if (tgt_dev != NULL) {
+ struct scst_tgt_dev *t;
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (t == tgt_dev)
+ break;
+ tgt_dev_num++;
+ }
+ }
+
+ for (i = 0; i < num; i++) {
+ struct scst_cmd_thread_t *thr;
+
+ thr = kmalloc(sizeof(*thr), GFP_KERNEL);
+ if (!thr) {
+ res = -ENOMEM;
+ PRINT_ERROR("fail to allocate thr %d", res);
+ goto out_error;
+ }
+
+ if (dev != NULL) {
+ char nm[14]; /* to limit the name's len */
+ strlcpy(nm, dev->virt_name, ARRAY_SIZE(nm));
+ thr->cmd_thread = kthread_create(scst_cmd_thread,
+ cmd_threads, "%s%d", nm, n++);
+ } else if (tgt_dev != NULL) {
+ char nm[11]; /* to limit the name's len */
+ strlcpy(nm, tgt_dev->dev->virt_name, ARRAY_SIZE(nm));
+ thr->cmd_thread = kthread_create(scst_cmd_thread,
+ cmd_threads, "%s%d_%d", nm, tgt_dev_num, n++);
+ } else
+ thr->cmd_thread = kthread_create(scst_cmd_thread,
+ cmd_threads, "scsi_tgt%d", n++);
+
+ if (IS_ERR(thr->cmd_thread)) {
+ res = PTR_ERR(thr->cmd_thread);
+ PRINT_ERROR("kthread_create() failed: %d", res);
+ kfree(thr);
+ goto out_error;
+ }
+
+ list_add(&thr->thread_list_entry, &cmd_threads->threads_list);
+ cmd_threads->nr_threads++;
+
+ wake_up_process(thr->cmd_thread);
+ }
+
+ res = 0;
+
+out:
+ return res;
+
+out_error:
+ scst_del_threads(cmd_threads, i);
+ goto out;
+}
+
+/* scst_mutex supposed to be held */
+void scst_del_threads(struct scst_cmd_threads *cmd_threads, int num)
+{
+ struct scst_cmd_thread_t *ct, *tmp;
+
+ if (num == 0)
+ goto out;
+
+ list_for_each_entry_safe_reverse(ct, tmp, &cmd_threads->threads_list,
+ thread_list_entry) {
+ int rc;
+ struct scst_device *dev;
+
+ rc = kthread_stop(ct->cmd_thread);
+ if (rc < 0)
+ TRACE_MGMT_DBG("kthread_stop() failed: %d", rc);
+
+ list_del(&ct->thread_list_entry);
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_del_thr_data(tgt_dev, ct->cmd_thread);
+ }
+ }
+
+ kfree(ct);
+
+ cmd_threads->nr_threads--;
+
+ --num;
+ if (num == 0)
+ break;
+ }
+
+ EXTRACHECKS_BUG_ON((cmd_threads->nr_threads == 0) &&
+ (cmd_threads->io_context != NULL));
+
+out:
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_stop_dev_threads(struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_tgt_dev_stop_threads(tgt_dev);
+ }
+
+ if ((dev->threads_num > 0) &&
+ (dev->threads_pool_type == SCST_THREADS_POOL_SHARED))
+ scst_del_threads(&dev->dev_cmd_threads, -1);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_create_dev_threads(struct scst_device *dev)
+{
+ int res = 0;
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ res = scst_tgt_dev_setup_threads(tgt_dev);
+ if (res != 0)
+ goto out_err;
+ }
+
+ if ((dev->threads_num > 0) &&
+ (dev->threads_pool_type == SCST_THREADS_POOL_SHARED)) {
+ res = scst_add_threads(&dev->dev_cmd_threads, dev, NULL,
+ dev->threads_num);
+ if (res != 0)
+ goto out_err;
+ }
+
+out:
+ return res;
+
+out_err:
+ scst_stop_dev_threads(dev);
+ goto out;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_assign_dev_handler(struct scst_device *dev,
+ struct scst_dev_type *handler)
+{
+ int res = 0;
+ struct scst_tgt_dev *tgt_dev;
+ LIST_HEAD(attached_tgt_devs);
+
+ BUG_ON(handler == NULL);
+
+ if (dev->handler == handler)
+ goto out;
+
+ if (dev->handler == NULL)
+ goto assign;
+
+ if (dev->handler->detach_tgt) {
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ TRACE_DBG("Calling dev handler's detach_tgt(%p)",
+ tgt_dev);
+ dev->handler->detach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's detach_tgt() returned");
+ }
+ }
+
+ if (dev->handler->detach) {
+ TRACE_DBG("%s", "Calling dev handler's detach()");
+ dev->handler->detach(dev);
+ TRACE_DBG("%s", "Old handler's detach() returned");
+ }
+
+ scst_stop_dev_threads(dev);
+
+ scst_devt_dev_sysfs_put(dev);
+
+assign:
+ dev->handler = handler;
+
+ if (handler == NULL)
+ goto out;
+
+ dev->threads_num = handler->threads_num;
+ dev->threads_pool_type = handler->threads_pool_type;
+
+ res = scst_create_devt_dev_sysfs(dev);
+ if (res != 0)
+ goto out_null;
+
+ if (handler->attach) {
+ TRACE_DBG("Calling new dev handler's attach(%p)", dev);
+ res = handler->attach(dev);
+ TRACE_DBG("New dev handler's attach() returned %d", res);
+ if (res != 0) {
+ PRINT_ERROR("New device handler's %s attach() "
+ "failed: %d", handler->name, res);
+ goto out_remove_sysfs;
+ }
+ }
+
+ if (handler->attach_tgt) {
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ TRACE_DBG("Calling dev handler's attach_tgt(%p)",
+ tgt_dev);
+ res = handler->attach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's attach_tgt() returned");
+ if (res != 0) {
+ PRINT_ERROR("Device handler's %s attach_tgt() "
+ "failed: %d", handler->name, res);
+ goto out_err_detach_tgt;
+ }
+ list_add_tail(&tgt_dev->extra_tgt_dev_list_entry,
+ &attached_tgt_devs);
+ }
+ }
+
+ res = scst_create_dev_threads(dev);
+ if (res != 0)
+ goto out_err_detach_tgt;
+
+out:
+ return res;
+
+out_err_detach_tgt:
+ if (handler && handler->detach_tgt) {
+ list_for_each_entry(tgt_dev, &attached_tgt_devs,
+ extra_tgt_dev_list_entry) {
+ TRACE_DBG("Calling handler's detach_tgt(%p)",
+ tgt_dev);
+ handler->detach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Handler's detach_tgt() returned");
+ }
+ }
+ if (handler && handler->detach) {
+ TRACE_DBG("%s", "Calling handler's detach()");
+ handler->detach(dev);
+ TRACE_DBG("%s", "Handler's detach() returned");
+ }
+
+out_remove_sysfs:
+ scst_devt_dev_sysfs_put(dev);
+
+out_null:
+ dev->handler = &scst_null_devtype;
+ goto out;
+}
+
+/**
+ * scst_init_threads() - initialize SCST processing threads pool
+ *
+ * Initializes scst_cmd_threads structure
+ */
+void scst_init_threads(struct scst_cmd_threads *cmd_threads)
+{
+
+ spin_lock_init(&cmd_threads->cmd_list_lock);
+ INIT_LIST_HEAD(&cmd_threads->active_cmd_list);
+ init_waitqueue_head(&cmd_threads->cmd_list_waitQ);
+ INIT_LIST_HEAD(&cmd_threads->threads_list);
+
+ mutex_lock(&scst_suspend_mutex);
+ list_add_tail(&cmd_threads->lists_list_entry,
+ &scst_cmd_threads_list);
+ mutex_unlock(&scst_suspend_mutex);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_init_threads);
+
+/**
+ * scst_deinit_threads() - deinitialize SCST processing threads pool
+ *
+ * Deinitializes scst_cmd_threads structure
+ */
+void scst_deinit_threads(struct scst_cmd_threads *cmd_threads)
+{
+
+ mutex_lock(&scst_suspend_mutex);
+ list_del(&cmd_threads->lists_list_entry);
+ mutex_unlock(&scst_suspend_mutex);
+
+ BUG_ON(cmd_threads->io_context);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_deinit_threads);
+
+static void scst_stop_all_threads(void)
+{
+
+ mutex_lock(&scst_mutex);
+
+ scst_del_threads(&scst_main_cmd_threads, -1);
+
+ if (scst_mgmt_cmd_thread)
+ kthread_stop(scst_mgmt_cmd_thread);
+ if (scst_mgmt_thread)
+ kthread_stop(scst_mgmt_thread);
+ if (scst_init_cmd_thread)
+ kthread_stop(scst_init_cmd_thread);
+
+ mutex_unlock(&scst_mutex);
+ return;
+}
+
+static int scst_start_all_threads(int num)
+{
+ int res;
+
+ mutex_lock(&scst_mutex);
+
+ res = scst_add_threads(&scst_main_cmd_threads, NULL, NULL, num);
+ if (res < 0)
+ goto out_unlock;
+
+ scst_init_cmd_thread = kthread_run(scst_init_thread,
+ NULL, "scsi_tgt_init");
+ if (IS_ERR(scst_init_cmd_thread)) {
+ res = PTR_ERR(scst_init_cmd_thread);
+ PRINT_ERROR("kthread_create() for init cmd failed: %d", res);
+ scst_init_cmd_thread = NULL;
+ goto out_unlock;
+ }
+
+ scst_mgmt_cmd_thread = kthread_run(scst_tm_thread,
+ NULL, "scsi_tm");
+ if (IS_ERR(scst_mgmt_cmd_thread)) {
+ res = PTR_ERR(scst_mgmt_cmd_thread);
+ PRINT_ERROR("kthread_create() for TM failed: %d", res);
+ scst_mgmt_cmd_thread = NULL;
+ goto out_unlock;
+ }
+
+ scst_mgmt_thread = kthread_run(scst_global_mgmt_thread,
+ NULL, "scsi_tgt_mgmt");
+ if (IS_ERR(scst_mgmt_thread)) {
+ res = PTR_ERR(scst_mgmt_thread);
+ PRINT_ERROR("kthread_create() for mgmt failed: %d", res);
+ scst_mgmt_thread = NULL;
+ goto out_unlock;
+ }
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+/**
+ * scst_get() - increase global SCST ref counter
+ *
+ * Increases global SCST ref counter that prevents from entering into suspended
+ * activities stage, so protects from any global management operations.
+ */
+void scst_get(void)
+{
+ __scst_get(0);
+}
+EXPORT_SYMBOL_GPL(scst_get);
+
+/**
+ * scst_put() - decrease global SCST ref counter
+ *
+ * Decreses global SCST ref counter that prevents from entering into suspended
+ * activities stage, so protects from any global management operations. On
+ * zero, if suspending activities is waiting, they will be suspended.
+ */
+void scst_put(void)
+{
+ __scst_put();
+}
+EXPORT_SYMBOL_GPL(scst_put);
+
+/**
+ * scst_get_setup_id() - return SCST setup ID
+ *
+ * Returns SCST setup ID. This ID can be used for multiple
+ * setups with the same configuration.
+ */
+unsigned int scst_get_setup_id(void)
+{
+ return scst_setup_id;
+}
+EXPORT_SYMBOL_GPL(scst_get_setup_id);
+
+static int scst_add(struct device *cdev, struct class_interface *intf)
+{
+ struct scsi_device *scsidp;
+ int res = 0;
+
+ scsidp = to_scsi_device(cdev->parent);
+
+ if (strcmp(scsidp->host->hostt->name, SCST_LOCAL_NAME) != 0)
+ res = scst_register_device(scsidp);
+ return res;
+}
+
+static void scst_remove(struct device *cdev, struct class_interface *intf)
+{
+ struct scsi_device *scsidp;
+
+ scsidp = to_scsi_device(cdev->parent);
+
+ if (strcmp(scsidp->host->hostt->name, SCST_LOCAL_NAME) != 0)
+ scst_unregister_device(scsidp);
+ return;
+}
+
+static struct class_interface scst_interface = {
+ .add_dev = scst_add,
+ .remove_dev = scst_remove,
+};
+
+static void __init scst_print_config(void)
+{
+ char buf[128];
+ int i, j;
+
+ i = snprintf(buf, sizeof(buf), "Enabled features: ");
+ j = i;
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ i += snprintf(&buf[i], sizeof(buf) - i, "STRICT_SERIALIZING");
+#endif
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sEXTRACHECKS",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_TRACING
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sTRACING",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_TM
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_TM",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_RETRY",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_OOM
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_OOM",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_SN
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sDEBUG_SN",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sUSE_EXPECTED_VALUES",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ
+ i += snprintf(&buf[i], sizeof(buf) - i,
+ "%sALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ",
+ (j == i) ? "" : ", ");
+#endif
+
+#ifdef CONFIG_SCST_STRICT_SECURITY
+ i += snprintf(&buf[i], sizeof(buf) - i, "%sSCST_STRICT_SECURITY",
+ (j == i) ? "" : ", ");
+#endif
+
+ if (j != i)
+ PRINT_INFO("%s", buf);
+}
+
+static int __init init_scst(void)
+{
+ int res, i;
+ int scst_num_cpus;
+
+ {
+ struct scsi_sense_hdr *shdr;
+ BUILD_BUG_ON(SCST_SENSE_BUFFERSIZE < sizeof(*shdr));
+ }
+ {
+ struct scst_tgt_dev *t;
+ struct scst_cmd *c;
+ BUILD_BUG_ON(sizeof(t->curr_sn) != sizeof(t->expected_sn));
+ BUILD_BUG_ON(sizeof(c->sn) != sizeof(t->expected_sn));
+ }
+
+ mutex_init(&scst_mutex);
+ INIT_LIST_HEAD(&scst_template_list);
+ INIT_LIST_HEAD(&scst_dev_list);
+ INIT_LIST_HEAD(&scst_dev_type_list);
+ spin_lock_init(&scst_main_lock);
+ spin_lock_init(&scst_init_lock);
+ init_waitqueue_head(&scst_init_cmd_list_waitQ);
+ INIT_LIST_HEAD(&scst_init_cmd_list);
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ scst_trace_flag = SCST_DEFAULT_LOG_FLAGS;
+#endif
+ atomic_set(&scst_cmd_count, 0);
+ spin_lock_init(&scst_mcmd_lock);
+ INIT_LIST_HEAD(&scst_active_mgmt_cmd_list);
+ INIT_LIST_HEAD(&scst_delayed_mgmt_cmd_list);
+ init_waitqueue_head(&scst_mgmt_cmd_list_waitQ);
+ init_waitqueue_head(&scst_mgmt_waitQ);
+ spin_lock_init(&scst_mgmt_lock);
+ INIT_LIST_HEAD(&scst_sess_init_list);
+ INIT_LIST_HEAD(&scst_sess_shut_list);
+ init_waitqueue_head(&scst_dev_cmd_waitQ);
+ mutex_init(&scst_suspend_mutex);
+ INIT_LIST_HEAD(&scst_cmd_threads_list);
+ scst_virt_dev_last_id = 1;
+
+ scst_init_threads(&scst_main_cmd_threads);
+
+ res = scst_lib_init();
+ if (res != 0)
+ goto out;
+
+ scst_num_cpus = num_online_cpus();
+
+ /* ToDo: register_cpu_notifier() */
+
+ if (scst_threads == 0)
+ scst_threads = scst_num_cpus;
+
+ if (scst_threads < 1) {
+ PRINT_ERROR("%s", "scst_threads can not be less than 1");
+ scst_threads = scst_num_cpus;
+ }
+
+#define INIT_CACHEP(p, s, o) do { \
+ p = KMEM_CACHE(s, SCST_SLAB_FLAGS); \
+ TRACE_MEM("Slab create: %s at %p size %zd", #s, p, \
+ sizeof(struct s)); \
+ if (p == NULL) { \
+ res = -ENOMEM; \
+ goto o; \
+ } \
+ } while (0)
+
+ INIT_CACHEP(scst_mgmt_cachep, scst_mgmt_cmd, out_lib_exit);
+ INIT_CACHEP(scst_mgmt_stub_cachep, scst_mgmt_cmd_stub,
+ out_destroy_mgmt_cache);
+ INIT_CACHEP(scst_ua_cachep, scst_tgt_dev_UA,
+ out_destroy_mgmt_stub_cache);
+ {
+ struct scst_sense { uint8_t s[SCST_SENSE_BUFFERSIZE]; };
+ INIT_CACHEP(scst_sense_cachep, scst_sense,
+ out_destroy_ua_cache);
+ }
+ INIT_CACHEP(scst_aen_cachep, scst_aen, out_destroy_sense_cache);
+ INIT_CACHEP(scst_cmd_cachep, scst_cmd, out_destroy_aen_cache);
+ INIT_CACHEP(scst_sess_cachep, scst_session, out_destroy_cmd_cache);
+ INIT_CACHEP(scst_tgtd_cachep, scst_tgt_dev, out_destroy_sess_cache);
+ INIT_CACHEP(scst_acgd_cachep, scst_acg_dev, out_destroy_tgt_cache);
+
+ scst_mgmt_mempool = mempool_create(64, mempool_alloc_slab,
+ mempool_free_slab, scst_mgmt_cachep);
+ if (scst_mgmt_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_acg_cache;
+ }
+
+ /*
+ * All mgmt stubs, UAs and sense buffers are bursty and loosing them
+ * may have fatal consequences, so let's have big pools for them.
+ */
+
+ scst_mgmt_stub_mempool = mempool_create(1024, mempool_alloc_slab,
+ mempool_free_slab, scst_mgmt_stub_cachep);
+ if (scst_mgmt_stub_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_mgmt_mempool;
+ }
+
+ scst_ua_mempool = mempool_create(512, mempool_alloc_slab,
+ mempool_free_slab, scst_ua_cachep);
+ if (scst_ua_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_mgmt_stub_mempool;
+ }
+
+ scst_sense_mempool = mempool_create(1024, mempool_alloc_slab,
+ mempool_free_slab, scst_sense_cachep);
+ if (scst_sense_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_ua_mempool;
+ }
+
+ scst_aen_mempool = mempool_create(100, mempool_alloc_slab,
+ mempool_free_slab, scst_aen_cachep);
+ if (scst_aen_mempool == NULL) {
+ res = -ENOMEM;
+ goto out_destroy_sense_mempool;
+ }
+
+ res = scst_sysfs_init();
+ if (res != 0)
+ goto out_destroy_aen_mempool;
+
+ if (scst_max_cmd_mem == 0) {
+ struct sysinfo si;
+ si_meminfo(&si);
+#if BITS_PER_LONG == 32
+ scst_max_cmd_mem = min(
+ (((uint64_t)(si.totalram - si.totalhigh) << PAGE_SHIFT)
+ >> 20) >> 2, (uint64_t)1 << 30);
+#else
+ scst_max_cmd_mem = (((si.totalram - si.totalhigh) << PAGE_SHIFT)
+ >> 20) >> 2;
+#endif
+ }
+
+ if (scst_max_dev_cmd_mem != 0) {
+ if (scst_max_dev_cmd_mem > scst_max_cmd_mem) {
+ PRINT_ERROR("scst_max_dev_cmd_mem (%d) > "
+ "scst_max_cmd_mem (%d)",
+ scst_max_dev_cmd_mem,
+ scst_max_cmd_mem);
+ scst_max_dev_cmd_mem = scst_max_cmd_mem;
+ }
+ } else
+ scst_max_dev_cmd_mem = scst_max_cmd_mem * 2 / 5;
+
+ res = scst_sgv_pools_init(
+ ((uint64_t)scst_max_cmd_mem << 10) >> (PAGE_SHIFT - 10), 0);
+ if (res != 0)
+ goto out_sysfs_cleanup;
+
+ res = scsi_register_interface(&scst_interface);
+ if (res != 0)
+ goto out_destroy_sgv_pool;
+
+ for (i = 0; i < (int)ARRAY_SIZE(scst_tasklets); i++) {
+ spin_lock_init(&scst_tasklets[i].tasklet_lock);
+ INIT_LIST_HEAD(&scst_tasklets[i].tasklet_cmd_list);
+ tasklet_init(&scst_tasklets[i].tasklet,
+ (void *)scst_cmd_tasklet,
+ (unsigned long)&scst_tasklets[i]);
+ }
+
+ TRACE_DBG("%d CPUs found, starting %d threads", scst_num_cpus,
+ scst_threads);
+
+ res = scst_start_all_threads(scst_threads);
+ if (res < 0)
+ goto out_thread_free;
+
+ PRINT_INFO("SCST version %s loaded successfully (max mem for "
+ "commands %dMB, per device %dMB)", SCST_VERSION_STRING,
+ scst_max_cmd_mem, scst_max_dev_cmd_mem);
+
+ scst_print_config();
+
+out:
+ return res;
+
+out_thread_free:
+ scst_stop_all_threads();
+
+ scsi_unregister_interface(&scst_interface);
+
+out_destroy_sgv_pool:
+ scst_sgv_pools_deinit();
+
+out_sysfs_cleanup:
+ scst_sysfs_cleanup();
+
+out_destroy_aen_mempool:
+ mempool_destroy(scst_aen_mempool);
+
+out_destroy_sense_mempool:
+ mempool_destroy(scst_sense_mempool);
+
+out_destroy_ua_mempool:
+ mempool_destroy(scst_ua_mempool);
+
+out_destroy_mgmt_stub_mempool:
+ mempool_destroy(scst_mgmt_stub_mempool);
+
+out_destroy_mgmt_mempool:
+ mempool_destroy(scst_mgmt_mempool);
+
+out_destroy_acg_cache:
+ kmem_cache_destroy(scst_acgd_cachep);
+
+out_destroy_tgt_cache:
+ kmem_cache_destroy(scst_tgtd_cachep);
+
+out_destroy_sess_cache:
+ kmem_cache_destroy(scst_sess_cachep);
+
+out_destroy_cmd_cache:
+ kmem_cache_destroy(scst_cmd_cachep);
+
+out_destroy_aen_cache:
+ kmem_cache_destroy(scst_aen_cachep);
+
+out_destroy_sense_cache:
+ kmem_cache_destroy(scst_sense_cachep);
+
+out_destroy_ua_cache:
+ kmem_cache_destroy(scst_ua_cachep);
+
+out_destroy_mgmt_stub_cache:
+ kmem_cache_destroy(scst_mgmt_stub_cachep);
+
+out_destroy_mgmt_cache:
+ kmem_cache_destroy(scst_mgmt_cachep);
+
+out_lib_exit:
+ scst_lib_exit();
+ goto out;
+}
+
+static void __exit exit_scst(void)
+{
+
+ /* ToDo: unregister_cpu_notifier() */
+
+ scst_sysfs_cleanup();
+
+ scst_stop_all_threads();
+
+ scst_deinit_threads(&scst_main_cmd_threads);
+
+ scsi_unregister_interface(&scst_interface);
+
+ scst_sgv_pools_deinit();
+
+#define DEINIT_CACHEP(p) do { \
+ kmem_cache_destroy(p); \
+ p = NULL; \
+ } while (0)
+
+ mempool_destroy(scst_mgmt_mempool);
+ mempool_destroy(scst_mgmt_stub_mempool);
+ mempool_destroy(scst_ua_mempool);
+ mempool_destroy(scst_sense_mempool);
+ mempool_destroy(scst_aen_mempool);
+
+ DEINIT_CACHEP(scst_mgmt_cachep);
+ DEINIT_CACHEP(scst_mgmt_stub_cachep);
+ DEINIT_CACHEP(scst_ua_cachep);
+ DEINIT_CACHEP(scst_sense_cachep);
+ DEINIT_CACHEP(scst_aen_cachep);
+ DEINIT_CACHEP(scst_cmd_cachep);
+ DEINIT_CACHEP(scst_sess_cachep);
+ DEINIT_CACHEP(scst_tgtd_cachep);
+ DEINIT_CACHEP(scst_acgd_cachep);
+
+ scst_lib_exit();
+
+ PRINT_INFO("%s", "SCST unloaded");
+ return;
+}
+
+module_init(init_scst);
+module_exit(exit_scst);
+
+MODULE_AUTHOR("Vladislav Bolkhovitin");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SCSI target core");
+MODULE_VERSION(SCST_VERSION_STRING);
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 4/12/1/5] SCST core's scst_targ.c
[not found] ` <4BC44D08.4060907@vlnb.net>
` (2 preceding siblings ...)
2010-04-13 13:04 ` [PATCH][RFC 3/12/1/5] SCST core's scst_main.c Vladislav Bolkhovitin
@ 2010-04-13 13:05 ` Vladislav Bolkhovitin
2010-04-13 13:05 ` [PATCH][RFC 5/12/1/5] SCST core's scst_lib.c Vladislav Bolkhovitin
` (5 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:05 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains file scst_targ.c.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_targ.c | 5712 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 5712 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/scst_targ.c linux-2.6.33/drivers/scst/scst_targ.c
--- orig/linux-2.6.33/drivers/scst/scst_targ.c
+++ linux-2.6.33/drivers/scst/scst_targ.c
@@ -0,0 +1,5712 @@
+/*
+ * scst_targ.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/smp_lock.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+#include <linux/kthread.h>
+#include <linux/delay.h>
+#include <linux/ktime.h>
+
+#include "scst.h"
+#include "scst_priv.h"
+
+#if 0 /* Temporary left for future performance investigations */
+/* Deleting it don't forget to delete write_cmd_count */
+#define CONFIG_SCST_ORDERED_READS
+#endif
+
+#if 0 /* Let's disable it for now to see if users will complain about it */
+/* Deleting it don't forget to delete write_cmd_count */
+#define CONFIG_SCST_PER_DEVICE_CMD_COUNT_LIMIT
+#endif
+
+static void scst_cmd_set_sn(struct scst_cmd *cmd);
+static int __scst_init_cmd(struct scst_cmd *cmd);
+static void scst_finish_cmd_mgmt(struct scst_cmd *cmd);
+static struct scst_cmd *__scst_find_cmd_by_tag(struct scst_session *sess,
+ uint64_t tag, bool to_abort);
+static void scst_process_redirect_cmd(struct scst_cmd *cmd,
+ enum scst_exec_context context, int check_retries);
+
+static inline void scst_schedule_tasklet(struct scst_cmd *cmd)
+{
+ struct scst_tasklet *t = &scst_tasklets[smp_processor_id()];
+ unsigned long flags;
+
+ spin_lock_irqsave(&t->tasklet_lock, flags);
+ TRACE_DBG("Adding cmd %p to tasklet %d cmd list", cmd,
+ smp_processor_id());
+ list_add_tail(&cmd->cmd_list_entry, &t->tasklet_cmd_list);
+ spin_unlock_irqrestore(&t->tasklet_lock, flags);
+
+ tasklet_schedule(&t->tasklet);
+}
+
+/**
+ * scst_rx_cmd() - create new command
+ * @sess: SCST session
+ * @lun: LUN for the command
+ * @lun_len: length of the LUN in bytes
+ * @cdb: CDB of the command
+ * @cdb_len: length of the CDB in bytes
+ * @atomic: true, if current context is atomic
+ *
+ * Description:
+ * Creates new SCST command. Returns new command on success or
+ * NULL otherwise.
+ *
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same session.
+ */
+struct scst_cmd *scst_rx_cmd(struct scst_session *sess,
+ const uint8_t *lun, int lun_len,
+ const uint8_t *cdb, int cdb_len, int atomic)
+{
+ struct scst_cmd *cmd;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(sess->shut_phase != SCST_SESS_SPH_READY)) {
+ PRINT_CRIT_ERROR("%s",
+ "New cmd while shutting down the session");
+ BUG();
+ }
+#endif
+
+ cmd = scst_alloc_cmd(atomic ? GFP_ATOMIC : GFP_KERNEL);
+ if (cmd == NULL)
+ goto out;
+
+ cmd->sess = sess;
+ cmd->tgt = sess->tgt;
+ cmd->tgtt = sess->tgt->tgtt;
+
+ cmd->lun = scst_unpack_lun(lun, lun_len);
+ if (unlikely(cmd->lun == NO_SUCH_LUN)) {
+ PRINT_ERROR("Wrong LUN %d, finishing cmd", -1);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_lun_not_supported));
+ }
+
+ /*
+ * For cdb_len 0 defer the error reporting until scst_cmd_init_done(),
+ * scst_set_cmd_error() supports nested calls.
+ */
+ if (unlikely(cdb_len > SCST_MAX_CDB_SIZE)) {
+ PRINT_ERROR("Too big CDB len %d, finishing cmd", cdb_len);
+ cdb_len = SCST_MAX_CDB_SIZE;
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ }
+
+ memcpy(cmd->cdb, cdb, cdb_len);
+ cmd->cdb_len = cdb_len;
+
+ TRACE_DBG("cmd %p, sess %p", cmd, sess);
+ scst_sess_get(sess);
+
+out:
+ return cmd;
+}
+EXPORT_SYMBOL(scst_rx_cmd);
+
+/*
+ * No locks, but might be on IRQ. Returns 0 on success, <0 if processing of
+ * this command should be stopped.
+ */
+static int scst_init_cmd(struct scst_cmd *cmd, enum scst_exec_context *context)
+{
+ int rc, res = 0;
+
+ /* See the comment in scst_do_job_init() */
+ if (unlikely(!list_empty(&scst_init_cmd_list))) {
+ TRACE_MGMT_DBG("%s", "init cmd list busy");
+ goto out_redirect;
+ }
+ /*
+ * Memory barrier isn't necessary here, because CPU appears to
+ * be self-consistent and we don't care about the race, described
+ * in comment in scst_do_job_init().
+ */
+
+ rc = __scst_init_cmd(cmd);
+ if (unlikely(rc > 0))
+ goto out_redirect;
+ else if (unlikely(rc != 0)) {
+ res = 1;
+ goto out;
+ }
+
+ EXTRACHECKS_BUG_ON(*context == SCST_CONTEXT_SAME);
+
+ /* Small context optimization */
+ if (((*context == SCST_CONTEXT_TASKLET) ||
+ (*context == SCST_CONTEXT_DIRECT_ATOMIC)) &&
+ scst_cmd_is_expected_set(cmd)) {
+ if (cmd->expected_data_direction & SCST_DATA_WRITE) {
+ if (!test_bit(SCST_TGT_DEV_AFTER_INIT_WR_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ *context = SCST_CONTEXT_THREAD;
+ } else {
+ if (!test_bit(SCST_TGT_DEV_AFTER_INIT_OTH_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ *context = SCST_CONTEXT_THREAD;
+ }
+ }
+
+out:
+ return res;
+
+out_redirect:
+ if (cmd->preprocessing_only) {
+ /*
+ * Poor man solution for single threaded targets, where
+ * blocking receiver at least sometimes means blocking all.
+ * For instance, iSCSI target won't be able to receive
+ * Data-Out PDUs.
+ */
+ BUG_ON(*context != SCST_CONTEXT_DIRECT);
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = 1;
+ /* Keep initiator away from too many BUSY commands */
+ msleep(50);
+ } else {
+ unsigned long flags;
+ spin_lock_irqsave(&scst_init_lock, flags);
+ TRACE_MGMT_DBG("Adding cmd %p to init cmd list (scst_cmd_count "
+ "%d)", cmd, atomic_read(&scst_cmd_count));
+ list_add_tail(&cmd->cmd_list_entry, &scst_init_cmd_list);
+ if (test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))
+ scst_init_poll_cnt++;
+ spin_unlock_irqrestore(&scst_init_lock, flags);
+ wake_up(&scst_init_cmd_list_waitQ);
+ res = -1;
+ }
+ goto out;
+}
+
+/**
+ * scst_cmd_init_done() - the command's initialization done
+ * @cmd: SCST command
+ * @pref_context: preferred command execution context
+ *
+ * Description:
+ * Notifies SCST that the driver finished its part of the command
+ * initialization, and the command is ready for execution.
+ * The second argument sets preferred command execition context.
+ * See SCST_CONTEXT_* constants for details.
+ *
+ * !!IMPORTANT!!
+ *
+ * If cmd->set_sn_on_restart_cmd not set, this function, as well as
+ * scst_cmd_init_stage1_done() and scst_restart_cmd(), must not be
+ * called simultaneously for the same session (more precisely,
+ * for the same session/LUN, i.e. tgt_dev), i.e. they must be
+ * somehow externally serialized. This is needed to have lock free fast
+ * path in scst_cmd_set_sn(). For majority of targets those functions are
+ * naturally serialized by the single source of commands. Only iSCSI
+ * immediate commands with multiple connections per session seems to be an
+ * exception. For it, some mutex/lock shall be used for the serialization.
+ */
+void scst_cmd_init_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context)
+{
+ unsigned long flags;
+ struct scst_session *sess = cmd->sess;
+ int rc;
+
+ scst_set_start_time(cmd);
+
+ TRACE_DBG("Preferred context: %d (cmd %p)", pref_context, cmd);
+ TRACE(TRACE_SCSI, "tag=%llu, lun=%lld, CDB len=%d, queue_type=%x "
+ "(cmd %p)", (long long unsigned int)cmd->tag,
+ (long long unsigned int)cmd->lun, cmd->cdb_len,
+ cmd->queue_type, cmd);
+ PRINT_BUFF_FLAG(TRACE_SCSI|TRACE_RCV_BOT, "Recieving CDB",
+ cmd->cdb, cmd->cdb_len);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely((in_irq() || irqs_disabled())) &&
+ ((pref_context == SCST_CONTEXT_DIRECT) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC))) {
+ PRINT_ERROR("Wrong context %d in IRQ from target %s, use "
+ "SCST_CONTEXT_THREAD instead", pref_context,
+ cmd->tgtt->name);
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+#endif
+
+ atomic_inc(&sess->sess_cmd_count);
+
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+
+ if (unlikely(sess->init_phase != SCST_SESS_IPH_READY)) {
+ /*
+ * We must always keep commands in the sess list from the
+ * very beginning, because otherwise they can be missed during
+ * TM processing. This check is needed because there might be
+ * old, i.e. deferred, commands and new, i.e. just coming, ones.
+ */
+ if (cmd->sess_cmd_list_entry.next == NULL)
+ list_add_tail(&cmd->sess_cmd_list_entry,
+ &sess->sess_cmd_list);
+ switch (sess->init_phase) {
+ case SCST_SESS_IPH_SUCCESS:
+ break;
+ case SCST_SESS_IPH_INITING:
+ TRACE_DBG("Adding cmd %p to init deferred cmd list",
+ cmd);
+ list_add_tail(&cmd->cmd_list_entry,
+ &sess->init_deferred_cmd_list);
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ goto out;
+ case SCST_SESS_IPH_FAILED:
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ goto active;
+ default:
+ BUG();
+ }
+ } else
+ list_add_tail(&cmd->sess_cmd_list_entry,
+ &sess->sess_cmd_list);
+
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+
+ if (unlikely(cmd->cdb_len == 0)) {
+ PRINT_ERROR("%s", "Wrong CDB len 0, finishing cmd");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ scst_set_cmd_abnormal_done_state(cmd);
+ goto active;
+ }
+
+ if (unlikely(cmd->queue_type >= SCST_CMD_QUEUE_ACA)) {
+ PRINT_ERROR("Unsupported queue type %d", cmd->queue_type);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ goto active;
+ }
+
+ /*
+ * Cmd must be inited here to preserve the order. In case if cmd
+ * already preliminary completed by target driver we need to init
+ * cmd anyway to find out in which format we should return sense.
+ */
+ cmd->state = SCST_CMD_STATE_INIT;
+ rc = scst_init_cmd(cmd, &pref_context);
+ if (unlikely(rc < 0))
+ goto out;
+
+active:
+ /* Here cmd must not be in any cmd list, no locks */
+ switch (pref_context) {
+ case SCST_CONTEXT_TASKLET:
+ scst_schedule_tasklet(cmd);
+ break;
+
+ case SCST_CONTEXT_DIRECT:
+ scst_process_active_cmd(cmd, false);
+ break;
+
+ case SCST_CONTEXT_DIRECT_ATOMIC:
+ scst_process_active_cmd(cmd, true);
+ break;
+
+ default:
+ PRINT_ERROR("Context %x is undefined, using the thread one",
+ pref_context);
+ /* go through */
+ case SCST_CONTEXT_THREAD:
+ spin_lock_irqsave(&cmd->cmd_threads->cmd_list_lock, flags);
+ TRACE_DBG("Adding cmd %p to active cmd list", cmd);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irqrestore(&cmd->cmd_threads->cmd_list_lock, flags);
+ break;
+ }
+
+out:
+ return;
+}
+EXPORT_SYMBOL(scst_cmd_init_done);
+
+static int scst_pre_parse(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+ struct scst_device *dev = cmd->dev;
+ int rc;
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ cmd->inc_expected_sn_on_done = 1;
+#else
+ cmd->inc_expected_sn_on_done = dev->handler->exec_sync ||
+ (!dev->has_own_order_mgmt &&
+ (dev->queue_alg == SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER ||
+ cmd->queue_type == SCST_CMD_QUEUE_ORDERED));
+#endif
+
+ /*
+ * Expected transfer data supplied by the SCSI transport via the
+ * target driver are untrusted, so we prefer to fetch them from CDB.
+ * Additionally, not all transports support supplying the expected
+ * transfer data.
+ */
+
+ rc = scst_get_cdb_info(cmd);
+ if (unlikely(rc != 0)) {
+ if (rc > 0) {
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_err;
+ }
+
+ EXTRACHECKS_BUG_ON(cmd->op_flags & SCST_INFO_VALID);
+
+ cmd->cdb_len = scst_get_cdb_len(cmd->cdb);
+
+ TRACE(TRACE_MINOR, "Unknown opcode 0x%02x for %s. "
+ "Should you update scst_scsi_op_table?",
+ cmd->cdb[0], dev->handler->name);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Failed CDB", cmd->cdb,
+ cmd->cdb_len);
+ } else {
+ EXTRACHECKS_BUG_ON(!(cmd->op_flags & SCST_INFO_VALID));
+ }
+
+ cmd->state = SCST_CMD_STATE_DEV_PARSE;
+
+ TRACE_DBG("op_name <%s> (cmd %p), direction=%d "
+ "(expected %d, set %s), transfer_len=%d (expected "
+ "len %d), flags=%d", cmd->op_name, cmd,
+ cmd->data_direction, cmd->expected_data_direction,
+ scst_cmd_is_expected_set(cmd) ? "yes" : "no",
+ cmd->bufflen, cmd->expected_transfer_len,
+ cmd->op_flags);
+
+out:
+ return res;
+
+out_err:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+#ifndef CONFIG_SCST_USE_EXPECTED_VALUES
+static bool scst_is_allowed_to_mismatch_cmd(struct scst_cmd *cmd)
+{
+ bool res = false;
+
+ /* VERIFY commands with BYTCHK unset shouldn't fail here */
+ if ((cmd->op_flags & SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED) &&
+ (cmd->cdb[1] & BYTCHK) == 0) {
+ res = true;
+ goto out;
+ }
+
+ switch (cmd->cdb[0]) {
+ case TEST_UNIT_READY:
+ /* Crazy VMware people sometimes do TUR with READ direction */
+ res = true;
+ break;
+ }
+
+out:
+ return res;
+}
+#endif
+
+static int scst_parse_cmd(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+ int state;
+ struct scst_device *dev = cmd->dev;
+ int orig_bufflen = cmd->bufflen;
+
+ if (likely(!scst_is_cmd_fully_local(cmd))) {
+ if (unlikely(!dev->handler->parse_atomic &&
+ scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Dev handler %s parse() needs thread "
+ "context, rescheduling", dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE_DBG("Calling dev handler %s parse(%p)",
+ dev->handler->name, cmd);
+ TRACE_BUFF_FLAG(TRACE_SND_BOT, "Parsing: ",
+ cmd->cdb, cmd->cdb_len);
+ scst_set_cur_start(cmd);
+ state = dev->handler->parse(cmd);
+ /* Caution: cmd can be already dead here */
+ TRACE_DBG("Dev handler %s parse() returned %d",
+ dev->handler->name, state);
+
+ switch (state) {
+ case SCST_CMD_STATE_NEED_THREAD_CTX:
+ scst_set_parse_time(cmd);
+ TRACE_DBG("Dev handler %s parse() requested thread "
+ "context, rescheduling", dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+
+ case SCST_CMD_STATE_STOP:
+ TRACE_DBG("Dev handler %s parse() requested stop "
+ "processing", dev->handler->name);
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ goto out;
+ }
+
+ scst_set_parse_time(cmd);
+
+ if (state == SCST_CMD_STATE_DEFAULT)
+ state = SCST_CMD_STATE_PREPARE_SPACE;
+ } else
+ state = SCST_CMD_STATE_PREPARE_SPACE;
+
+ if (unlikely(state == SCST_CMD_STATE_PRE_XMIT_RESP))
+ goto set_res;
+
+ if (unlikely(!(cmd->op_flags & SCST_INFO_VALID))) {
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+ if (scst_cmd_is_expected_set(cmd)) {
+ TRACE(TRACE_MINOR, "Using initiator supplied values: "
+ "direction %d, transfer_len %d",
+ cmd->expected_data_direction,
+ cmd->expected_transfer_len);
+ cmd->data_direction = cmd->expected_data_direction;
+ cmd->bufflen = cmd->expected_transfer_len;
+ } else {
+ PRINT_ERROR("Unknown opcode 0x%02x for %s and "
+ "target %s not supplied expected values",
+ cmd->cdb[0], dev->handler->name, cmd->tgtt->name);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+ }
+#else
+ PRINT_ERROR("Unknown opcode %x", cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+#endif
+ }
+
+ if (unlikely(cmd->cdb_len == -1)) {
+ PRINT_ERROR("Unable to get CDB length for "
+ "opcode 0x%02x. Returning INVALID "
+ "OPCODE", cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_opcode));
+ goto out_done;
+ }
+
+ EXTRACHECKS_BUG_ON(cmd->cdb_len == 0);
+
+ TRACE(TRACE_SCSI, "op_name <%s> (cmd %p), direction=%d "
+ "(expected %d, set %s), transfer_len=%d (expected "
+ "len %d), flags=%d", cmd->op_name, cmd,
+ cmd->data_direction, cmd->expected_data_direction,
+ scst_cmd_is_expected_set(cmd) ? "yes" : "no",
+ cmd->bufflen, cmd->expected_transfer_len,
+ cmd->op_flags);
+
+ if (unlikely((cmd->op_flags & SCST_UNKNOWN_LENGTH) != 0)) {
+ if (scst_cmd_is_expected_set(cmd)) {
+ /*
+ * Command data length can't be easily
+ * determined from the CDB. ToDo, all such
+ * commands processing should be fixed. Until
+ * it's done, get the length from the supplied
+ * expected value, but limit it to some
+ * reasonable value (15MB).
+ */
+ cmd->bufflen = min(cmd->expected_transfer_len,
+ 15*1024*1024);
+ cmd->op_flags &= ~SCST_UNKNOWN_LENGTH;
+ } else {
+ PRINT_ERROR("Unknown data transfer length for opcode "
+ "0x%x (handler %s, target %s)", cmd->cdb[0],
+ dev->handler->name, cmd->tgtt->name);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ goto out_done;
+ }
+ }
+
+ if (unlikely(cmd->cdb[cmd->cdb_len - 1] & CONTROL_BYTE_NACA_BIT)) {
+ PRINT_ERROR("NACA bit in control byte CDB is not supported "
+ "(opcode 0x%02x)", cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_done;
+ }
+
+ if (unlikely(cmd->cdb[cmd->cdb_len - 1] & CONTROL_BYTE_LINK_BIT)) {
+ PRINT_ERROR("Linked commands are not supported "
+ "(opcode 0x%02x)", cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_done;
+ }
+
+ if (cmd->dh_data_buf_alloced &&
+ unlikely((orig_bufflen > cmd->bufflen))) {
+ PRINT_ERROR("Dev handler supplied data buffer (size %d), "
+ "is less, than required (size %d)", cmd->bufflen,
+ orig_bufflen);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_hw_error;
+ }
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((cmd->bufflen != 0) &&
+ ((cmd->data_direction == SCST_DATA_NONE) ||
+ ((cmd->sg == NULL) && (state > SCST_CMD_STATE_PREPARE_SPACE)))) {
+ PRINT_ERROR("Dev handler %s parse() returned "
+ "invalid cmd data_direction %d, bufflen %d, state %d "
+ "or sg %p (opcode 0x%x)", dev->handler->name,
+ cmd->data_direction, cmd->bufflen, state, cmd->sg,
+ cmd->cdb[0]);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_hw_error;
+ }
+#endif
+
+ if (scst_cmd_is_expected_set(cmd)) {
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+# ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely((cmd->data_direction != cmd->expected_data_direction) ||
+ (cmd->bufflen != cmd->expected_transfer_len))) {
+ TRACE(TRACE_MINOR, "Expected values don't match "
+ "decoded ones: data_direction %d, "
+ "expected_data_direction %d, "
+ "bufflen %d, expected_transfer_len %d",
+ cmd->data_direction,
+ cmd->expected_data_direction,
+ cmd->bufflen, cmd->expected_transfer_len);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Suspicious CDB",
+ cmd->cdb, cmd->cdb_len);
+ }
+# endif
+ cmd->data_direction = cmd->expected_data_direction;
+ cmd->bufflen = cmd->expected_transfer_len;
+#else
+ if (unlikely(cmd->data_direction !=
+ cmd->expected_data_direction)) {
+ if (((cmd->expected_data_direction != SCST_DATA_NONE) ||
+ (cmd->bufflen != 0)) &&
+ !scst_is_allowed_to_mismatch_cmd(cmd)) {
+ PRINT_ERROR("Expected data direction %d for "
+ "opcode 0x%02x (handler %s, target %s) "
+ "doesn't match decoded value %d",
+ cmd->expected_data_direction,
+ cmd->cdb[0], dev->handler->name,
+ cmd->tgtt->name, cmd->data_direction);
+ PRINT_BUFFER("Failed CDB", cmd->cdb,
+ cmd->cdb_len);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ goto out_done;
+ }
+ }
+ if (unlikely(cmd->bufflen != cmd->expected_transfer_len)) {
+ TRACE(TRACE_MINOR, "Warning: expected "
+ "transfer length %d for opcode 0x%02x "
+ "(handler %s, target %s) doesn't match "
+ "decoded value %d. Faulty initiator "
+ "(e.g. VMware is known to be such) or "
+ "scst_scsi_op_table should be updated?",
+ cmd->expected_transfer_len, cmd->cdb[0],
+ dev->handler->name, cmd->tgtt->name,
+ cmd->bufflen);
+ PRINT_BUFF_FLAG(TRACE_MINOR, "Suspicious CDB",
+ cmd->cdb, cmd->cdb_len);
+ /* Needed, e.g., to get immediate iSCSI data */
+ cmd->bufflen = max(cmd->bufflen,
+ cmd->expected_transfer_len);
+ }
+#endif
+ }
+
+ if (unlikely(cmd->data_direction == SCST_DATA_UNKNOWN)) {
+ PRINT_ERROR("Unknown data direction. Opcode 0x%x, handler %s, "
+ "target %s", cmd->cdb[0], dev->handler->name,
+ cmd->tgtt->name);
+ PRINT_BUFFER("Failed CDB", cmd->cdb, cmd->cdb_len);
+ goto out_hw_error;
+ }
+
+set_res:
+ if (cmd->data_len == -1)
+ cmd->data_len = cmd->bufflen;
+
+ if (cmd->bufflen == 0) {
+ /*
+ * According to SPC bufflen 0 for data transfer commands isn't
+ * an error, so we need to fix the transfer direction.
+ */
+ cmd->data_direction = SCST_DATA_NONE;
+ }
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ switch (state) {
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ case SCST_CMD_STATE_PRE_PARSE:
+ case SCST_CMD_STATE_DEV_PARSE:
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ case SCST_CMD_STATE_DEV_DONE:
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ case SCST_CMD_STATE_XMIT_RESP:
+ case SCST_CMD_STATE_FINISHED:
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+#endif
+ cmd->state = state;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ break;
+
+ default:
+ if (state >= 0) {
+ PRINT_ERROR("Dev handler %s parse() returned "
+ "invalid cmd state %d (opcode %d)",
+ dev->handler->name, state, cmd->cdb[0]);
+ } else {
+ PRINT_ERROR("Dev handler %s parse() returned "
+ "error %d (opcode %d)", dev->handler->name,
+ state, cmd->cdb[0]);
+ }
+ goto out_hw_error;
+ }
+#endif
+
+ if (cmd->resp_data_len == -1) {
+ if (cmd->data_direction & SCST_DATA_READ)
+ cmd->resp_data_len = cmd->bufflen;
+ else
+ cmd->resp_data_len = 0;
+ }
+
+ /* We already completed (with an error) */
+ if (unlikely(cmd->completed))
+ goto out_done;
+
+out:
+ return res;
+
+out_hw_error:
+ /* dev_done() will be called as part of the regular cmd's finish */
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+
+out_done:
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+static int scst_prepare_space(struct scst_cmd *cmd)
+{
+ int r = 0, res = SCST_CMD_STATE_RES_CONT_SAME;
+
+ if (cmd->data_direction == SCST_DATA_NONE)
+ goto done;
+
+ if (cmd->tgt_need_alloc_data_buf) {
+ int orig_bufflen = cmd->bufflen;
+
+ TRACE_MEM("Custom tgt data buf allocation requested (cmd %p)",
+ cmd);
+
+ scst_set_cur_start(cmd);
+ r = cmd->tgtt->alloc_data_buf(cmd);
+ scst_set_alloc_buf_time(cmd);
+
+ if (r > 0)
+ goto alloc;
+ else if (r == 0) {
+ if (unlikely(cmd->bufflen == 0)) {
+ /* See comment in scst_alloc_space() */
+ if (cmd->sg == NULL)
+ goto alloc;
+ }
+
+ cmd->tgt_data_buf_alloced = 1;
+
+ if (unlikely(orig_bufflen < cmd->bufflen)) {
+ PRINT_ERROR("Target driver allocated data "
+ "buffer (size %d), is less, than "
+ "required (size %d)", orig_bufflen,
+ cmd->bufflen);
+ goto out_error;
+ }
+ TRACE_MEM("tgt_data_buf_alloced (cmd %p)", cmd);
+ } else
+ goto check;
+ }
+
+alloc:
+ if (!cmd->tgt_data_buf_alloced && !cmd->dh_data_buf_alloced) {
+ r = scst_alloc_space(cmd);
+ } else if (cmd->dh_data_buf_alloced && !cmd->tgt_data_buf_alloced) {
+ TRACE_MEM("dh_data_buf_alloced set (cmd %p)", cmd);
+ r = 0;
+ } else if (cmd->tgt_data_buf_alloced && !cmd->dh_data_buf_alloced) {
+ TRACE_MEM("tgt_data_buf_alloced set (cmd %p)", cmd);
+ cmd->sg = cmd->tgt_sg;
+ cmd->sg_cnt = cmd->tgt_sg_cnt;
+ cmd->in_sg = cmd->tgt_in_sg;
+ cmd->in_sg_cnt = cmd->tgt_in_sg_cnt;
+ r = 0;
+ } else {
+ TRACE_MEM("Both *_data_buf_alloced set (cmd %p, sg %p, "
+ "sg_cnt %d, tgt_sg %p, tgt_sg_cnt %d)", cmd, cmd->sg,
+ cmd->sg_cnt, cmd->tgt_sg, cmd->tgt_sg_cnt);
+ r = 0;
+ }
+
+check:
+ if (r != 0) {
+ if (scst_cmd_atomic(cmd)) {
+ TRACE_MEM("%s", "Atomic memory allocation failed, "
+ "rescheduling to the thread");
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ } else
+ goto out_no_space;
+ }
+
+done:
+ if (cmd->preprocessing_only)
+ cmd->state = SCST_CMD_STATE_PREPROCESSING_DONE;
+ else if (cmd->data_direction & SCST_DATA_WRITE)
+ cmd->state = SCST_CMD_STATE_RDY_TO_XFER;
+ else
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+
+out:
+ return res;
+
+out_no_space:
+ TRACE(TRACE_OUT_OF_MEM, "Unable to allocate or build requested buffer "
+ "(size %d), sending BUSY or QUEUE FULL status", cmd->bufflen);
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+
+out_error:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+static int scst_preprocessing_done(struct scst_cmd *cmd)
+{
+ int res;
+
+ EXTRACHECKS_BUG_ON(!cmd->preprocessing_only);
+
+ cmd->preprocessing_only = 0;
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ cmd->state = SCST_CMD_STATE_PREPROCESSING_DONE_CALLED;
+
+ TRACE_DBG("Calling preprocessing_done(cmd %p)", cmd);
+ scst_set_cur_start(cmd);
+ cmd->tgtt->preprocessing_done(cmd);
+ TRACE_DBG("%s", "preprocessing_done() returned");
+ return res;
+}
+
+/**
+ * scst_restart_cmd() - restart execution of the command
+ * @cmd: SCST commands
+ * @status: completion status
+ * @pref_context: preferred command execition context
+ *
+ * Description:
+ * Notifies SCST that the driver finished its part of the command's
+ * preprocessing and it is ready for further processing.
+ *
+ * The second argument sets completion status
+ * (see SCST_PREPROCESS_STATUS_* constants for details)
+ *
+ * See also comment for scst_cmd_init_done() for the serialization
+ * requirements.
+ */
+void scst_restart_cmd(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context)
+{
+
+ scst_set_restart_waiting_time(cmd);
+
+ TRACE_DBG("Preferred context: %d", pref_context);
+ TRACE_DBG("tag=%llu, status=%#x",
+ (long long unsigned int)scst_cmd_get_tag(cmd),
+ status);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((in_irq() || irqs_disabled()) &&
+ ((pref_context == SCST_CONTEXT_DIRECT) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC))) {
+ PRINT_ERROR("Wrong context %d in IRQ from target %s, use "
+ "SCST_CONTEXT_THREAD instead", pref_context,
+ cmd->tgtt->name);
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+#endif
+
+ switch (status) {
+ case SCST_PREPROCESS_STATUS_SUCCESS:
+ if (cmd->data_direction & SCST_DATA_WRITE)
+ cmd->state = SCST_CMD_STATE_RDY_TO_XFER;
+ else
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+ if (cmd->set_sn_on_restart_cmd)
+ scst_cmd_set_sn(cmd);
+ /* Small context optimization */
+ if ((pref_context == SCST_CONTEXT_TASKLET) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC) ||
+ ((pref_context == SCST_CONTEXT_SAME) &&
+ scst_cmd_atomic(cmd))) {
+ if (cmd->data_direction & SCST_DATA_WRITE) {
+ if (!test_bit(SCST_TGT_DEV_AFTER_RESTART_WR_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ pref_context = SCST_CONTEXT_THREAD;
+ } else {
+ if (!test_bit(SCST_TGT_DEV_AFTER_RESTART_OTH_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+ }
+ break;
+
+ case SCST_PREPROCESS_STATUS_ERROR_SENSE_SET:
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ case SCST_PREPROCESS_STATUS_ERROR_FATAL:
+ set_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags);
+ /* go through */
+ case SCST_PREPROCESS_STATUS_ERROR:
+ if (cmd->sense != NULL)
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ default:
+ PRINT_ERROR("%s() received unknown status %x", __func__,
+ status);
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ }
+
+ scst_process_redirect_cmd(cmd, pref_context, 1);
+ return;
+}
+EXPORT_SYMBOL(scst_restart_cmd);
+
+static int scst_rdy_to_xfer(struct scst_cmd *cmd)
+{
+ int res, rc;
+ struct scst_tgt_template *tgtt = cmd->tgtt;
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("ABORTED set, aborting cmd %p", cmd);
+ goto out_dev_done;
+ }
+
+ if ((tgtt->rdy_to_xfer == NULL) || unlikely(cmd->internal)) {
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+ }
+
+ if (unlikely(!tgtt->rdy_to_xfer_atomic && scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Target driver %s rdy_to_xfer() needs thread "
+ "context, rescheduling", tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ while (1) {
+ int finished_cmds = atomic_read(&cmd->tgt->finished_cmds);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ cmd->state = SCST_CMD_STATE_DATA_WAIT;
+
+ if (tgtt->on_hw_pending_cmd_timeout != NULL) {
+ struct scst_session *sess = cmd->sess;
+ cmd->hw_pending_start = jiffies;
+ cmd->cmd_hw_pending = 1;
+ if (!test_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags)) {
+ TRACE_DBG("Sched HW pending work for sess %p "
+ "(max time %d)", sess,
+ tgtt->max_hw_pending_time);
+ set_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED,
+ &sess->sess_aflags);
+ schedule_delayed_work(&sess->hw_pending_work,
+ tgtt->max_hw_pending_time * HZ);
+ }
+ }
+
+ scst_set_cur_start(cmd);
+
+ TRACE_DBG("Calling rdy_to_xfer(%p)", cmd);
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ if (((scst_random() % 100) == 75))
+ rc = SCST_TGT_RES_QUEUE_FULL;
+ else
+#endif
+ rc = tgtt->rdy_to_xfer(cmd);
+ TRACE_DBG("rdy_to_xfer() returned %d", rc);
+
+ if (likely(rc == SCST_TGT_RES_SUCCESS))
+ goto out;
+
+ scst_set_rdy_to_xfer_time(cmd);
+
+ cmd->cmd_hw_pending = 0;
+
+ /* Restore the previous state */
+ cmd->state = SCST_CMD_STATE_RDY_TO_XFER;
+
+ switch (rc) {
+ case SCST_TGT_RES_QUEUE_FULL:
+ if (scst_queue_retry_cmd(cmd, finished_cmds) == 0)
+ break;
+ else
+ continue;
+
+ case SCST_TGT_RES_NEED_THREAD_CTX:
+ TRACE_DBG("Target driver %s "
+ "rdy_to_xfer() requested thread "
+ "context, rescheduling", tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+
+ default:
+ goto out_error_rc;
+ }
+ break;
+ }
+
+out:
+ return res;
+
+out_error_rc:
+ if (rc == SCST_TGT_RES_FATAL_ERROR) {
+ PRINT_ERROR("Target driver %s rdy_to_xfer() returned "
+ "fatal error", tgtt->name);
+ } else {
+ PRINT_ERROR("Target driver %s rdy_to_xfer() returned invalid "
+ "value %d", tgtt->name, rc);
+ }
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+
+out_dev_done:
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+/* No locks, but might be in IRQ */
+static void scst_process_redirect_cmd(struct scst_cmd *cmd,
+ enum scst_exec_context context, int check_retries)
+{
+ struct scst_tgt *tgt = cmd->tgt;
+ unsigned long flags;
+
+ TRACE_DBG("Context: %x", context);
+
+ if (context == SCST_CONTEXT_SAME)
+ context = scst_cmd_atomic(cmd) ? SCST_CONTEXT_DIRECT_ATOMIC :
+ SCST_CONTEXT_DIRECT;
+
+ switch (context) {
+ case SCST_CONTEXT_DIRECT_ATOMIC:
+ scst_process_active_cmd(cmd, true);
+ break;
+
+ case SCST_CONTEXT_DIRECT:
+ if (check_retries)
+ scst_check_retries(tgt);
+ scst_process_active_cmd(cmd, false);
+ break;
+
+ default:
+ PRINT_ERROR("Context %x is unknown, using the thread one",
+ context);
+ /* go through */
+ case SCST_CONTEXT_THREAD:
+ if (check_retries)
+ scst_check_retries(tgt);
+ spin_lock_irqsave(&cmd->cmd_threads->cmd_list_lock, flags);
+ TRACE_DBG("Adding cmd %p to active cmd list", cmd);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irqrestore(&cmd->cmd_threads->cmd_list_lock, flags);
+ break;
+
+ case SCST_CONTEXT_TASKLET:
+ if (check_retries)
+ scst_check_retries(tgt);
+ scst_schedule_tasklet(cmd);
+ break;
+ }
+ return;
+}
+
+/**
+ * scst_rx_data() - the command's data received
+ * @cmd: SCST commands
+ * @status: data receiving completion status
+ * @pref_context: preferred command execition context
+ *
+ * Description:
+ * Notifies SCST that the driver received all the necessary data
+ * and the command is ready for further processing.
+ *
+ * The second argument sets data receiving completion status
+ * (see SCST_RX_STATUS_* constants for details)
+ */
+void scst_rx_data(struct scst_cmd *cmd, int status,
+ enum scst_exec_context pref_context)
+{
+
+ scst_set_rdy_to_xfer_time(cmd);
+
+ TRACE_DBG("Preferred context: %d", pref_context);
+ TRACE(TRACE_SCSI, "cmd %p, status %#x", cmd, status);
+
+ cmd->cmd_hw_pending = 0;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((in_irq() || irqs_disabled()) &&
+ ((pref_context == SCST_CONTEXT_DIRECT) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC))) {
+ PRINT_ERROR("Wrong context %d in IRQ from target %s, use "
+ "SCST_CONTEXT_THREAD instead", pref_context,
+ cmd->tgtt->name);
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+#endif
+
+ switch (status) {
+ case SCST_RX_STATUS_SUCCESS:
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (trace_flag & TRACE_RCV_BOT) {
+ int i;
+ struct scatterlist *sg;
+ if (cmd->in_sg != NULL)
+ sg = cmd->in_sg;
+ else if (cmd->tgt_in_sg != NULL)
+ sg = cmd->tgt_in_sg;
+ else if (cmd->tgt_sg != NULL)
+ sg = cmd->tgt_sg;
+ else
+ sg = cmd->sg;
+ if (sg != NULL) {
+ TRACE_RECV_BOT("RX data for cmd %p "
+ "(sg_cnt %d, sg %p, sg[0].page %p)",
+ cmd, cmd->tgt_sg_cnt, sg,
+ (void *)sg_page(&sg[0]));
+ for (i = 0; i < cmd->tgt_sg_cnt; ++i) {
+ PRINT_BUFF_FLAG(TRACE_RCV_BOT, "RX sg",
+ sg_virt(&sg[i]), sg[i].length);
+ }
+ }
+ }
+#endif
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+
+ /* Small context optimization */
+ if ((pref_context == SCST_CONTEXT_TASKLET) ||
+ (pref_context == SCST_CONTEXT_DIRECT_ATOMIC) ||
+ ((pref_context == SCST_CONTEXT_SAME) &&
+ scst_cmd_atomic(cmd))) {
+ if (!test_bit(SCST_TGT_DEV_AFTER_RX_DATA_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ pref_context = SCST_CONTEXT_THREAD;
+ }
+ break;
+
+ case SCST_RX_STATUS_ERROR_SENSE_SET:
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ case SCST_RX_STATUS_ERROR_FATAL:
+ set_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags);
+ /* go through */
+ case SCST_RX_STATUS_ERROR:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+
+ default:
+ PRINT_ERROR("scst_rx_data() received unknown status %x",
+ status);
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ }
+
+ scst_process_redirect_cmd(cmd, pref_context, 1);
+ return;
+}
+EXPORT_SYMBOL(scst_rx_data);
+
+static int scst_tgt_pre_exec(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME, rc;
+
+ cmd->state = SCST_CMD_STATE_SEND_FOR_EXEC;
+
+ if ((cmd->tgtt->pre_exec == NULL) || unlikely(cmd->internal))
+ goto out;
+
+ TRACE_DBG("Calling pre_exec(%p)", cmd);
+ scst_set_cur_start(cmd);
+ rc = cmd->tgtt->pre_exec(cmd);
+ scst_set_pre_exec_time(cmd);
+ TRACE_DBG("pre_exec() returned %d", rc);
+
+ if (unlikely(rc != SCST_PREPROCESS_STATUS_SUCCESS)) {
+ switch (rc) {
+ case SCST_PREPROCESS_STATUS_ERROR_SENSE_SET:
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ case SCST_PREPROCESS_STATUS_ERROR_FATAL:
+ set_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags);
+ /* go through */
+ case SCST_PREPROCESS_STATUS_ERROR:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+ case SCST_PREPROCESS_STATUS_NEED_THREAD:
+ TRACE_DBG("Target driver's %s pre_exec() requested "
+ "thread context, rescheduling",
+ cmd->tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ cmd->state = SCST_CMD_STATE_TGT_PRE_EXEC;
+ break;
+ default:
+ BUG();
+ break;
+ }
+ }
+
+out:
+ return res;
+}
+
+static void scst_do_cmd_done(struct scst_cmd *cmd, int result,
+ const uint8_t *rq_sense, int rq_sense_len, int resid)
+{
+
+ scst_set_exec_time(cmd);
+
+ cmd->status = result & 0xff;
+ cmd->msg_status = msg_byte(result);
+ cmd->host_status = host_byte(result);
+ cmd->driver_status = driver_byte(result);
+ if (unlikely(resid != 0)) {
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((resid < 0) || (resid > cmd->resp_data_len)) {
+ PRINT_ERROR("Wrong resid %d (cmd->resp_data_len=%d, "
+ "op %x)", resid, cmd->resp_data_len,
+ cmd->cdb[0]);
+ } else
+#endif
+ scst_set_resp_data_len(cmd, cmd->resp_data_len - resid);
+ }
+
+ if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION)) {
+ /* We might have double reset UA here */
+ cmd->dbl_ua_orig_resp_data_len = cmd->resp_data_len;
+ cmd->dbl_ua_orig_data_direction = cmd->data_direction;
+
+ scst_alloc_set_sense(cmd, 1, rq_sense, rq_sense_len);
+ }
+
+ TRACE(TRACE_SCSI, "cmd %p, result=%x, cmd->status=%x, resid=%d, "
+ "cmd->msg_status=%x, cmd->host_status=%x, "
+ "cmd->driver_status=%x (cmd %p)", cmd, result, cmd->status, resid,
+ cmd->msg_status, cmd->host_status, cmd->driver_status, cmd);
+
+ cmd->completed = 1;
+ return;
+}
+
+/* For small context optimization */
+static inline enum scst_exec_context scst_optimize_post_exec_context(
+ struct scst_cmd *cmd, enum scst_exec_context context)
+{
+ if (((context == SCST_CONTEXT_SAME) && scst_cmd_atomic(cmd)) ||
+ (context == SCST_CONTEXT_TASKLET) ||
+ (context == SCST_CONTEXT_DIRECT_ATOMIC)) {
+ if (!test_bit(SCST_TGT_DEV_AFTER_EXEC_ATOMIC,
+ &cmd->tgt_dev->tgt_dev_flags))
+ context = SCST_CONTEXT_THREAD;
+ }
+ return context;
+}
+
+static void scst_cmd_done(void *data, char *sense, int result, int resid)
+{
+ struct scst_cmd *cmd;
+
+ cmd = (struct scst_cmd *)data;
+ if (cmd == NULL)
+ goto out;
+
+ scst_do_cmd_done(cmd, result, sense, SCSI_SENSE_BUFFERSIZE, resid);
+
+ cmd->state = SCST_CMD_STATE_PRE_DEV_DONE;
+
+ scst_process_redirect_cmd(cmd,
+ scst_optimize_post_exec_context(cmd, scst_estimate_context()), 0);
+
+out:
+ return;
+}
+
+static void scst_cmd_done_local(struct scst_cmd *cmd, int next_state,
+ enum scst_exec_context pref_context)
+{
+
+ scst_set_exec_time(cmd);
+
+ if (next_state == SCST_CMD_STATE_DEFAULT)
+ next_state = SCST_CMD_STATE_PRE_DEV_DONE;
+
+#if defined(CONFIG_SCST_DEBUG)
+ if (next_state == SCST_CMD_STATE_PRE_DEV_DONE) {
+ if ((trace_flag & TRACE_RCV_TOP) && (cmd->sg != NULL)) {
+ int i;
+ struct scatterlist *sg = cmd->sg;
+ TRACE_RECV_TOP("Exec'd %d S/G(s) at %p sg[0].page at "
+ "%p", cmd->sg_cnt, sg, (void *)sg_page(&sg[0]));
+ for (i = 0; i < cmd->sg_cnt; ++i) {
+ TRACE_BUFF_FLAG(TRACE_RCV_TOP,
+ "Exec'd sg", sg_virt(&sg[i]),
+ sg[i].length);
+ }
+ }
+ }
+#endif
+
+ cmd->state = next_state;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if ((next_state != SCST_CMD_STATE_PRE_DEV_DONE) &&
+ (next_state != SCST_CMD_STATE_PRE_XMIT_RESP) &&
+ (next_state != SCST_CMD_STATE_FINISHED) &&
+ (next_state != SCST_CMD_STATE_FINISHED_INTERNAL)) {
+ PRINT_ERROR("%s() received invalid cmd state %d (opcode %d)",
+ __func__, next_state, cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ }
+#endif
+ pref_context = scst_optimize_post_exec_context(cmd, pref_context);
+ scst_process_redirect_cmd(cmd, pref_context, 0);
+ return;
+}
+
+static int scst_report_luns_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_COMPLETED, rc;
+ int dev_cnt = 0;
+ int buffer_size;
+ int i;
+ struct scst_tgt_dev *tgt_dev = NULL;
+ uint8_t *buffer;
+ int offs, overflow = 0;
+
+ if (scst_cmd_atomic(cmd)) {
+ res = SCST_EXEC_NEED_THREAD;
+ goto out;
+ }
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ if ((cmd->cdb[2] != 0) && (cmd->cdb[2] != 2)) {
+ PRINT_ERROR("Unsupported SELECT REPORT value %x in REPORT "
+ "LUNS command", cmd->cdb[2]);
+ goto out_err;
+ }
+
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size == 0))
+ goto out_compl;
+ else if (unlikely(buffer_size < 0))
+ goto out_hw_err;
+
+ if (buffer_size < 16)
+ goto out_put_err;
+
+ memset(buffer, 0, buffer_size);
+ offs = 8;
+
+ /*
+ * cmd won't allow to suspend activities, so we can access
+ * sess->sess_tgt_dev_list_hash without any additional protection.
+ */
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (!overflow) {
+ if (offs >= buffer_size) {
+ scst_put_buf(cmd, buffer);
+ buffer_size = scst_get_buf_next(cmd,
+ &buffer);
+ if (buffer_size > 0) {
+ memset(buffer, 0, buffer_size);
+ offs = 0;
+ } else {
+ overflow = 1;
+ goto inc_dev_cnt;
+ }
+ }
+ if ((buffer_size - offs) < 8) {
+ PRINT_ERROR("Buffer allocated for "
+ "REPORT LUNS command doesn't "
+ "allow to fit 8 byte entry "
+ "(buffer_size=%d)",
+ buffer_size);
+ goto out_put_hw_err;
+ }
+ if ((cmd->sess->acg->addr_method == SCST_LUN_ADDR_METHOD_FLAT) &&
+ (tgt_dev->lun != 0)) {
+ buffer[offs] = (tgt_dev->lun >> 8) & 0x3f;
+ buffer[offs] = buffer[offs] | 0x40;
+ buffer[offs+1] = tgt_dev->lun & 0xff;
+ } else {
+ buffer[offs] = (tgt_dev->lun >> 8) & 0xff;
+ buffer[offs+1] = tgt_dev->lun & 0xff;
+ }
+ offs += 8;
+ }
+inc_dev_cnt:
+ dev_cnt++;
+ }
+ }
+ if (!overflow)
+ scst_put_buf(cmd, buffer);
+
+ /* Set the response header */
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size == 0))
+ goto out_compl;
+ else if (unlikely(buffer_size < 0))
+ goto out_hw_err;
+
+ dev_cnt *= 8;
+ buffer[0] = (dev_cnt >> 24) & 0xff;
+ buffer[1] = (dev_cnt >> 16) & 0xff;
+ buffer[2] = (dev_cnt >> 8) & 0xff;
+ buffer[3] = dev_cnt & 0xff;
+
+ scst_put_buf(cmd, buffer);
+
+ dev_cnt += 8;
+ if (dev_cnt < cmd->resp_data_len)
+ scst_set_resp_data_len(cmd, dev_cnt);
+
+out_compl:
+ cmd->completed = 1;
+
+ /* Clear left sense_reported_luns_data_changed UA, if any. */
+
+ /*
+ * cmd won't allow to suspend activities, so we can access
+ * sess->sess_tgt_dev_list_hash without any additional protection.
+ */
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ struct scst_tgt_dev_UA *ua;
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ list_for_each_entry(ua, &tgt_dev->UA_list,
+ UA_list_entry) {
+ if (scst_analyze_sense(ua->UA_sense_buffer,
+ ua->UA_valid_sense_len,
+ SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed))) {
+ TRACE_MGMT_DBG("Freeing not needed "
+ "REPORTED LUNS DATA CHANGED UA "
+ "%p", ua);
+ list_del(&ua->UA_list_entry);
+ mempool_free(ua, scst_ua_mempool);
+ break;
+ }
+ }
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+
+out:
+ return res;
+
+out_put_err:
+ scst_put_buf(cmd, buffer);
+
+out_err:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_compl;
+
+out_put_hw_err:
+ scst_put_buf(cmd, buffer);
+
+out_hw_err:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_compl;
+}
+
+static int scst_request_sense_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_COMPLETED, rc;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ uint8_t *buffer;
+ int buffer_size = 0, sl = 0;
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+
+ if (tgt_dev->tgt_dev_valid_sense_len == 0)
+ goto out_not_completed;
+
+ TRACE(TRACE_SCSI, "%s: Returning stored sense", cmd->op_name);
+
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size == 0))
+ goto out_compl;
+ else if (unlikely(buffer_size < 0))
+ goto out_hw_err;
+
+ memset(buffer, 0, buffer_size);
+
+ if (((tgt_dev->tgt_dev_sense[0] == 0x70) ||
+ (tgt_dev->tgt_dev_sense[0] == 0x71)) && (cmd->cdb[1] & 1)) {
+ PRINT_WARNING("%s: Fixed format of the saved sense, but "
+ "descriptor format requested. Convertion will "
+ "truncated data", cmd->op_name);
+ PRINT_BUFFER("Original sense", tgt_dev->tgt_dev_sense,
+ tgt_dev->tgt_dev_valid_sense_len);
+
+ buffer_size = min(SCST_STANDARD_SENSE_LEN, buffer_size);
+ sl = scst_set_sense(buffer, buffer_size, true,
+ tgt_dev->tgt_dev_sense[2], tgt_dev->tgt_dev_sense[12],
+ tgt_dev->tgt_dev_sense[13]);
+ } else if (((tgt_dev->tgt_dev_sense[0] == 0x72) ||
+ (tgt_dev->tgt_dev_sense[0] == 0x73)) && !(cmd->cdb[1] & 1)) {
+ PRINT_WARNING("%s: Descriptor format of the "
+ "saved sense, but fixed format requested. Convertion "
+ "will truncated data", cmd->op_name);
+ PRINT_BUFFER("Original sense", tgt_dev->tgt_dev_sense,
+ tgt_dev->tgt_dev_valid_sense_len);
+
+ buffer_size = min(SCST_STANDARD_SENSE_LEN, buffer_size);
+ sl = scst_set_sense(buffer, buffer_size, false,
+ tgt_dev->tgt_dev_sense[1], tgt_dev->tgt_dev_sense[2],
+ tgt_dev->tgt_dev_sense[3]);
+ } else {
+ if (buffer_size >= tgt_dev->tgt_dev_valid_sense_len)
+ sl = tgt_dev->tgt_dev_valid_sense_len;
+ else {
+ sl = buffer_size;
+ PRINT_WARNING("%s: Being returned sense truncated to "
+ "size %d (needed %d)", cmd->op_name,
+ buffer_size, tgt_dev->tgt_dev_valid_sense_len);
+ }
+ memcpy(buffer, tgt_dev->tgt_dev_sense, sl);
+ }
+
+ scst_put_buf(cmd, buffer);
+
+ tgt_dev->tgt_dev_valid_sense_len = 0;
+
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+
+ scst_set_resp_data_len(cmd, sl);
+
+out_compl:
+ cmd->completed = 1;
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+
+out:
+ return res;
+
+out_hw_err:
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_compl;
+
+out_not_completed:
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ res = SCST_EXEC_NOT_COMPLETED;
+ goto out;
+}
+
+static int scst_pre_select(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED;
+
+ if (scst_cmd_atomic(cmd)) {
+ res = SCST_EXEC_NEED_THREAD;
+ goto out;
+ }
+
+ scst_block_dev_cmd(cmd, 1);
+
+ /* Check for local events will be done when cmd will be executed */
+
+out:
+ return res;
+}
+
+static int scst_reserve_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED, rc;
+ struct scst_device *dev;
+ struct scst_tgt_dev *tgt_dev_tmp;
+
+ if (scst_cmd_atomic(cmd)) {
+ res = SCST_EXEC_NEED_THREAD;
+ goto out;
+ }
+
+ if ((cmd->cdb[0] == RESERVE_10) && (cmd->cdb[2] & SCST_RES_3RDPTY)) {
+ PRINT_ERROR("RESERVE_10: 3rdPty RESERVE not implemented "
+ "(lun=%lld)", (long long unsigned int)cmd->lun);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ goto out_done;
+ }
+
+ dev = cmd->dev;
+
+ if (dev->tst == SCST_CONTR_MODE_ONE_TASK_SET)
+ scst_block_dev_cmd(cmd, 1);
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ spin_lock_bh(&dev->dev_lock);
+
+ if (test_bit(SCST_TGT_DEV_RESERVED, &cmd->tgt_dev->tgt_dev_flags)) {
+ spin_unlock_bh(&dev->dev_lock);
+ scst_set_cmd_error_status(cmd, SAM_STAT_RESERVATION_CONFLICT);
+ goto out_done;
+ }
+
+ list_for_each_entry(tgt_dev_tmp, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (cmd->tgt_dev != tgt_dev_tmp)
+ set_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 1;
+
+ spin_unlock_bh(&dev->dev_lock);
+
+out:
+ return res;
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ res = SCST_EXEC_COMPLETED;
+ goto out;
+}
+
+static int scst_release_local(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED, rc;
+ struct scst_tgt_dev *tgt_dev_tmp;
+ struct scst_device *dev;
+
+ if (scst_cmd_atomic(cmd)) {
+ res = SCST_EXEC_NEED_THREAD;
+ goto out;
+ }
+
+ dev = cmd->dev;
+
+ if (dev->tst == SCST_CONTR_MODE_ONE_TASK_SET)
+ scst_block_dev_cmd(cmd, 1);
+
+ rc = scst_check_local_events(cmd);
+ if (unlikely(rc != 0))
+ goto out_done;
+
+ spin_lock_bh(&dev->dev_lock);
+
+ /*
+ * The device could be RELEASED behind us, if RESERVING session
+ * is closed (see scst_free_tgt_dev()), but this actually doesn't
+ * matter, so use lock and no retest for DEV_RESERVED bits again
+ */
+ if (test_bit(SCST_TGT_DEV_RESERVED, &cmd->tgt_dev->tgt_dev_flags)) {
+ res = SCST_EXEC_COMPLETED;
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+ cmd->completed = 1;
+ } else {
+ list_for_each_entry(tgt_dev_tmp,
+ &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ }
+
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (res == SCST_EXEC_COMPLETED)
+ goto out_done;
+
+out:
+ return res;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out;
+}
+
+/**
+ * scst_check_local_events() - check if there are any local SCSI events
+ *
+ * Description:
+ * Checks if the command can be executed or there are local events,
+ * like reservatons, pending UAs, etc. Returns < 0 if command must be
+ * aborted, > 0 if there is an event and command should be immediately
+ * completed, or 0 otherwise.
+ *
+ * !! Dev handlers implementing exec() callback must call this function there
+ * !! just before the actual command's execution!
+ *
+ * On call no locks, no IRQ or IRQ-disabled context allowed.
+ */
+int scst_check_local_events(struct scst_cmd *cmd)
+{
+ int res, rc;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_device *dev = cmd->dev;
+
+ /*
+ * There's no race here, because we need to trace commands sent
+ * *after* dev_double_ua_possible flag was set.
+ */
+ if (unlikely(dev->dev_double_ua_possible))
+ cmd->double_ua_possible = 1;
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("ABORTED set, aborting cmd %p", cmd);
+ goto out_uncomplete;
+ }
+
+ /* Reserve check before Unit Attention */
+ if (unlikely(test_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev->tgt_dev_flags))) {
+ if ((cmd->op_flags & SCST_REG_RESERVE_ALLOWED) == 0) {
+ scst_set_cmd_error_status(cmd,
+ SAM_STAT_RESERVATION_CONFLICT);
+ goto out_complete;
+ }
+ }
+
+ /* If we had internal bus reset, set the command error unit attention */
+ if ((dev->scsi_dev != NULL) &&
+ unlikely(dev->scsi_dev->was_reset)) {
+ if (scst_is_ua_command(cmd)) {
+ int done = 0;
+ /*
+ * Prevent more than 1 cmd to be triggered by
+ * was_reset.
+ */
+ spin_lock_bh(&dev->dev_lock);
+ if (dev->scsi_dev->was_reset) {
+ TRACE(TRACE_MGMT, "was_reset is %d", 1);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_reset_UA));
+ /*
+ * It looks like it is safe to clear was_reset
+ * here.
+ */
+ dev->scsi_dev->was_reset = 0;
+ done = 1;
+ }
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (done)
+ goto out_complete;
+ }
+ }
+
+ if (unlikely(test_bit(SCST_TGT_DEV_UA_PENDING,
+ &cmd->tgt_dev->tgt_dev_flags))) {
+ if (scst_is_ua_command(cmd)) {
+ rc = scst_set_pending_UA(cmd);
+ if (rc == 0)
+ goto out_complete;
+ }
+ }
+
+ res = 0;
+
+out:
+ return res;
+
+out_complete:
+ res = 1;
+ BUG_ON(!cmd->completed);
+ goto out;
+
+out_uncomplete:
+ res = -1;
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_check_local_events);
+
+/* No locks */
+void scst_inc_expected_sn(struct scst_tgt_dev *tgt_dev, atomic_t *slot)
+{
+ if (slot == NULL)
+ goto inc;
+
+ /* Optimized for lockless fast path */
+
+ TRACE_SN("Slot %zd, *cur_sn_slot %d", slot - tgt_dev->sn_slots,
+ atomic_read(slot));
+
+ if (!atomic_dec_and_test(slot))
+ goto out;
+
+ TRACE_SN("Slot is 0 (num_free_sn_slots=%d)",
+ tgt_dev->num_free_sn_slots);
+ if (tgt_dev->num_free_sn_slots < (int)ARRAY_SIZE(tgt_dev->sn_slots)-1) {
+ spin_lock_irq(&tgt_dev->sn_lock);
+ if (likely(tgt_dev->num_free_sn_slots < (int)ARRAY_SIZE(tgt_dev->sn_slots)-1)) {
+ if (tgt_dev->num_free_sn_slots < 0)
+ tgt_dev->cur_sn_slot = slot;
+ /*
+ * To be in-sync with SIMPLE case in scst_cmd_set_sn()
+ */
+ smp_mb();
+ tgt_dev->num_free_sn_slots++;
+ TRACE_SN("Incremented num_free_sn_slots (%d)",
+ tgt_dev->num_free_sn_slots);
+
+ }
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ }
+
+inc:
+ /*
+ * No protection of expected_sn is needed, because only one thread
+ * at time can be here (serialized by sn). Also it is supposed that
+ * there could not be half-incremented halves.
+ */
+ tgt_dev->expected_sn++;
+ /*
+ * Write must be before def_cmd_count read to be in sync. with
+ * scst_post_exec_sn(). See comment in scst_send_for_exec().
+ */
+ smp_mb();
+ TRACE_SN("Next expected_sn: %d", tgt_dev->expected_sn);
+
+out:
+ return;
+}
+
+/* No locks */
+static struct scst_cmd *scst_post_exec_sn(struct scst_cmd *cmd,
+ bool make_active)
+{
+ /* For HQ commands SN is not set */
+ bool inc_expected_sn = !cmd->inc_expected_sn_on_done &&
+ cmd->sn_set && !cmd->retry;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ struct scst_cmd *res;
+
+ if (inc_expected_sn)
+ scst_inc_expected_sn(tgt_dev, cmd->sn_slot);
+
+ if (make_active) {
+ scst_make_deferred_commands_active(tgt_dev);
+ res = NULL;
+ } else
+ res = scst_check_deferred_commands(tgt_dev);
+ return res;
+}
+
+/* cmd must be additionally referenced to not die inside */
+static int scst_do_real_exec(struct scst_cmd *cmd)
+{
+ int res = SCST_EXEC_NOT_COMPLETED;
+ int rc;
+ bool atomic = scst_cmd_atomic(cmd);
+ struct scst_device *dev = cmd->dev;
+ struct scst_dev_type *handler = dev->handler;
+ struct io_context *old_ctx = NULL;
+ bool ctx_changed = false;
+
+ if (!atomic)
+ ctx_changed = scst_set_io_context(cmd, &old_ctx);
+
+ cmd->state = SCST_CMD_STATE_REAL_EXECUTING;
+
+ if (handler->exec) {
+ if (unlikely(!dev->handler->exec_atomic && atomic)) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Dev handler %s exec() needs thread "
+ "context, rescheduling", dev->handler->name);
+ res = SCST_EXEC_NEED_THREAD;
+ goto out_restore;
+ }
+
+ TRACE_DBG("Calling dev handler %s exec(%p)",
+ handler->name, cmd);
+ TRACE_BUFF_FLAG(TRACE_SND_TOP, "Execing: ", cmd->cdb,
+ cmd->cdb_len);
+ scst_set_cur_start(cmd);
+ res = handler->exec(cmd);
+ TRACE_DBG("Dev handler %s exec() returned %d",
+ handler->name, res);
+
+ if (res == SCST_EXEC_COMPLETED)
+ goto out_complete;
+ else if (res == SCST_EXEC_NEED_THREAD)
+ goto out_restore;
+
+ scst_set_exec_time(cmd);
+
+ BUG_ON(res != SCST_EXEC_NOT_COMPLETED);
+ }
+
+ TRACE_DBG("Sending cmd %p to SCSI mid-level", cmd);
+
+ if (unlikely(dev->scsi_dev == NULL)) {
+ PRINT_ERROR("Command for virtual device must be "
+ "processed by device handler (LUN %lld)!",
+ (long long unsigned int)cmd->lun);
+ goto out_error;
+ }
+
+ res = scst_check_local_events(cmd);
+ if (unlikely(res != 0))
+ goto out_done;
+
+#ifndef CONFIG_SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ
+ if (unlikely(atomic)) {
+ TRACE_DBG("Pass-through exec() can not be called in atomic "
+ "context, rescheduling to the thread (handler %s)",
+ handler->name);
+ res = SCST_EXEC_NEED_THREAD;
+ goto out_restore;
+ }
+#endif
+
+ scst_set_cur_start(cmd);
+
+ rc = scst_scsi_exec_async(cmd, scst_cmd_done);
+ if (unlikely(rc != 0)) {
+ if (atomic) {
+ res = SCST_EXEC_NEED_THREAD;
+ goto out_restore;
+ } else {
+ PRINT_ERROR("scst pass-through exec failed: %x", rc);
+ goto out_error;
+ }
+ }
+
+out_complete:
+ res = SCST_EXEC_COMPLETED;
+
+out_reset_ctx:
+ if (ctx_changed)
+ scst_reset_io_context(cmd->tgt_dev, old_ctx);
+ return res;
+
+out_restore:
+ scst_set_exec_time(cmd);
+ /* Restore the state */
+ cmd->state = SCST_CMD_STATE_REAL_EXEC;
+ goto out_reset_ctx;
+
+out_error:
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ goto out_done;
+
+out_done:
+ res = SCST_EXEC_COMPLETED;
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ goto out_complete;
+}
+
+static inline int scst_real_exec(struct scst_cmd *cmd)
+{
+ int res;
+
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_SAME != SCST_EXEC_NOT_COMPLETED);
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_NEXT != SCST_EXEC_COMPLETED);
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_NEED_THREAD != SCST_EXEC_NEED_THREAD);
+
+ __scst_cmd_get(cmd);
+
+ res = scst_do_real_exec(cmd);
+
+ if (likely(res == SCST_EXEC_COMPLETED)) {
+ scst_post_exec_sn(cmd, true);
+ if (cmd->dev->scsi_dev != NULL)
+ generic_unplug_device(
+ cmd->dev->scsi_dev->request_queue);
+ } else
+ BUG_ON(res != SCST_EXEC_NEED_THREAD);
+
+ __scst_cmd_put(cmd);
+
+ /* SCST_EXEC_* match SCST_CMD_STATE_RES_* */
+ return res;
+}
+
+static int scst_do_local_exec(struct scst_cmd *cmd)
+{
+ int res;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ /* Check READ_ONLY device status */
+ if ((cmd->op_flags & SCST_WRITE_MEDIUM) &&
+ (tgt_dev->acg_dev->rd_only || cmd->dev->swp ||
+ cmd->dev->rd_only)) {
+ PRINT_WARNING("Attempt of write access to read-only device: "
+ "initiator %s, LUN %lld, op %x",
+ cmd->sess->initiator_name, cmd->lun, cmd->cdb[0]);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_data_protect));
+ goto out_done;
+ }
+
+ if (!scst_is_cmd_local(cmd)) {
+ res = SCST_EXEC_NOT_COMPLETED;
+ goto out;
+ }
+
+ switch (cmd->cdb[0]) {
+ case MODE_SELECT:
+ case MODE_SELECT_10:
+ case LOG_SELECT:
+ res = scst_pre_select(cmd);
+ break;
+ case RESERVE:
+ case RESERVE_10:
+ res = scst_reserve_local(cmd);
+ break;
+ case RELEASE:
+ case RELEASE_10:
+ res = scst_release_local(cmd);
+ break;
+ case REPORT_LUNS:
+ res = scst_report_luns_local(cmd);
+ break;
+ case REQUEST_SENSE:
+ res = scst_request_sense_local(cmd);
+ break;
+ default:
+ res = SCST_EXEC_NOT_COMPLETED;
+ break;
+ }
+
+out:
+ return res;
+
+out_done:
+ /* Report the result */
+ cmd->scst_cmd_done(cmd, SCST_CMD_STATE_DEFAULT, SCST_CONTEXT_SAME);
+ res = SCST_EXEC_COMPLETED;
+ goto out;
+}
+
+static int scst_local_exec(struct scst_cmd *cmd)
+{
+ int res;
+
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_SAME != SCST_EXEC_NOT_COMPLETED);
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_CONT_NEXT != SCST_EXEC_COMPLETED);
+ BUILD_BUG_ON(SCST_CMD_STATE_RES_NEED_THREAD != SCST_EXEC_NEED_THREAD);
+
+ __scst_cmd_get(cmd);
+
+ res = scst_do_local_exec(cmd);
+ if (likely(res == SCST_EXEC_NOT_COMPLETED))
+ cmd->state = SCST_CMD_STATE_REAL_EXEC;
+ else if (res == SCST_EXEC_COMPLETED)
+ scst_post_exec_sn(cmd, true);
+ else
+ BUG_ON(res != SCST_EXEC_NEED_THREAD);
+
+ __scst_cmd_put(cmd);
+
+ /* SCST_EXEC_* match SCST_CMD_STATE_RES_* */
+ return res;
+}
+
+static int scst_exec(struct scst_cmd **active_cmd)
+{
+ struct scst_cmd *cmd = *active_cmd;
+ struct scst_cmd *ref_cmd;
+ struct scst_device *dev = cmd->dev;
+ int res = SCST_CMD_STATE_RES_CONT_NEXT, count;
+
+ if (unlikely(scst_inc_on_dev_cmd(cmd) != 0))
+ goto out;
+
+ /* To protect tgt_dev */
+ ref_cmd = cmd;
+ __scst_cmd_get(ref_cmd);
+
+ count = 0;
+ while (1) {
+ int rc;
+
+ cmd->sent_for_exec = 1;
+ /*
+ * To sync with scst_abort_cmd(). The above assignment must
+ * be before SCST_CMD_ABORTED test, done later in
+ * scst_check_local_events(). It's far from here, so the order
+ * is virtually guaranteed, but let's have it just in case.
+ */
+ smp_mb();
+
+ cmd->scst_cmd_done = scst_cmd_done_local;
+ cmd->state = SCST_CMD_STATE_LOCAL_EXEC;
+
+ rc = scst_do_local_exec(cmd);
+ if (likely(rc == SCST_EXEC_NOT_COMPLETED))
+ /* Nothing to do */;
+ else if (rc == SCST_EXEC_NEED_THREAD) {
+ TRACE_DBG("%s", "scst_do_local_exec() requested "
+ "thread context, rescheduling");
+ scst_dec_on_dev_cmd(cmd);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+ } else {
+ BUG_ON(rc != SCST_EXEC_COMPLETED);
+ goto done;
+ }
+
+ cmd->state = SCST_CMD_STATE_REAL_EXEC;
+
+ rc = scst_do_real_exec(cmd);
+ if (likely(rc == SCST_EXEC_COMPLETED))
+ /* Nothing to do */;
+ else if (rc == SCST_EXEC_NEED_THREAD) {
+ TRACE_DBG("scst_real_exec() requested thread "
+ "context, rescheduling (cmd %p)", cmd);
+ scst_dec_on_dev_cmd(cmd);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+ } else
+ BUG();
+
+done:
+ count++;
+
+ cmd = scst_post_exec_sn(cmd, false);
+ if (cmd == NULL)
+ break;
+
+ if (unlikely(scst_inc_on_dev_cmd(cmd) != 0))
+ break;
+
+ __scst_cmd_put(ref_cmd);
+ ref_cmd = cmd;
+ __scst_cmd_get(ref_cmd);
+ }
+
+ *active_cmd = cmd;
+
+ if (count == 0)
+ goto out_put;
+
+ if (dev->scsi_dev != NULL)
+ generic_unplug_device(dev->scsi_dev->request_queue);
+
+out_put:
+ __scst_cmd_put(ref_cmd);
+ /* !! At this point sess, dev and tgt_dev can be already freed !! */
+
+out:
+ return res;
+}
+
+static int scst_send_for_exec(struct scst_cmd **active_cmd)
+{
+ int res;
+ struct scst_cmd *cmd = *active_cmd;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ typeof(tgt_dev->expected_sn) expected_sn;
+
+ if (unlikely(cmd->internal))
+ goto exec;
+
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ goto exec;
+
+ BUG_ON(!cmd->sn_set);
+
+ expected_sn = tgt_dev->expected_sn;
+ /* Optimized for lockless fast path */
+ if ((cmd->sn != expected_sn) || (tgt_dev->hq_cmd_count > 0)) {
+ spin_lock_irq(&tgt_dev->sn_lock);
+
+ tgt_dev->def_cmd_count++;
+ /*
+ * Memory barrier is needed here to implement lockless fast
+ * path. We need the exact order of read and write between
+ * def_cmd_count and expected_sn. Otherwise, we can miss case,
+ * when expected_sn was changed to be equal to cmd->sn while
+ * we are queuing cmd the deferred list after the expected_sn
+ * below. It will lead to a forever stuck command. But with
+ * the barrier in such case __scst_check_deferred_commands()
+ * will be called and it will take sn_lock, so we will be
+ * synchronized.
+ */
+ smp_mb();
+
+ expected_sn = tgt_dev->expected_sn;
+ if ((cmd->sn != expected_sn) || (tgt_dev->hq_cmd_count > 0)) {
+ if (unlikely(test_bit(SCST_CMD_ABORTED,
+ &cmd->cmd_flags))) {
+ /* Necessary to allow aborting out of sn cmds */
+ TRACE_MGMT_DBG("Aborting out of sn cmd %p "
+ "(tag %llu, sn %u)", cmd,
+ (long long unsigned)cmd->tag, cmd->sn);
+ tgt_dev->def_cmd_count--;
+ scst_set_cmd_abnormal_done_state(cmd);
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ } else {
+ TRACE_SN("Deferring cmd %p (sn=%d, set %d, "
+ "expected_sn=%d)", cmd, cmd->sn,
+ cmd->sn_set, expected_sn);
+ list_add_tail(&cmd->sn_cmd_list_entry,
+ &tgt_dev->deferred_cmd_list);
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ }
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ goto out;
+ } else {
+ TRACE_SN("Somebody incremented expected_sn %d, "
+ "continuing", expected_sn);
+ tgt_dev->def_cmd_count--;
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ }
+ }
+
+exec:
+ res = scst_exec(active_cmd);
+
+out:
+ return res;
+}
+
+/* No locks supposed to be held */
+static int scst_check_sense(struct scst_cmd *cmd)
+{
+ int res = 0;
+ struct scst_device *dev = cmd->dev;
+
+ if (unlikely(cmd->ua_ignore))
+ goto out;
+
+ /* If we had internal bus reset behind us, set the command error UA */
+ if ((dev->scsi_dev != NULL) &&
+ unlikely(cmd->host_status == DID_RESET) &&
+ scst_is_ua_command(cmd)) {
+ TRACE(TRACE_MGMT, "DID_RESET: was_reset=%d host_status=%x",
+ dev->scsi_dev->was_reset, cmd->host_status);
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_reset_UA));
+ /* It looks like it is safe to clear was_reset here */
+ dev->scsi_dev->was_reset = 0;
+ }
+
+ if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ SCST_SENSE_VALID(cmd->sense)) {
+ PRINT_BUFF_FLAG(TRACE_SCSI, "Sense", cmd->sense,
+ cmd->sense_valid_len);
+
+ /* Check Unit Attention Sense Key */
+ if (scst_is_ua_sense(cmd->sense, cmd->sense_valid_len)) {
+ if (scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, SCST_SENSE_ASC_UA_RESET, 0)) {
+ if (cmd->double_ua_possible) {
+ TRACE_MGMT_DBG("Double UA "
+ "detected for device %p", dev);
+ TRACE_MGMT_DBG("Retrying cmd"
+ " %p (tag %llu)", cmd,
+ (long long unsigned)cmd->tag);
+
+ cmd->status = 0;
+ cmd->msg_status = 0;
+ cmd->host_status = DID_OK;
+ cmd->driver_status = 0;
+ cmd->completed = 0;
+
+ mempool_free(cmd->sense,
+ scst_sense_mempool);
+ cmd->sense = NULL;
+
+ scst_check_restore_sg_buff(cmd);
+
+ BUG_ON(cmd->dbl_ua_orig_resp_data_len < 0);
+ cmd->data_direction =
+ cmd->dbl_ua_orig_data_direction;
+ cmd->resp_data_len =
+ cmd->dbl_ua_orig_resp_data_len;
+
+ cmd->state = SCST_CMD_STATE_REAL_EXEC;
+ cmd->retry = 1;
+ res = 1;
+ goto out;
+ }
+ }
+ scst_dev_check_set_UA(dev, cmd, cmd->sense,
+ cmd->sense_valid_len);
+ }
+ }
+
+ if (unlikely(cmd->double_ua_possible)) {
+ if (scst_is_ua_command(cmd)) {
+ TRACE_MGMT_DBG("Clearing dbl_ua_possible flag (dev %p, "
+ "cmd %p)", dev, cmd);
+ /*
+ * Lock used to protect other flags in the bitfield
+ * (just in case, actually). Those flags can't be
+ * changed in parallel, because the device is
+ * serialized.
+ */
+ spin_lock_bh(&dev->dev_lock);
+ dev->dev_double_ua_possible = 0;
+ spin_unlock_bh(&dev->dev_lock);
+ }
+ }
+
+out:
+ return res;
+}
+
+static int scst_check_auto_sense(struct scst_cmd *cmd)
+{
+ int res = 0;
+
+ if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ (!SCST_SENSE_VALID(cmd->sense) ||
+ SCST_NO_SENSE(cmd->sense))) {
+ TRACE(TRACE_SCSI|TRACE_MINOR_AND_MGMT_DBG, "CHECK_CONDITION, "
+ "but no sense: cmd->status=%x, cmd->msg_status=%x, "
+ "cmd->host_status=%x, cmd->driver_status=%x (cmd %p)",
+ cmd->status, cmd->msg_status, cmd->host_status,
+ cmd->driver_status, cmd);
+ res = 1;
+ } else if (unlikely(cmd->host_status)) {
+ if ((cmd->host_status == DID_REQUEUE) ||
+ (cmd->host_status == DID_IMM_RETRY) ||
+ (cmd->host_status == DID_SOFT_ERROR) ||
+ (cmd->host_status == DID_ABORT)) {
+ scst_set_busy(cmd);
+ } else {
+ TRACE(TRACE_SCSI|TRACE_MINOR_AND_MGMT_DBG, "Host "
+ "status %x received, returning HARDWARE ERROR "
+ "instead (cmd %p)", cmd->host_status, cmd);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ }
+ return res;
+}
+
+static int scst_pre_dev_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME, rc;
+
+ if (unlikely(scst_check_auto_sense(cmd))) {
+ PRINT_INFO("Command finished with CHECK CONDITION, but "
+ "without sense data (opcode 0x%x), issuing "
+ "REQUEST SENSE", cmd->cdb[0]);
+ rc = scst_prepare_request_sense(cmd);
+ if (rc == 0)
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ else {
+ PRINT_ERROR("%s", "Unable to issue REQUEST SENSE, "
+ "returning HARDWARE ERROR");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+ goto out;
+ } else if (unlikely(scst_check_sense(cmd)))
+ goto out;
+
+ if (likely(scsi_status_is_good(cmd->status))) {
+ unsigned char type = cmd->dev->type;
+ if (unlikely((cmd->cdb[0] == MODE_SENSE ||
+ cmd->cdb[0] == MODE_SENSE_10)) &&
+ (cmd->tgt_dev->acg_dev->rd_only || cmd->dev->swp ||
+ cmd->dev->rd_only) &&
+ (type == TYPE_DISK ||
+ type == TYPE_WORM ||
+ type == TYPE_MOD ||
+ type == TYPE_TAPE)) {
+ int32_t length;
+ uint8_t *address;
+ bool err = false;
+
+ length = scst_get_buf_first(cmd, &address);
+ if (length < 0) {
+ PRINT_ERROR("%s", "Unable to get "
+ "MODE_SENSE buffer");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(
+ scst_sense_hardw_error));
+ err = true;
+ } else if (length > 2 && cmd->cdb[0] == MODE_SENSE)
+ address[2] |= 0x80; /* Write Protect*/
+ else if (length > 3 && cmd->cdb[0] == MODE_SENSE_10)
+ address[3] |= 0x80; /* Write Protect*/
+ scst_put_buf(cmd, address);
+
+ if (err)
+ goto out;
+ }
+
+ /*
+ * Check and clear NormACA option for the device, if necessary,
+ * since we don't support ACA
+ */
+ if (unlikely((cmd->cdb[0] == INQUIRY)) &&
+ /* Std INQUIRY data (no EVPD) */
+ !(cmd->cdb[1] & SCST_INQ_EVPD) &&
+ (cmd->resp_data_len > SCST_INQ_BYTE3)) {
+ uint8_t *buffer;
+ int buflen;
+ bool err = false;
+
+ /* ToDo: all pages ?? */
+ buflen = scst_get_buf_first(cmd, &buffer);
+ if (buflen > SCST_INQ_BYTE3) {
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (buffer[SCST_INQ_BYTE3] & SCST_INQ_NORMACA_BIT) {
+ PRINT_INFO("NormACA set for device: "
+ "lun=%lld, type 0x%02x. Clear it, "
+ "since it's unsupported.",
+ (long long unsigned int)cmd->lun,
+ buffer[0]);
+ }
+#endif
+ buffer[SCST_INQ_BYTE3] &= ~SCST_INQ_NORMACA_BIT;
+ } else if (buflen != 0) {
+ PRINT_ERROR("%s", "Unable to get INQUIRY "
+ "buffer");
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ err = true;
+ }
+ if (buflen > 0)
+ scst_put_buf(cmd, buffer);
+
+ if (err)
+ goto out;
+ }
+
+ if (unlikely((cmd->cdb[0] == MODE_SELECT) ||
+ (cmd->cdb[0] == MODE_SELECT_10) ||
+ (cmd->cdb[0] == LOG_SELECT))) {
+ TRACE(TRACE_SCSI,
+ "MODE/LOG SELECT succeeded (LUN %lld)",
+ (long long unsigned int)cmd->lun);
+ cmd->state = SCST_CMD_STATE_MODE_SELECT_CHECKS;
+ goto out;
+ }
+ } else {
+ TRACE(TRACE_SCSI, "cmd %p not succeeded with status %x",
+ cmd, cmd->status);
+
+ if ((cmd->cdb[0] == RESERVE) || (cmd->cdb[0] == RESERVE_10)) {
+ if (!test_bit(SCST_TGT_DEV_RESERVED,
+ &cmd->tgt_dev->tgt_dev_flags)) {
+ struct scst_tgt_dev *tgt_dev_tmp;
+ struct scst_device *dev = cmd->dev;
+
+ TRACE(TRACE_SCSI, "RESERVE failed lun=%lld, "
+ "status=%x",
+ (long long unsigned int)cmd->lun,
+ cmd->status);
+ PRINT_BUFF_FLAG(TRACE_SCSI, "Sense", cmd->sense,
+ cmd->sense_valid_len);
+
+ /* Clearing the reservation */
+ spin_lock_bh(&dev->dev_lock);
+ list_for_each_entry(tgt_dev_tmp,
+ &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ spin_unlock_bh(&dev->dev_lock);
+ }
+ }
+
+ /* Check for MODE PARAMETERS CHANGED UA */
+ if ((cmd->dev->scsi_dev != NULL) &&
+ (cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ scst_is_ua_sense(cmd->sense, cmd->sense_valid_len) &&
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASCx_VALID,
+ 0, 0x2a, 0x01)) {
+ TRACE(TRACE_SCSI, "MODE PARAMETERS CHANGED UA (lun "
+ "%lld)", (long long unsigned int)cmd->lun);
+ cmd->state = SCST_CMD_STATE_MODE_SELECT_CHECKS;
+ goto out;
+ }
+ }
+
+ cmd->state = SCST_CMD_STATE_DEV_DONE;
+
+out:
+ return res;
+}
+
+static int scst_mode_select_checks(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+ int atomic = scst_cmd_atomic(cmd);
+
+ if (likely(scsi_status_is_good(cmd->status))) {
+ if (unlikely((cmd->cdb[0] == MODE_SELECT) ||
+ (cmd->cdb[0] == MODE_SELECT_10) ||
+ (cmd->cdb[0] == LOG_SELECT))) {
+ struct scst_device *dev = cmd->dev;
+ int sl;
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+
+ if (atomic && (dev->scsi_dev != NULL)) {
+ TRACE_DBG("%s", "MODE/LOG SELECT: thread "
+ "context required");
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE(TRACE_SCSI, "MODE/LOG SELECT succeeded, "
+ "setting the SELECT UA (lun=%lld)",
+ (long long unsigned int)cmd->lun);
+
+ spin_lock_bh(&dev->dev_lock);
+ if (cmd->cdb[0] == LOG_SELECT) {
+ sl = scst_set_sense(sense_buffer,
+ sizeof(sense_buffer),
+ dev->d_sense,
+ UNIT_ATTENTION, 0x2a, 0x02);
+ } else {
+ sl = scst_set_sense(sense_buffer,
+ sizeof(sense_buffer),
+ dev->d_sense,
+ UNIT_ATTENTION, 0x2a, 0x01);
+ }
+ scst_dev_check_set_local_UA(dev, cmd, sense_buffer, sl);
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (dev->scsi_dev != NULL)
+ scst_obtain_device_parameters(dev);
+ }
+ } else if ((cmd->status == SAM_STAT_CHECK_CONDITION) &&
+ scst_is_ua_sense(cmd->sense, cmd->sense_valid_len) &&
+ /* mode parameters changed */
+ (scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASCx_VALID,
+ 0, 0x2a, 0x01) ||
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, 0x29, 0) /* reset */ ||
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, 0x28, 0) /* medium changed */ ||
+ /* cleared by another ini (just in case) */
+ scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ASC_VALID,
+ 0, 0x2F, 0))) {
+ if (atomic) {
+ TRACE_DBG("Possible parameters changed UA %x: "
+ "thread context required", cmd->sense[12]);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE(TRACE_SCSI, "Possible parameters changed UA %x "
+ "(LUN %lld): getting new parameters", cmd->sense[12],
+ (long long unsigned int)cmd->lun);
+
+ scst_obtain_device_parameters(cmd->dev);
+ } else
+ BUG();
+
+ cmd->state = SCST_CMD_STATE_DEV_DONE;
+
+out:
+ return res;
+}
+
+static void scst_inc_check_expected_sn(struct scst_cmd *cmd)
+{
+ if (likely(cmd->sn_set))
+ scst_inc_expected_sn(cmd->tgt_dev, cmd->sn_slot);
+
+ scst_make_deferred_commands_active(cmd->tgt_dev);
+}
+
+static int scst_dev_done(struct scst_cmd *cmd)
+{
+ int res = SCST_CMD_STATE_RES_CONT_SAME;
+ int state;
+ struct scst_device *dev = cmd->dev;
+
+ state = SCST_CMD_STATE_PRE_XMIT_RESP;
+
+ if (likely(!scst_is_cmd_fully_local(cmd)) &&
+ likely(dev->handler->dev_done != NULL)) {
+ int rc;
+
+ if (unlikely(!dev->handler->dev_done_atomic &&
+ scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Dev handler %s dev_done() needs thread "
+ "context, rescheduling", dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ TRACE_DBG("Calling dev handler %s dev_done(%p)",
+ dev->handler->name, cmd);
+ scst_set_cur_start(cmd);
+ rc = dev->handler->dev_done(cmd);
+ scst_set_dev_done_time(cmd);
+ TRACE_DBG("Dev handler %s dev_done() returned %d",
+ dev->handler->name, rc);
+ if (rc != SCST_CMD_STATE_DEFAULT)
+ state = rc;
+ }
+
+ switch (state) {
+#ifdef CONFIG_SCST_EXTRACHECKS
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ case SCST_CMD_STATE_DEV_PARSE:
+ case SCST_CMD_STATE_PRE_PARSE:
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ case SCST_CMD_STATE_MODE_SELECT_CHECKS:
+ case SCST_CMD_STATE_DEV_DONE:
+ case SCST_CMD_STATE_XMIT_RESP:
+ case SCST_CMD_STATE_FINISHED:
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+#else
+ default:
+#endif
+ cmd->state = state;
+ break;
+ case SCST_CMD_STATE_NEED_THREAD_CTX:
+ TRACE_DBG("Dev handler %s dev_done() requested "
+ "thread context, rescheduling",
+ dev->handler->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ default:
+ if (state >= 0) {
+ PRINT_ERROR("Dev handler %s dev_done() returned "
+ "invalid cmd state %d",
+ dev->handler->name, state);
+ } else {
+ PRINT_ERROR("Dev handler %s dev_done() returned "
+ "error %d", dev->handler->name,
+ state);
+ }
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ scst_set_cmd_abnormal_done_state(cmd);
+ break;
+#endif
+ }
+
+ if (cmd->needs_unblocking)
+ scst_unblock_dev_cmd(cmd);
+
+ if (likely(cmd->dec_on_dev_needed))
+ scst_dec_on_dev_cmd(cmd);
+
+ if (cmd->inc_expected_sn_on_done && cmd->sent_for_exec)
+ scst_inc_check_expected_sn(cmd);
+
+ if (unlikely(cmd->internal))
+ cmd->state = SCST_CMD_STATE_FINISHED_INTERNAL;
+
+out:
+ return res;
+}
+
+static int scst_pre_xmit_response(struct scst_cmd *cmd)
+{
+ int res;
+
+ EXTRACHECKS_BUG_ON(cmd->internal);
+
+#ifdef CONFIG_SCST_DEBUG_TM
+ if (cmd->tm_dbg_delayed &&
+ !test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ if (scst_cmd_atomic(cmd)) {
+ TRACE_MGMT_DBG("%s",
+ "DEBUG_TM delayed cmd needs a thread");
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ return res;
+ }
+ TRACE_MGMT_DBG("Delaying cmd %p (tag %llu) for 1 second",
+ cmd, cmd->tag);
+ schedule_timeout_uninterruptible(HZ);
+ }
+#endif
+
+ if (likely(cmd->tgt_dev != NULL)) {
+ /*
+ * Those counters protect from not getting too long processing
+ * latency, so we should decrement them after cmd completed.
+ */
+ atomic_dec(&cmd->tgt_dev->tgt_dev_cmd_count);
+#ifdef CONFIG_SCST_PER_DEVICE_CMD_COUNT_LIMIT
+ atomic_dec(&cmd->dev->dev_cmd_count);
+#endif
+#ifdef CONFIG_SCST_ORDERED_READS
+ /* If expected values not set, expected direction is UNKNOWN */
+ if (cmd->expected_data_direction & SCST_DATA_WRITE)
+ atomic_dec(&cmd->dev->write_cmd_count);
+#endif
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ scst_on_hq_cmd_response(cmd);
+
+ if (unlikely(!cmd->sent_for_exec)) {
+ TRACE_SN("cmd %p was not sent to mid-lev"
+ " (sn %d, set %d)",
+ cmd, cmd->sn, cmd->sn_set);
+ scst_unblock_deferred(cmd->tgt_dev, cmd);
+ cmd->sent_for_exec = 1;
+ }
+ }
+
+ cmd->done = 1;
+ smp_mb(); /* to sync with scst_abort_cmd() */
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)))
+ scst_xmit_process_aborted_cmd(cmd);
+ else if (unlikely(cmd->status == SAM_STAT_CHECK_CONDITION))
+ scst_store_sense(cmd);
+
+ if (unlikely(test_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("Flag NO_RESP set for cmd %p (tag %llu),"
+ " skipping",
+ cmd, (long long unsigned int)cmd->tag);
+ cmd->state = SCST_CMD_STATE_FINISHED;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+ }
+
+ cmd->state = SCST_CMD_STATE_XMIT_RESP;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+
+out:
+ return res;
+}
+
+static int scst_xmit_response(struct scst_cmd *cmd)
+{
+ struct scst_tgt_template *tgtt = cmd->tgtt;
+ int res, rc;
+
+ EXTRACHECKS_BUG_ON(cmd->internal);
+
+ if (unlikely(!tgtt->xmit_response_atomic &&
+ scst_cmd_atomic(cmd))) {
+ /*
+ * It shouldn't be because of the SCST_TGT_DEV_AFTER_*
+ * optimization.
+ */
+ TRACE_DBG("Target driver %s xmit_response() needs thread "
+ "context, rescheduling", tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+
+ while (1) {
+ int finished_cmds = atomic_read(&cmd->tgt->finished_cmds);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ cmd->state = SCST_CMD_STATE_XMIT_WAIT;
+
+ TRACE_DBG("Calling xmit_response(%p)", cmd);
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (trace_flag & TRACE_SND_BOT) {
+ int i;
+ struct scatterlist *sg;
+ if (cmd->tgt_sg != NULL)
+ sg = cmd->tgt_sg;
+ else
+ sg = cmd->sg;
+ if (sg != NULL) {
+ TRACE(TRACE_SND_BOT, "Xmitting data for cmd %p "
+ "(sg_cnt %d, sg %p, sg[0].page %p)",
+ cmd, cmd->tgt_sg_cnt, sg,
+ (void *)sg_page(&sg[0]));
+ for (i = 0; i < cmd->tgt_sg_cnt; ++i) {
+ PRINT_BUFF_FLAG(TRACE_SND_BOT,
+ "Xmitting sg", sg_virt(&sg[i]),
+ sg[i].length);
+ }
+ }
+ }
+#endif
+
+ if (tgtt->on_hw_pending_cmd_timeout != NULL) {
+ struct scst_session *sess = cmd->sess;
+ cmd->hw_pending_start = jiffies;
+ cmd->cmd_hw_pending = 1;
+ if (!test_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags)) {
+ TRACE_DBG("Sched HW pending work for sess %p "
+ "(max time %d)", sess,
+ tgtt->max_hw_pending_time);
+ set_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED,
+ &sess->sess_aflags);
+ schedule_delayed_work(&sess->hw_pending_work,
+ tgtt->max_hw_pending_time * HZ);
+ }
+ }
+
+ scst_set_cur_start(cmd);
+
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ if (((scst_random() % 100) == 77))
+ rc = SCST_TGT_RES_QUEUE_FULL;
+ else
+#endif
+ rc = tgtt->xmit_response(cmd);
+ TRACE_DBG("xmit_response() returned %d", rc);
+
+ if (likely(rc == SCST_TGT_RES_SUCCESS))
+ goto out;
+
+ scst_set_xmit_time(cmd);
+
+ cmd->cmd_hw_pending = 0;
+
+ /* Restore the previous state */
+ cmd->state = SCST_CMD_STATE_XMIT_RESP;
+
+ switch (rc) {
+ case SCST_TGT_RES_QUEUE_FULL:
+ if (scst_queue_retry_cmd(cmd, finished_cmds) == 0)
+ break;
+ else
+ continue;
+
+ case SCST_TGT_RES_NEED_THREAD_CTX:
+ TRACE_DBG("Target driver %s xmit_response() "
+ "requested thread context, rescheduling",
+ tgtt->name);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ break;
+
+ default:
+ goto out_error;
+ }
+ break;
+ }
+
+out:
+ /* Caution: cmd can be already dead here */
+ return res;
+
+out_error:
+ if (rc == SCST_TGT_RES_FATAL_ERROR) {
+ PRINT_ERROR("Target driver %s xmit_response() returned "
+ "fatal error", tgtt->name);
+ } else {
+ PRINT_ERROR("Target driver %s xmit_response() returned "
+ "invalid value %d", tgtt->name, rc);
+ }
+ scst_set_cmd_error(cmd, SCST_LOAD_SENSE(scst_sense_hardw_error));
+ cmd->state = SCST_CMD_STATE_FINISHED;
+ res = SCST_CMD_STATE_RES_CONT_SAME;
+ goto out;
+}
+
+/**
+ * scst_tgt_cmd_done() - the command's processing done
+ * @cmd: SCST command
+ * @pref_context: preferred command execution context
+ *
+ * Description:
+ * Notifies SCST that the driver sent the response and the command
+ * can be freed now. Don't forget to set the delivery status, if it
+ * isn't success, using scst_set_delivery_status() before calling
+ * this function. The third argument sets preferred command execition
+ * context (see SCST_CONTEXT_* constants for details)
+ */
+void scst_tgt_cmd_done(struct scst_cmd *cmd,
+ enum scst_exec_context pref_context)
+{
+
+ BUG_ON(cmd->state != SCST_CMD_STATE_XMIT_WAIT);
+
+ scst_set_xmit_time(cmd);
+
+ cmd->cmd_hw_pending = 0;
+
+ cmd->state = SCST_CMD_STATE_FINISHED;
+ scst_process_redirect_cmd(cmd, pref_context, 1);
+ return;
+}
+EXPORT_SYMBOL(scst_tgt_cmd_done);
+
+static int scst_finish_cmd(struct scst_cmd *cmd)
+{
+ int res;
+ struct scst_session *sess = cmd->sess;
+
+ scst_update_lat_stats(cmd);
+
+ if (unlikely(cmd->delivery_status != SCST_CMD_DELIVERY_SUCCESS)) {
+ if ((cmd->tgt_dev != NULL) &&
+ scst_is_ua_sense(cmd->sense, cmd->sense_valid_len)) {
+ /* This UA delivery failed, so we need to requeue it */
+ if (scst_cmd_atomic(cmd) &&
+ scst_is_ua_global(cmd->sense, cmd->sense_valid_len)) {
+ TRACE_MGMT_DBG("Requeuing of global UA for "
+ "failed cmd %p needs a thread", cmd);
+ res = SCST_CMD_STATE_RES_NEED_THREAD;
+ goto out;
+ }
+ scst_requeue_ua(cmd);
+ }
+ }
+
+ atomic_dec(&sess->sess_cmd_count);
+
+ spin_lock_irq(&sess->sess_list_lock);
+ list_del(&cmd->sess_cmd_list_entry);
+ spin_unlock_irq(&sess->sess_list_lock);
+
+ cmd->finished = 1;
+ smp_mb(); /* to sync with scst_abort_cmd() */
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("Aborted cmd %p finished (cmd_ref %d, "
+ "scst_cmd_count %d)", cmd, atomic_read(&cmd->cmd_ref),
+ atomic_read(&scst_cmd_count));
+
+ scst_finish_cmd_mgmt(cmd);
+ }
+
+ __scst_cmd_put(cmd);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+
+out:
+ return res;
+}
+
+/*
+ * No locks, but it must be externally serialized (see comment for
+ * scst_cmd_init_done() in scst.h)
+ */
+static void scst_cmd_set_sn(struct scst_cmd *cmd)
+{
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ unsigned long flags;
+
+ if (scst_is_implicit_hq(cmd)) {
+ TRACE_SN("Implicit HQ cmd %p", cmd);
+ cmd->queue_type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ }
+
+ EXTRACHECKS_BUG_ON(cmd->sn_set || cmd->hq_cmd_inced);
+
+ /* Optimized for lockless fast path */
+
+ scst_check_debug_sn(cmd);
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ cmd->queue_type = SCST_CMD_QUEUE_ORDERED;
+#endif
+
+ if (cmd->dev->queue_alg == SCST_CONTR_MODE_QUEUE_ALG_RESTRICTED_REORDER) {
+ /*
+ * Not the best way, but good enough until there is a
+ * possibility to specify queue type during pass-through
+ * commands submission.
+ */
+ cmd->queue_type = SCST_CMD_QUEUE_ORDERED;
+ }
+
+ switch (cmd->queue_type) {
+ case SCST_CMD_QUEUE_SIMPLE:
+ case SCST_CMD_QUEUE_UNTAGGED:
+#ifdef CONFIG_SCST_ORDERED_READS
+ if (scst_cmd_is_expected_set(cmd)) {
+ if ((cmd->expected_data_direction == SCST_DATA_READ) &&
+ (atomic_read(&cmd->dev->write_cmd_count) == 0))
+ goto ordered;
+ } else
+ goto ordered;
+#endif
+ if (likely(tgt_dev->num_free_sn_slots >= 0)) {
+ /*
+ * atomic_inc_return() implies memory barrier to sync
+ * with scst_inc_expected_sn()
+ */
+ if (atomic_inc_return(tgt_dev->cur_sn_slot) == 1) {
+ tgt_dev->curr_sn++;
+ TRACE_SN("Incremented curr_sn %d",
+ tgt_dev->curr_sn);
+ }
+ cmd->sn_slot = tgt_dev->cur_sn_slot;
+ cmd->sn = tgt_dev->curr_sn;
+
+ tgt_dev->prev_cmd_ordered = 0;
+ } else {
+ TRACE(TRACE_MINOR, "***WARNING*** Not enough SN slots "
+ "%zd", ARRAY_SIZE(tgt_dev->sn_slots));
+ goto ordered;
+ }
+ break;
+
+ case SCST_CMD_QUEUE_ORDERED:
+ TRACE_SN("ORDERED cmd %p (op %x)", cmd, cmd->cdb[0]);
+ordered:
+ if (!tgt_dev->prev_cmd_ordered) {
+ spin_lock_irqsave(&tgt_dev->sn_lock, flags);
+ if (tgt_dev->num_free_sn_slots >= 0) {
+ tgt_dev->num_free_sn_slots--;
+ if (tgt_dev->num_free_sn_slots >= 0) {
+ int i = 0;
+ /* Commands can finish in any order, so
+ * we don't know which slot is empty.
+ */
+ while (1) {
+ tgt_dev->cur_sn_slot++;
+ if (tgt_dev->cur_sn_slot ==
+ tgt_dev->sn_slots + ARRAY_SIZE(tgt_dev->sn_slots))
+ tgt_dev->cur_sn_slot = tgt_dev->sn_slots;
+
+ if (atomic_read(tgt_dev->cur_sn_slot) == 0)
+ break;
+
+ i++;
+ BUG_ON(i == ARRAY_SIZE(tgt_dev->sn_slots));
+ }
+ TRACE_SN("New cur SN slot %zd",
+ tgt_dev->cur_sn_slot -
+ tgt_dev->sn_slots);
+ }
+ }
+ spin_unlock_irqrestore(&tgt_dev->sn_lock, flags);
+ }
+ tgt_dev->prev_cmd_ordered = 1;
+ tgt_dev->curr_sn++;
+ cmd->sn = tgt_dev->curr_sn;
+ break;
+
+ case SCST_CMD_QUEUE_HEAD_OF_QUEUE:
+ TRACE_SN("HQ cmd %p (op %x)", cmd, cmd->cdb[0]);
+ spin_lock_irqsave(&tgt_dev->sn_lock, flags);
+ tgt_dev->hq_cmd_count++;
+ spin_unlock_irqrestore(&tgt_dev->sn_lock, flags);
+ cmd->hq_cmd_inced = 1;
+ goto out;
+
+ default:
+ BUG();
+ }
+
+ TRACE_SN("cmd(%p)->sn: %d (tgt_dev %p, *cur_sn_slot %d, "
+ "num_free_sn_slots %d, prev_cmd_ordered %ld, "
+ "cur_sn_slot %zd)", cmd, cmd->sn, tgt_dev,
+ atomic_read(tgt_dev->cur_sn_slot),
+ tgt_dev->num_free_sn_slots, tgt_dev->prev_cmd_ordered,
+ tgt_dev->cur_sn_slot-tgt_dev->sn_slots);
+
+ cmd->sn_set = 1;
+
+out:
+ return;
+}
+
+/*
+ * Returns 0 on success, > 0 when we need to wait for unblock,
+ * < 0 if there is no device (lun) or device type handler.
+ *
+ * No locks, but might be on IRQ, protection is done by the
+ * suspended activity.
+ */
+static int scst_translate_lun(struct scst_cmd *cmd)
+{
+ struct scst_tgt_dev *tgt_dev = NULL;
+ int res;
+
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ __scst_get(1);
+
+ if (likely(!test_bit(SCST_FLAG_SUSPENDED, &scst_flags))) {
+ struct list_head *sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[HASH_VAL(cmd->lun)];
+ TRACE_DBG("Finding tgt_dev for cmd %p (lun %lld)", cmd,
+ (long long unsigned int)cmd->lun);
+ res = -1;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == cmd->lun) {
+ TRACE_DBG("tgt_dev %p found", tgt_dev);
+
+ if (unlikely(tgt_dev->dev->handler ==
+ &scst_null_devtype)) {
+ PRINT_INFO("Dev handler for device "
+ "%lld is NULL, the device will not "
+ "be visible remotely",
+ (long long unsigned int)cmd->lun);
+ break;
+ }
+
+ cmd->cmd_threads = tgt_dev->active_cmd_threads;
+ cmd->tgt_dev = tgt_dev;
+ cmd->dev = tgt_dev->dev;
+
+ res = 0;
+ break;
+ }
+ }
+ if (res != 0) {
+ TRACE(TRACE_MINOR,
+ "tgt_dev for LUN %lld not found, command to "
+ "unexisting LU?",
+ (long long unsigned int)cmd->lun);
+ __scst_put();
+ }
+ } else {
+ TRACE_MGMT_DBG("%s", "FLAG SUSPENDED set, skipping");
+ __scst_put();
+ res = 1;
+ }
+ return res;
+}
+
+/*
+ * No locks, but might be on IRQ.
+ *
+ * Returns 0 on success, > 0 when we need to wait for unblock,
+ * < 0 if there is no device (lun) or device type handler.
+ */
+static int __scst_init_cmd(struct scst_cmd *cmd)
+{
+ int res = 0;
+
+ res = scst_translate_lun(cmd);
+ if (likely(res == 0)) {
+ int cnt;
+ bool failure = false;
+
+ cmd->state = SCST_CMD_STATE_PRE_PARSE;
+
+ cnt = atomic_inc_return(&cmd->tgt_dev->tgt_dev_cmd_count);
+ if (unlikely(cnt > SCST_MAX_TGT_DEV_COMMANDS)) {
+ TRACE(TRACE_FLOW_CONTROL,
+ "Too many pending commands (%d) in "
+ "session, returning BUSY to initiator \"%s\"",
+ cnt, (cmd->sess->initiator_name[0] == '\0') ?
+ "Anonymous" : cmd->sess->initiator_name);
+ failure = true;
+ }
+
+#ifdef CONFIG_SCST_PER_DEVICE_CMD_COUNT_LIMIT
+ cnt = atomic_inc_return(&cmd->dev->dev_cmd_count);
+ if (unlikely(cnt > SCST_MAX_DEV_COMMANDS)) {
+ if (!failure) {
+ TRACE(TRACE_FLOW_CONTROL,
+ "Too many pending device "
+ "commands (%d), returning BUSY to "
+ "initiator \"%s\"", cnt,
+ (cmd->sess->initiator_name[0] == '\0') ?
+ "Anonymous" :
+ cmd->sess->initiator_name);
+ failure = true;
+ }
+ }
+#endif
+
+#ifdef CONFIG_SCST_ORDERED_READS
+ /* If expected values not set, expected direction is UNKNOWN */
+ if (cmd->expected_data_direction & SCST_DATA_WRITE)
+ atomic_inc(&cmd->dev->write_cmd_count);
+#endif
+
+ if (unlikely(failure))
+ goto out_busy;
+
+ if (!cmd->set_sn_on_restart_cmd)
+ scst_cmd_set_sn(cmd);
+ } else if (res < 0) {
+ TRACE_DBG("Finishing cmd %p", cmd);
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_lun_not_supported));
+ scst_set_cmd_abnormal_done_state(cmd);
+ } else
+ goto out;
+
+out:
+ return res;
+
+out_busy:
+ scst_set_busy(cmd);
+ scst_set_cmd_abnormal_done_state(cmd);
+ goto out;
+}
+
+/* Called under scst_init_lock and IRQs disabled */
+static void scst_do_job_init(void)
+ __releases(&scst_init_lock)
+ __acquires(&scst_init_lock)
+{
+ struct scst_cmd *cmd;
+ int susp;
+
+restart:
+ /*
+ * There is no need for read barrier here, because we don't care where
+ * this check will be done.
+ */
+ susp = test_bit(SCST_FLAG_SUSPENDED, &scst_flags);
+ if (scst_init_poll_cnt > 0)
+ scst_init_poll_cnt--;
+
+ list_for_each_entry(cmd, &scst_init_cmd_list, cmd_list_entry) {
+ int rc;
+ if (susp && !test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))
+ continue;
+ if (!test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ spin_unlock_irq(&scst_init_lock);
+ rc = __scst_init_cmd(cmd);
+ spin_lock_irq(&scst_init_lock);
+ if (rc > 0) {
+ TRACE_MGMT_DBG("%s",
+ "FLAG SUSPENDED set, restarting");
+ goto restart;
+ }
+ } else {
+ TRACE_MGMT_DBG("Aborting not inited cmd %p (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+ scst_set_cmd_abnormal_done_state(cmd);
+ }
+
+ /*
+ * Deleting cmd from init cmd list after __scst_init_cmd()
+ * is necessary to keep the check in scst_init_cmd() correct
+ * to preserve the commands order.
+ *
+ * We don't care about the race, when init cmd list is empty
+ * and one command detected that it just was not empty, so
+ * it's inserting to it, but another command at the same time
+ * seeing init cmd list empty and goes directly, because it
+ * could affect only commands from the same initiator to the
+ * same tgt_dev, but scst_cmd_init_done*() doesn't guarantee
+ * the order in case of simultaneous such calls anyway.
+ */
+ TRACE_MGMT_DBG("Deleting cmd %p from init cmd list", cmd);
+ smp_wmb(); /* enforce the required order */
+ list_del(&cmd->cmd_list_entry);
+ spin_unlock(&scst_init_lock);
+
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ TRACE_MGMT_DBG("Adding cmd %p to active cmd list", cmd);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+
+ spin_lock(&scst_init_lock);
+ goto restart;
+ }
+
+ /* It isn't really needed, but let's keep it */
+ if (susp != test_bit(SCST_FLAG_SUSPENDED, &scst_flags))
+ goto restart;
+ return;
+}
+
+static inline int test_init_cmd_list(void)
+{
+ int res = (!list_empty(&scst_init_cmd_list) &&
+ !test_bit(SCST_FLAG_SUSPENDED, &scst_flags)) ||
+ unlikely(kthread_should_stop()) ||
+ (scst_init_poll_cnt > 0);
+ return res;
+}
+
+int scst_init_thread(void *arg)
+{
+
+ PRINT_INFO("Init thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ set_user_nice(current, -10);
+
+ spin_lock_irq(&scst_init_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_init_cmd_list()) {
+ add_wait_queue_exclusive(&scst_init_cmd_list_waitQ,
+ &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_init_cmd_list())
+ break;
+ spin_unlock_irq(&scst_init_lock);
+ schedule();
+ spin_lock_irq(&scst_init_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&scst_init_cmd_list_waitQ, &wait);
+ }
+ scst_do_job_init();
+ }
+ spin_unlock_irq(&scst_init_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so scst_init_cmd_list must be empty.
+ */
+ BUG_ON(!list_empty(&scst_init_cmd_list));
+
+ PRINT_INFO("Init thread PID %d finished", current->pid);
+ return 0;
+}
+
+/**
+ * scst_process_active_cmd() - process active command
+ *
+ * Description:
+ * Main SCST commands processing routing. Must be used only by dev handlers.
+ *
+ * Argument atomic is true, if function called in atomic context.
+ *
+ * Must be called with no locks held.
+ */
+void scst_process_active_cmd(struct scst_cmd *cmd, bool atomic)
+{
+ int res;
+
+ /*
+ * Checkpatch will complain on the use of in_atomic() below. You
+ * can safely ignore this warning since in_atomic() is used here only
+ * for debugging purposes.
+ */
+ EXTRACHECKS_BUG_ON(in_irq() || irqs_disabled());
+ EXTRACHECKS_WARN_ON((in_atomic() || in_interrupt() || irqs_disabled()) &&
+ !atomic);
+
+ cmd->atomic = atomic;
+
+ TRACE_DBG("cmd %p, atomic %d", cmd, atomic);
+
+ do {
+ switch (cmd->state) {
+ case SCST_CMD_STATE_PRE_PARSE:
+ res = scst_pre_parse(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ case SCST_CMD_STATE_DEV_PARSE:
+ res = scst_parse_cmd(cmd);
+ break;
+
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ res = scst_prepare_space(cmd);
+ break;
+
+ case SCST_CMD_STATE_PREPROCESSING_DONE:
+ res = scst_preprocessing_done(cmd);
+ break;
+
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ res = scst_rdy_to_xfer(cmd);
+ break;
+
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ res = scst_tgt_pre_exec(cmd);
+ break;
+
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ if (tm_dbg_check_cmd(cmd) != 0) {
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ TRACE_MGMT_DBG("Skipping cmd %p (tag %llu), "
+ "because of TM DBG delay", cmd,
+ (long long unsigned int)cmd->tag);
+ break;
+ }
+ res = scst_send_for_exec(&cmd);
+ /*
+ * !! At this point cmd, sess & tgt_dev can already be
+ * freed !!
+ */
+ break;
+
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ res = scst_local_exec(cmd);
+ /*
+ * !! At this point cmd, sess & tgt_dev can already be
+ * freed !!
+ */
+ break;
+
+ case SCST_CMD_STATE_REAL_EXEC:
+ res = scst_real_exec(cmd);
+ /*
+ * !! At this point cmd, sess & tgt_dev can already be
+ * freed !!
+ */
+ break;
+
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ res = scst_pre_dev_done(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ case SCST_CMD_STATE_MODE_SELECT_CHECKS:
+ res = scst_mode_select_checks(cmd);
+ break;
+
+ case SCST_CMD_STATE_DEV_DONE:
+ res = scst_dev_done(cmd);
+ break;
+
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ res = scst_pre_xmit_response(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ case SCST_CMD_STATE_XMIT_RESP:
+ res = scst_xmit_response(cmd);
+ break;
+
+ case SCST_CMD_STATE_FINISHED:
+ res = scst_finish_cmd(cmd);
+ break;
+
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+ res = scst_finish_internal_cmd(cmd);
+ EXTRACHECKS_BUG_ON(res ==
+ SCST_CMD_STATE_RES_NEED_THREAD);
+ break;
+
+ default:
+ PRINT_CRIT_ERROR("cmd (%p) in state %d, but shouldn't "
+ "be", cmd, cmd->state);
+ BUG();
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ break;
+ }
+ } while (res == SCST_CMD_STATE_RES_CONT_SAME);
+
+ if (res == SCST_CMD_STATE_RES_CONT_NEXT) {
+ /* None */
+ } else if (res == SCST_CMD_STATE_RES_NEED_THREAD) {
+ spin_lock_irq(&cmd->cmd_threads->cmd_list_lock);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ switch (cmd->state) {
+ case SCST_CMD_STATE_DEV_PARSE:
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_DEV_DONE:
+ case SCST_CMD_STATE_XMIT_RESP:
+#endif
+ TRACE_DBG("Adding cmd %p to head of active cmd list",
+ cmd);
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ break;
+ default:
+ PRINT_CRIT_ERROR("cmd %p is in invalid state %d)", cmd,
+ cmd->state);
+ spin_unlock_irq(&cmd->cmd_threads->cmd_list_lock);
+ BUG();
+ spin_lock_irq(&cmd->cmd_threads->cmd_list_lock);
+ break;
+ }
+#endif
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&cmd->cmd_threads->cmd_list_lock);
+ } else
+ BUG();
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_process_active_cmd);
+
+/**
+ * scst_post_parse_process_active_cmd() - process command after parse
+ *
+ * SCST commands processing routine, which should be called by dev handler
+ * after its parse() callback returned SCST_CMD_STATE_STOP. Arguments are
+ * the same as for scst_process_active_cmd().
+ */
+void scst_post_parse_process_active_cmd(struct scst_cmd *cmd, bool atomic)
+{
+ scst_set_parse_time(cmd);
+ scst_process_active_cmd(cmd, atomic);
+}
+EXPORT_SYMBOL_GPL(scst_post_parse_process_active_cmd);
+
+/* Called under cmd_list_lock and IRQs disabled */
+static void scst_do_job_active(struct list_head *cmd_list,
+ spinlock_t *cmd_list_lock, bool atomic)
+ __releases(cmd_list_lock)
+ __acquires(cmd_list_lock)
+{
+
+ while (!list_empty(cmd_list)) {
+ struct scst_cmd *cmd = list_entry(cmd_list->next, typeof(*cmd),
+ cmd_list_entry);
+ TRACE_DBG("Deleting cmd %p from active cmd list", cmd);
+ list_del(&cmd->cmd_list_entry);
+ spin_unlock_irq(cmd_list_lock);
+ scst_process_active_cmd(cmd, atomic);
+ spin_lock_irq(cmd_list_lock);
+ }
+ return;
+}
+
+static inline int test_cmd_threads(struct scst_cmd_threads *p_cmd_threads)
+{
+ int res = !list_empty(&p_cmd_threads->active_cmd_list) ||
+ unlikely(kthread_should_stop()) ||
+ tm_dbg_is_release();
+ return res;
+}
+
+int scst_cmd_thread(void *arg)
+{
+ struct scst_cmd_threads *p_cmd_threads = (struct scst_cmd_threads *)arg;
+ static DEFINE_MUTEX(io_context_mutex);
+
+ PRINT_INFO("Processing thread %s (PID %d) started", current->comm,
+ current->pid);
+
+#if 0
+ set_user_nice(current, 10);
+#endif
+ current->flags |= PF_NOFREEZE;
+
+ mutex_lock(&io_context_mutex);
+
+ WARN_ON(current->io_context);
+
+ if (p_cmd_threads != &scst_main_cmd_threads) {
+ if (p_cmd_threads->io_context == NULL) {
+ p_cmd_threads->io_context = get_io_context(GFP_KERNEL, -1);
+ TRACE_MGMT_DBG("Alloced new IO context %p "
+ "(p_cmd_threads %p)",
+ p_cmd_threads->io_context,
+ p_cmd_threads);
+ /* It's ref counted via threads */
+ put_io_context(p_cmd_threads->io_context);
+ } else {
+ put_io_context(current->io_context);
+ current->io_context = ioc_task_link(p_cmd_threads->io_context);
+ TRACE_MGMT_DBG("Linked IO context %p "
+ "(p_cmd_threads %p)", p_cmd_threads->io_context,
+ p_cmd_threads);
+ }
+ }
+
+ mutex_unlock(&io_context_mutex);
+
+ spin_lock_irq(&p_cmd_threads->cmd_list_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_cmd_threads(p_cmd_threads)) {
+ add_wait_queue_exclusive_head(
+ &p_cmd_threads->cmd_list_waitQ,
+ &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_cmd_threads(p_cmd_threads))
+ break;
+ spin_unlock_irq(&p_cmd_threads->cmd_list_lock);
+ schedule();
+ spin_lock_irq(&p_cmd_threads->cmd_list_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&p_cmd_threads->cmd_list_waitQ, &wait);
+ }
+
+ if (tm_dbg_is_release()) {
+ spin_unlock_irq(&p_cmd_threads->cmd_list_lock);
+ tm_dbg_check_released_cmds();
+ spin_lock_irq(&p_cmd_threads->cmd_list_lock);
+ }
+
+ scst_do_job_active(&p_cmd_threads->active_cmd_list,
+ &p_cmd_threads->cmd_list_lock, false);
+ }
+ spin_unlock_irq(&p_cmd_threads->cmd_list_lock);
+
+ EXTRACHECKS_BUG_ON((p_cmd_threads->nr_threads == 1) &&
+ !list_empty(&p_cmd_threads->active_cmd_list));
+
+ if ((p_cmd_threads->nr_threads == 1) &&
+ (p_cmd_threads != &scst_main_cmd_threads))
+ p_cmd_threads->io_context = NULL;
+
+ PRINT_INFO("Processing thread %s (PID %d) finished", current->comm,
+ current->pid);
+ return 0;
+}
+
+void scst_cmd_tasklet(long p)
+{
+ struct scst_tasklet *t = (struct scst_tasklet *)p;
+
+ spin_lock_irq(&t->tasklet_lock);
+ scst_do_job_active(&t->tasklet_cmd_list, &t->tasklet_lock, true);
+ spin_unlock_irq(&t->tasklet_lock);
+ return;
+}
+
+/*
+ * Returns 0 on success, < 0 if there is no device handler or
+ * > 0 if SCST_FLAG_SUSPENDED set and SCST_FLAG_SUSPENDING - not.
+ * No locks, protection is done by the suspended activity.
+ */
+static int scst_mgmt_translate_lun(struct scst_mgmt_cmd *mcmd)
+{
+ struct scst_tgt_dev *tgt_dev = NULL;
+ struct list_head *sess_tgt_dev_list_head;
+ int res = -1;
+
+ TRACE_DBG("Finding tgt_dev for mgmt cmd %p (lun %lld)", mcmd,
+ (long long unsigned int)mcmd->lun);
+
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ __scst_get(1);
+
+ if (unlikely(test_bit(SCST_FLAG_SUSPENDED, &scst_flags) &&
+ !test_bit(SCST_FLAG_SUSPENDING, &scst_flags))) {
+ TRACE_MGMT_DBG("%s", "FLAG SUSPENDED set, skipping");
+ __scst_put();
+ res = 1;
+ goto out;
+ }
+
+ sess_tgt_dev_list_head =
+ &mcmd->sess->sess_tgt_dev_list_hash[HASH_VAL(mcmd->lun)];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == mcmd->lun) {
+ TRACE_DBG("tgt_dev %p found", tgt_dev);
+ mcmd->mcmd_tgt_dev = tgt_dev;
+ res = 0;
+ break;
+ }
+ }
+ if (mcmd->mcmd_tgt_dev == NULL)
+ __scst_put();
+
+out:
+ return res;
+}
+
+/* No locks */
+void scst_done_cmd_mgmt(struct scst_cmd *cmd)
+{
+ struct scst_mgmt_cmd_stub *mstb;
+ bool wake = 0;
+ unsigned long flags;
+
+ TRACE_MGMT_DBG("cmd %p done (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+
+ list_for_each_entry(mstb, &cmd->mgmt_cmd_list,
+ cmd_mgmt_cmd_list_entry) {
+ struct scst_mgmt_cmd *mcmd;
+
+ if (!mstb->done_counted)
+ continue;
+
+ mcmd = mstb->mcmd;
+ TRACE_MGMT_DBG("mcmd %p, mcmd->cmd_done_wait_count %d",
+ mcmd, mcmd->cmd_done_wait_count);
+
+ mcmd->cmd_done_wait_count--;
+ if (mcmd->cmd_done_wait_count > 0) {
+ TRACE_MGMT_DBG("cmd_done_wait_count(%d) not 0, "
+ "skipping", mcmd->cmd_done_wait_count);
+ continue;
+ }
+
+ if (mcmd->completed) {
+ BUG_ON(mcmd->affected_cmds_done_called);
+ mcmd->completed = 0;
+ mcmd->state = SCST_MCMD_STATE_POST_AFFECTED_CMDS_DONE;
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to active mgmt cmd "
+ "list", mcmd);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ wake = 1;
+ }
+ }
+
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ if (wake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+ return;
+}
+
+/* Called under scst_mcmd_lock and IRQs disabled */
+static int __scst_dec_finish_wait_count(struct scst_mgmt_cmd *mcmd, bool *wake)
+{
+
+ mcmd->cmd_finish_wait_count--;
+ if (mcmd->cmd_finish_wait_count > 0) {
+ TRACE_MGMT_DBG("cmd_finish_wait_count(%d) not 0, "
+ "skipping", mcmd->cmd_finish_wait_count);
+ goto out;
+ }
+
+ if (mcmd->completed) {
+ mcmd->state = SCST_MCMD_STATE_DONE;
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to active mgmt cmd "
+ "list", mcmd);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ *wake = true;
+ }
+
+out:
+ return mcmd->cmd_finish_wait_count;
+}
+
+/**
+ * scst_prepare_async_mcmd() - prepare async management command
+ *
+ * Notifies SCST that management command is going to be async, i.e.
+ * will be completed in another context.
+ *
+ * No SCST locks supposed to be held on entrance.
+ */
+void scst_prepare_async_mcmd(struct scst_mgmt_cmd *mcmd)
+{
+ unsigned long flags;
+
+ TRACE_MGMT_DBG("Preparing mcmd %p for async execution "
+ "(cmd_finish_wait_count %d)", mcmd,
+ mcmd->cmd_finish_wait_count);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+ mcmd->cmd_finish_wait_count++;
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_prepare_async_mcmd);
+
+/**
+ * scst_async_mcmd_completed() - async management command completed
+ *
+ * Notifies SCST that async management command, prepared by
+ * scst_prepare_async_mcmd(), completed.
+ *
+ * No SCST locks supposed to be held on entrance.
+ */
+void scst_async_mcmd_completed(struct scst_mgmt_cmd *mcmd, int status)
+{
+ unsigned long flags;
+ bool wake = false;
+
+ TRACE_MGMT_DBG("Async mcmd %p completed (status %d)", mcmd, status);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+
+ if (status != SCST_MGMT_STATUS_SUCCESS)
+ mcmd->status = status;
+
+ __scst_dec_finish_wait_count(mcmd, &wake);
+
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ if (wake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_async_mcmd_completed);
+
+/* No locks */
+static void scst_finish_cmd_mgmt(struct scst_cmd *cmd)
+{
+ struct scst_mgmt_cmd_stub *mstb, *t;
+ bool wake = false;
+ unsigned long flags;
+
+ TRACE_MGMT_DBG("cmd %p finished (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+
+ list_for_each_entry_safe(mstb, t, &cmd->mgmt_cmd_list,
+ cmd_mgmt_cmd_list_entry) {
+ struct scst_mgmt_cmd *mcmd = mstb->mcmd;
+
+ TRACE_MGMT_DBG("mcmd %p, mcmd->cmd_finish_wait_count %d",
+ mcmd, mcmd->cmd_finish_wait_count);
+
+ list_del(&mstb->cmd_mgmt_cmd_list_entry);
+ mempool_free(mstb, scst_mgmt_stub_mempool);
+
+ if (cmd->completed)
+ mcmd->completed_cmd_count++;
+
+ if (__scst_dec_finish_wait_count(mcmd, &wake) > 0) {
+ TRACE_MGMT_DBG("cmd_finish_wait_count(%d) not 0, "
+ "skipping", mcmd->cmd_finish_wait_count);
+ continue;
+ }
+ }
+
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ if (wake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+ return;
+}
+
+static int scst_call_dev_task_mgmt_fn(struct scst_mgmt_cmd *mcmd,
+ struct scst_tgt_dev *tgt_dev, int set_status)
+{
+ int res = SCST_DEV_TM_NOT_COMPLETED;
+ struct scst_dev_type *h = tgt_dev->dev->handler;
+
+ if (h->task_mgmt_fn) {
+ TRACE_MGMT_DBG("Calling dev handler %s task_mgmt_fn(fn=%d)",
+ h->name, mcmd->fn);
+ EXTRACHECKS_BUG_ON(in_irq() || irqs_disabled());
+ res = h->task_mgmt_fn(mcmd, tgt_dev);
+ TRACE_MGMT_DBG("Dev handler %s task_mgmt_fn() returned %d",
+ h->name, res);
+ if (set_status && (res != SCST_DEV_TM_NOT_COMPLETED))
+ mcmd->status = res;
+ }
+ return res;
+}
+
+static inline int scst_is_strict_mgmt_fn(int mgmt_fn)
+{
+ switch (mgmt_fn) {
+#ifdef CONFIG_SCST_ABORT_CONSIDER_FINISHED_TASKS_AS_NOT_EXISTING
+ case SCST_ABORT_TASK:
+#endif
+#if 0
+ case SCST_ABORT_TASK_SET:
+ case SCST_CLEAR_TASK_SET:
+#endif
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+/* Might be called under sess_list_lock and IRQ off + BHs also off */
+void scst_abort_cmd(struct scst_cmd *cmd, struct scst_mgmt_cmd *mcmd,
+ int other_ini, int call_dev_task_mgmt_fn)
+{
+ unsigned long flags;
+ static DEFINE_SPINLOCK(other_ini_lock);
+
+ TRACE(TRACE_MGMT, "Aborting cmd %p (tag %llu, op %x)",
+ cmd, (long long unsigned int)cmd->tag, cmd->cdb[0]);
+
+ /* To protect from concurrent aborts */
+ spin_lock_irqsave(&other_ini_lock, flags);
+
+ if (other_ini) {
+ struct scst_device *dev = NULL;
+
+ /* Might be necessary if command aborted several times */
+ if (!test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))
+ set_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+
+ /* Necessary for scst_xmit_process_aborted_cmd */
+ if (cmd->dev != NULL)
+ dev = cmd->dev;
+ else if ((mcmd != NULL) && (mcmd->mcmd_tgt_dev != NULL))
+ dev = mcmd->mcmd_tgt_dev->dev;
+
+ if (dev != NULL) {
+ if (dev->tas)
+ set_bit(SCST_CMD_DEVICE_TAS, &cmd->cmd_flags);
+ } else
+ PRINT_WARNING("Abort cmd %p from other initiator, but "
+ "neither cmd, nor mcmd %p have tgt_dev set, so "
+ "TAS information can be lost", cmd, mcmd);
+ } else {
+ /* Might be necessary if command aborted several times */
+ clear_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+ }
+
+ set_bit(SCST_CMD_ABORTED, &cmd->cmd_flags);
+
+ spin_unlock_irqrestore(&other_ini_lock, flags);
+
+ /*
+ * To sync with cmd->finished/done set in
+ * scst_finish_cmd()/scst_pre_xmit_response()
+ */
+ smp_mb__after_set_bit();
+
+ if (cmd->tgt_dev == NULL) {
+ spin_lock_irqsave(&scst_init_lock, flags);
+ scst_init_poll_cnt++;
+ spin_unlock_irqrestore(&scst_init_lock, flags);
+ wake_up(&scst_init_cmd_list_waitQ);
+ }
+
+ if (call_dev_task_mgmt_fn && (cmd->tgt_dev != NULL)) {
+ EXTRACHECKS_BUG_ON(irqs_disabled());
+ scst_call_dev_task_mgmt_fn(mcmd, cmd->tgt_dev, 1);
+ }
+
+ spin_lock_irqsave(&scst_mcmd_lock, flags);
+ if ((mcmd != NULL) && !cmd->finished) {
+ struct scst_mgmt_cmd_stub *mstb;
+
+ mstb = mempool_alloc(scst_mgmt_stub_mempool, GFP_ATOMIC);
+ if (mstb == NULL) {
+ PRINT_CRIT_ERROR("Allocation of management command "
+ "stub failed (mcmd %p, cmd %p)", mcmd, cmd);
+ goto unlock;
+ }
+ memset(mstb, 0, sizeof(*mstb));
+
+ mstb->mcmd = mcmd;
+
+ /*
+ * cmd can't die here or sess_list_lock already taken and
+ * cmd is in the sess list
+ */
+ list_add_tail(&mstb->cmd_mgmt_cmd_list_entry,
+ &cmd->mgmt_cmd_list);
+
+ /*
+ * Delay the response until the command's finish in order to
+ * guarantee that "no further responses from the task are sent
+ * to the SCSI initiator port" after response from the TM
+ * function is sent (SAM). Plus, we must wait here to be sure
+ * that we won't receive double commands with the same tag.
+ * Moreover, if we don't wait here, we might have a possibility
+ * for data corruption, when aborted and reported as completed
+ * command actually gets executed *after* new commands sent
+ * after this TM command completed.
+ */
+ TRACE_MGMT_DBG("cmd %p (tag %llu, sn %u) being "
+ "executed/xmitted (state %d, op %x, proc time %ld "
+ "sec., timeout %d sec.), deferring ABORT...", cmd,
+ (long long unsigned int)cmd->tag, cmd->sn, cmd->state,
+ cmd->cdb[0], (long)(jiffies - cmd->start_time) / HZ,
+ cmd->timeout / HZ);
+
+ mcmd->cmd_finish_wait_count++;
+
+ if (cmd->sent_for_exec && !cmd->done) {
+ TRACE_MGMT_DBG("cmd %p (tag %llu) is being executed "
+ "and not done yet", cmd,
+ (long long unsigned int)cmd->tag);
+ mstb->done_counted = 1;
+ mcmd->cmd_done_wait_count++;
+ }
+ }
+unlock:
+ spin_unlock_irqrestore(&scst_mcmd_lock, flags);
+
+ tm_dbg_release_cmd(cmd);
+ return;
+}
+
+/* No locks */
+static int scst_set_mcmd_next_state(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+
+ spin_lock_irq(&scst_mcmd_lock);
+
+ if (mcmd->cmd_finish_wait_count == 0) {
+ if (!mcmd->affected_cmds_done_called)
+ mcmd->state = SCST_MCMD_STATE_POST_AFFECTED_CMDS_DONE;
+ else
+ mcmd->state = SCST_MCMD_STATE_DONE;
+ res = 0;
+ } else if ((mcmd->cmd_done_wait_count == 0) &&
+ (!mcmd->affected_cmds_done_called)) {
+ mcmd->state = SCST_MCMD_STATE_POST_AFFECTED_CMDS_DONE;
+ res = 0;
+ goto out_unlock;
+ } else {
+ TRACE_MGMT_DBG("cmd_finish_wait_count(%d) not 0, preparing to "
+ "wait", mcmd->cmd_finish_wait_count);
+ mcmd->state = SCST_MCMD_STATE_EXECUTING;
+ res = -1;
+ }
+
+ mcmd->completed = 1;
+
+out_unlock:
+ spin_unlock_irq(&scst_mcmd_lock);
+ return res;
+}
+
+static bool __scst_check_unblock_aborted_cmd(struct scst_cmd *cmd,
+ struct list_head *list_entry)
+{
+ bool res;
+ if (test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ list_del(list_entry);
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ res = 1;
+ } else
+ res = 0;
+ return res;
+}
+
+static void scst_unblock_aborted_cmds(int scst_mutex_held)
+{
+ struct scst_device *dev;
+
+ if (!scst_mutex_held)
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(dev, &scst_dev_list, dev_list_entry) {
+ struct scst_cmd *cmd, *tcmd;
+ struct scst_tgt_dev *tgt_dev;
+ spin_lock_bh(&dev->dev_lock);
+ local_irq_disable();
+ list_for_each_entry_safe(cmd, tcmd, &dev->blocked_cmd_list,
+ blocked_cmd_list_entry) {
+ if (__scst_check_unblock_aborted_cmd(cmd,
+ &cmd->blocked_cmd_list_entry)) {
+ TRACE_MGMT_DBG("Unblock aborted blocked cmd %p",
+ cmd);
+ }
+ }
+ local_irq_enable();
+ spin_unlock_bh(&dev->dev_lock);
+
+ local_irq_disable();
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ spin_lock(&tgt_dev->sn_lock);
+ list_for_each_entry_safe(cmd, tcmd,
+ &tgt_dev->deferred_cmd_list,
+ sn_cmd_list_entry) {
+ if (__scst_check_unblock_aborted_cmd(cmd,
+ &cmd->sn_cmd_list_entry)) {
+ TRACE_MGMT_DBG("Unblocked aborted SN "
+ "cmd %p (sn %u)",
+ cmd, cmd->sn);
+ tgt_dev->def_cmd_count--;
+ }
+ }
+ spin_unlock(&tgt_dev->sn_lock);
+ }
+ local_irq_enable();
+ }
+
+ if (!scst_mutex_held)
+ mutex_unlock(&scst_mutex);
+ return;
+}
+
+static void __scst_abort_task_set(struct scst_mgmt_cmd *mcmd,
+ struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_cmd *cmd;
+ struct scst_session *sess = tgt_dev->sess;
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if ((cmd->tgt_dev == tgt_dev) ||
+ ((cmd->tgt_dev == NULL) &&
+ (cmd->lun == tgt_dev->lun))) {
+ if (mcmd->cmd_sn_set) {
+ BUG_ON(!cmd->tgt_sn_set);
+ if (scst_sn_before(mcmd->cmd_sn, cmd->tgt_sn) ||
+ (mcmd->cmd_sn == cmd->tgt_sn))
+ continue;
+ }
+ scst_abort_cmd(cmd, mcmd, 0, 0);
+ }
+ }
+ spin_unlock_irq(&sess->sess_list_lock);
+ return;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_abort_task_set(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+ struct scst_tgt_dev *tgt_dev = mcmd->mcmd_tgt_dev;
+
+ TRACE(TRACE_MGMT, "Aborting task set (lun=%lld, mcmd=%p)",
+ (long long unsigned int)tgt_dev->lun, mcmd);
+
+ __scst_abort_task_set(mcmd, tgt_dev);
+
+ tm_dbg_task_mgmt(mcmd->mcmd_tgt_dev->dev, "ABORT TASK SET", 0);
+
+ scst_unblock_aborted_cmds(0);
+
+ scst_call_dev_task_mgmt_fn(mcmd, tgt_dev, 0);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+static int scst_is_cmd_belongs_to_dev(struct scst_cmd *cmd,
+ struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev = NULL;
+ struct list_head *sess_tgt_dev_list_head;
+ int res = 0;
+
+ TRACE_DBG("Finding match for dev %p and cmd %p (lun %lld)", dev, cmd,
+ (long long unsigned int)cmd->lun);
+
+ sess_tgt_dev_list_head =
+ &cmd->sess->sess_tgt_dev_list_hash[HASH_VAL(cmd->lun)];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == cmd->lun) {
+ TRACE_DBG("dev %p found", tgt_dev->dev);
+ res = (tgt_dev->dev == dev);
+ goto out;
+ }
+ }
+
+out:
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_clear_task_set(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+ struct scst_device *dev = mcmd->mcmd_tgt_dev->dev;
+ struct scst_tgt_dev *tgt_dev;
+ LIST_HEAD(UA_tgt_devs);
+
+ TRACE(TRACE_MGMT, "Clearing task set (lun=%lld, mcmd=%p)",
+ (long long unsigned int)mcmd->lun, mcmd);
+
+#if 0 /* we are SAM-3 */
+ /*
+ * When a logical unit is aborting one or more tasks from a SCSI
+ * initiator port with the TASK ABORTED status it should complete all
+ * of those tasks before entering additional tasks from that SCSI
+ * initiator port into the task set - SAM2
+ */
+ mcmd->needs_unblocking = 1;
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ spin_unlock_bh(&dev->dev_lock);
+#endif
+
+ __scst_abort_task_set(mcmd, mcmd->mcmd_tgt_dev);
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ struct scst_session *sess = tgt_dev->sess;
+ struct scst_cmd *cmd;
+ int aborted = 0;
+
+ if (tgt_dev == mcmd->mcmd_tgt_dev)
+ continue;
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if ((cmd->dev == dev) ||
+ ((cmd->dev == NULL) &&
+ scst_is_cmd_belongs_to_dev(cmd, dev))) {
+ scst_abort_cmd(cmd, mcmd, 1, 0);
+ aborted = 1;
+ }
+ }
+ spin_unlock_irq(&sess->sess_list_lock);
+
+ if (aborted)
+ list_add_tail(&tgt_dev->extra_tgt_dev_list_entry,
+ &UA_tgt_devs);
+ }
+
+ tm_dbg_task_mgmt(mcmd->mcmd_tgt_dev->dev, "CLEAR TASK SET", 0);
+
+ scst_unblock_aborted_cmds(1);
+
+ mutex_unlock(&scst_mutex);
+
+ if (!dev->tas) {
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl;
+
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_cleared_by_another_ini_UA));
+
+ list_for_each_entry(tgt_dev, &UA_tgt_devs,
+ extra_tgt_dev_list_entry) {
+ scst_check_set_UA(tgt_dev, sense_buffer, sl, 0);
+ }
+ }
+
+ scst_call_dev_task_mgmt_fn(mcmd, mcmd->mcmd_tgt_dev, 0);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued,
+ * >0, if it should be requeued, <0 otherwise */
+static int scst_mgmt_cmd_init(struct scst_mgmt_cmd *mcmd)
+{
+ int res = 0, rc;
+
+ switch (mcmd->fn) {
+ case SCST_ABORT_TASK:
+ {
+ struct scst_session *sess = mcmd->sess;
+ struct scst_cmd *cmd;
+
+ spin_lock_irq(&sess->sess_list_lock);
+ cmd = __scst_find_cmd_by_tag(sess, mcmd->tag, true);
+ if (cmd == NULL) {
+ TRACE_MGMT_DBG("ABORT TASK: command "
+ "for tag %llu not found",
+ (long long unsigned int)mcmd->tag);
+ mcmd->status = SCST_MGMT_STATUS_TASK_NOT_EXIST;
+ mcmd->state = SCST_MCMD_STATE_DONE;
+ spin_unlock_irq(&sess->sess_list_lock);
+ goto out;
+ }
+ __scst_cmd_get(cmd);
+ spin_unlock_irq(&sess->sess_list_lock);
+ TRACE_MGMT_DBG("Cmd %p for tag %llu (sn %d, set %d, "
+ "queue_type %x) found, aborting it",
+ cmd, (long long unsigned int)mcmd->tag,
+ cmd->sn, cmd->sn_set, cmd->queue_type);
+ mcmd->cmd_to_abort = cmd;
+ if (mcmd->lun_set && (mcmd->lun != cmd->lun)) {
+ PRINT_ERROR("ABORT TASK: LUN mismatch: mcmd LUN %llx, "
+ "cmd LUN %llx, cmd tag %llu",
+ (long long unsigned int)mcmd->lun,
+ (long long unsigned int)cmd->lun,
+ (long long unsigned int)mcmd->tag);
+ mcmd->status = SCST_MGMT_STATUS_REJECTED;
+ } else if (mcmd->cmd_sn_set &&
+ (scst_sn_before(mcmd->cmd_sn, cmd->tgt_sn) ||
+ (mcmd->cmd_sn == cmd->tgt_sn))) {
+ PRINT_ERROR("ABORT TASK: SN mismatch: mcmd SN %x, "
+ "cmd SN %x, cmd tag %llu", mcmd->cmd_sn,
+ cmd->tgt_sn, (long long unsigned int)mcmd->tag);
+ mcmd->status = SCST_MGMT_STATUS_REJECTED;
+ } else {
+ scst_abort_cmd(cmd, mcmd, 0, 1);
+ scst_unblock_aborted_cmds(0);
+ }
+ res = scst_set_mcmd_next_state(mcmd);
+ mcmd->cmd_to_abort = NULL; /* just in case */
+ __scst_cmd_put(cmd);
+ break;
+ }
+
+ case SCST_TARGET_RESET:
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_ABORT_ALL_TASKS_SESS:
+ case SCST_NEXUS_LOSS:
+ case SCST_ABORT_ALL_TASKS:
+ case SCST_UNREG_SESS_TM:
+ mcmd->state = SCST_MCMD_STATE_READY;
+ break;
+
+ case SCST_ABORT_TASK_SET:
+ case SCST_CLEAR_ACA:
+ case SCST_CLEAR_TASK_SET:
+ case SCST_LUN_RESET:
+ rc = scst_mgmt_translate_lun(mcmd);
+ if (rc == 0)
+ mcmd->state = SCST_MCMD_STATE_READY;
+ else if (rc < 0) {
+ PRINT_ERROR("Corresponding device for LUN %lld not "
+ "found", (long long unsigned int)mcmd->lun);
+ mcmd->status = SCST_MGMT_STATUS_LUN_NOT_EXIST;
+ mcmd->state = SCST_MCMD_STATE_DONE;
+ } else
+ res = rc;
+ break;
+
+ default:
+ BUG();
+ }
+
+out:
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_target_reset(struct scst_mgmt_cmd *mcmd)
+{
+ int res, rc;
+ struct scst_device *dev;
+ struct scst_acg *acg = mcmd->sess->acg;
+ struct scst_acg_dev *acg_dev;
+ int cont, c;
+ LIST_HEAD(host_devs);
+
+ TRACE(TRACE_MGMT, "Target reset (mcmd %p, cmd count %d)",
+ mcmd, atomic_read(&mcmd->sess->sess_cmd_count));
+
+ mcmd->needs_unblocking = 1;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ struct scst_device *d;
+ struct scst_tgt_dev *tgt_dev;
+ int found = 0;
+
+ dev = acg_dev->dev;
+
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ scst_process_reset(dev, mcmd->sess, NULL, mcmd, true);
+ spin_unlock_bh(&dev->dev_lock);
+
+ cont = 0;
+ c = 0;
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ cont = 1;
+ if (mcmd->sess == tgt_dev->sess) {
+ rc = scst_call_dev_task_mgmt_fn(mcmd,
+ tgt_dev, 0);
+ if (rc == SCST_DEV_TM_NOT_COMPLETED)
+ c = 1;
+ else if ((rc < 0) &&
+ (mcmd->status == SCST_MGMT_STATUS_SUCCESS))
+ mcmd->status = rc;
+ break;
+ }
+ }
+ if (cont && !c)
+ continue;
+
+ if (dev->scsi_dev == NULL)
+ continue;
+
+ list_for_each_entry(d, &host_devs, tm_dev_list_entry) {
+ if (dev->scsi_dev->host->host_no ==
+ d->scsi_dev->host->host_no) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found)
+ list_add_tail(&dev->tm_dev_list_entry, &host_devs);
+
+ tm_dbg_task_mgmt(dev, "TARGET RESET", 0);
+ }
+
+ scst_unblock_aborted_cmds(1);
+
+ /*
+ * We suppose here that for all commands that already on devices
+ * on/after scsi_reset_provider() completion callbacks will be called.
+ */
+
+ list_for_each_entry(dev, &host_devs, tm_dev_list_entry) {
+ /* dev->scsi_dev must be non-NULL here */
+ TRACE(TRACE_MGMT, "Resetting host %d bus ",
+ dev->scsi_dev->host->host_no);
+ rc = scsi_reset_provider(dev->scsi_dev, SCSI_TRY_RESET_TARGET);
+ TRACE(TRACE_MGMT, "Result of host %d target reset: %s",
+ dev->scsi_dev->host->host_no,
+ (rc == SUCCESS) ? "SUCCESS" : "FAILED");
+#if 0
+ if ((rc != SUCCESS) &&
+ (mcmd->status == SCST_MGMT_STATUS_SUCCESS)) {
+ /*
+ * SCSI_TRY_RESET_BUS is also done by
+ * scsi_reset_provider()
+ */
+ mcmd->status = SCST_MGMT_STATUS_FAILED;
+ }
+#else
+ /*
+ * scsi_reset_provider() returns very weird status, so let's
+ * always succeed
+ */
+#endif
+ }
+
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ dev = acg_dev->dev;
+ if (dev->scsi_dev != NULL)
+ dev->scsi_dev->was_reset = 0;
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_lun_reset(struct scst_mgmt_cmd *mcmd)
+{
+ int res, rc;
+ struct scst_tgt_dev *tgt_dev = mcmd->mcmd_tgt_dev;
+ struct scst_device *dev = tgt_dev->dev;
+
+ TRACE(TRACE_MGMT, "Resetting LUN %lld (mcmd %p)",
+ (long long unsigned int)tgt_dev->lun, mcmd);
+
+ mcmd->needs_unblocking = 1;
+
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ scst_process_reset(dev, mcmd->sess, NULL, mcmd, true);
+ spin_unlock_bh(&dev->dev_lock);
+
+ rc = scst_call_dev_task_mgmt_fn(mcmd, tgt_dev, 1);
+ if (rc != SCST_DEV_TM_NOT_COMPLETED)
+ goto out_tm_dbg;
+
+ if (dev->scsi_dev != NULL) {
+ TRACE(TRACE_MGMT, "Resetting host %d bus ",
+ dev->scsi_dev->host->host_no);
+ rc = scsi_reset_provider(dev->scsi_dev, SCSI_TRY_RESET_DEVICE);
+#if 0
+ if (rc != SUCCESS && mcmd->status == SCST_MGMT_STATUS_SUCCESS)
+ mcmd->status = SCST_MGMT_STATUS_FAILED;
+#else
+ /*
+ * scsi_reset_provider() returns very weird status, so let's
+ * always succeed
+ */
+#endif
+ dev->scsi_dev->was_reset = 0;
+ }
+
+ scst_unblock_aborted_cmds(0);
+
+out_tm_dbg:
+ tm_dbg_task_mgmt(mcmd->mcmd_tgt_dev->dev, "LUN RESET", 0);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* scst_mutex supposed to be held */
+static void scst_do_nexus_loss_sess(struct scst_mgmt_cmd *mcmd)
+{
+ int i;
+ struct scst_session *sess = mcmd->sess;
+ struct scst_tgt_dev *tgt_dev;
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ scst_nexus_loss(tgt_dev,
+ (mcmd->fn != SCST_UNREG_SESS_TM));
+ }
+ }
+ return;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_abort_all_nexus_loss_sess(struct scst_mgmt_cmd *mcmd,
+ int nexus_loss)
+{
+ int res;
+ int i;
+ struct scst_session *sess = mcmd->sess;
+ struct scst_tgt_dev *tgt_dev;
+
+ if (nexus_loss) {
+ TRACE_MGMT_DBG("Nexus loss for sess %p (mcmd %p)",
+ sess, mcmd);
+ } else {
+ TRACE_MGMT_DBG("Aborting all from sess %p (mcmd %p)",
+ sess, mcmd);
+ }
+
+ mutex_lock(&scst_mutex);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ int rc;
+
+ __scst_abort_task_set(mcmd, tgt_dev);
+
+ rc = scst_call_dev_task_mgmt_fn(mcmd, tgt_dev, 0);
+ if (rc < 0 && mcmd->status == SCST_MGMT_STATUS_SUCCESS)
+ mcmd->status = rc;
+
+ tm_dbg_task_mgmt(tgt_dev->dev, "NEXUS LOSS SESS or "
+ "ABORT ALL SESS or UNREG SESS",
+ (mcmd->fn == SCST_UNREG_SESS_TM));
+ }
+ }
+
+ scst_unblock_aborted_cmds(1);
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* scst_mutex supposed to be held */
+static void scst_do_nexus_loss_tgt(struct scst_mgmt_cmd *mcmd)
+{
+ int i;
+ struct scst_tgt *tgt = mcmd->sess->tgt;
+ struct scst_session *sess;
+
+ list_for_each_entry(sess, &tgt->sess_list, sess_list_entry) {
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ scst_nexus_loss(tgt_dev, true);
+ }
+ }
+ }
+ return;
+}
+
+static int scst_abort_all_nexus_loss_tgt(struct scst_mgmt_cmd *mcmd,
+ int nexus_loss)
+{
+ int res;
+ int i;
+ struct scst_tgt *tgt = mcmd->sess->tgt;
+ struct scst_session *sess;
+
+ if (nexus_loss) {
+ TRACE_MGMT_DBG("I_T Nexus loss (tgt %p, mcmd %p)",
+ tgt, mcmd);
+ } else {
+ TRACE_MGMT_DBG("Aborting all from tgt %p (mcmd %p)",
+ tgt, mcmd);
+ }
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(sess, &tgt->sess_list, sess_list_entry) {
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ int rc;
+
+ __scst_abort_task_set(mcmd, tgt_dev);
+
+ if (nexus_loss)
+ scst_nexus_loss(tgt_dev, true);
+
+ if (mcmd->sess == tgt_dev->sess) {
+ rc = scst_call_dev_task_mgmt_fn(
+ mcmd, tgt_dev, 0);
+ if ((rc < 0) &&
+ (mcmd->status == SCST_MGMT_STATUS_SUCCESS))
+ mcmd->status = rc;
+ }
+
+ tm_dbg_task_mgmt(tgt_dev->dev, "NEXUS LOSS or "
+ "ABORT ALL", 0);
+ }
+ }
+ }
+
+ scst_unblock_aborted_cmds(1);
+
+ mutex_unlock(&scst_mutex);
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+/* Returns 0 if the command processing should be continued, <0 otherwise */
+static int scst_mgmt_cmd_exec(struct scst_mgmt_cmd *mcmd)
+{
+ int res = 0;
+
+ mcmd->status = SCST_MGMT_STATUS_SUCCESS;
+
+ switch (mcmd->fn) {
+ case SCST_ABORT_TASK_SET:
+ res = scst_abort_task_set(mcmd);
+ break;
+
+ case SCST_CLEAR_TASK_SET:
+ if (mcmd->mcmd_tgt_dev->dev->tst ==
+ SCST_CONTR_MODE_SEP_TASK_SETS)
+ res = scst_abort_task_set(mcmd);
+ else
+ res = scst_clear_task_set(mcmd);
+ break;
+
+ case SCST_LUN_RESET:
+ res = scst_lun_reset(mcmd);
+ break;
+
+ case SCST_TARGET_RESET:
+ res = scst_target_reset(mcmd);
+ break;
+
+ case SCST_ABORT_ALL_TASKS_SESS:
+ res = scst_abort_all_nexus_loss_sess(mcmd, 0);
+ break;
+
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_UNREG_SESS_TM:
+ res = scst_abort_all_nexus_loss_sess(mcmd, 1);
+ break;
+
+ case SCST_ABORT_ALL_TASKS:
+ res = scst_abort_all_nexus_loss_tgt(mcmd, 0);
+ break;
+
+ case SCST_NEXUS_LOSS:
+ res = scst_abort_all_nexus_loss_tgt(mcmd, 1);
+ break;
+
+ case SCST_CLEAR_ACA:
+ if (scst_call_dev_task_mgmt_fn(mcmd, mcmd->mcmd_tgt_dev, 1) ==
+ SCST_DEV_TM_NOT_COMPLETED) {
+ mcmd->status = SCST_MGMT_STATUS_FN_NOT_SUPPORTED;
+ /* Nothing to do (yet) */
+ }
+ goto out_done;
+
+ default:
+ PRINT_ERROR("Unknown task management function %d", mcmd->fn);
+ mcmd->status = SCST_MGMT_STATUS_REJECTED;
+ goto out_done;
+ }
+
+out:
+ return res;
+
+out_done:
+ mcmd->state = SCST_MCMD_STATE_DONE;
+ goto out;
+}
+
+static void scst_call_task_mgmt_affected_cmds_done(struct scst_mgmt_cmd *mcmd)
+{
+ struct scst_session *sess = mcmd->sess;
+
+ if ((sess->tgt->tgtt->task_mgmt_affected_cmds_done != NULL) &&
+ (mcmd->fn != SCST_UNREG_SESS_TM)) {
+ TRACE_DBG("Calling target %s task_mgmt_affected_cmds_done(%p)",
+ sess->tgt->tgtt->name, sess);
+ sess->tgt->tgtt->task_mgmt_affected_cmds_done(mcmd);
+ TRACE_MGMT_DBG("Target's %s task_mgmt_affected_cmds_done() "
+ "returned", sess->tgt->tgtt->name);
+ }
+ return;
+}
+
+static int scst_mgmt_affected_cmds_done(struct scst_mgmt_cmd *mcmd)
+{
+ int res;
+
+ mutex_lock(&scst_mutex);
+
+ switch (mcmd->fn) {
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_UNREG_SESS_TM:
+ scst_do_nexus_loss_sess(mcmd);
+ break;
+
+ case SCST_NEXUS_LOSS:
+ scst_do_nexus_loss_tgt(mcmd);
+ break;
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ scst_call_task_mgmt_affected_cmds_done(mcmd);
+
+ mcmd->affected_cmds_done_called = 1;
+
+ res = scst_set_mcmd_next_state(mcmd);
+ return res;
+}
+
+static void scst_mgmt_cmd_send_done(struct scst_mgmt_cmd *mcmd)
+{
+ struct scst_device *dev;
+ struct scst_session *sess = mcmd->sess;
+
+ mcmd->state = SCST_MCMD_STATE_FINISHED;
+ if (scst_is_strict_mgmt_fn(mcmd->fn) && (mcmd->completed_cmd_count > 0))
+ mcmd->status = SCST_MGMT_STATUS_TASK_NOT_EXIST;
+
+ TRACE(TRACE_MINOR_AND_MGMT_DBG, "TM command fn %d finished, "
+ "status %x", mcmd->fn, mcmd->status);
+
+ if (!mcmd->affected_cmds_done_called) {
+ /* It might happen in case of errors */
+ scst_call_task_mgmt_affected_cmds_done(mcmd);
+ }
+
+ if ((sess->tgt->tgtt->task_mgmt_fn_done != NULL) &&
+ (mcmd->fn != SCST_UNREG_SESS_TM)) {
+ TRACE_DBG("Calling target %s task_mgmt_fn_done(%p)",
+ sess->tgt->tgtt->name, sess);
+ sess->tgt->tgtt->task_mgmt_fn_done(mcmd);
+ TRACE_MGMT_DBG("Target's %s task_mgmt_fn_done() "
+ "returned", sess->tgt->tgtt->name);
+ }
+
+ if (mcmd->needs_unblocking) {
+ switch (mcmd->fn) {
+ case SCST_LUN_RESET:
+ case SCST_CLEAR_TASK_SET:
+ scst_unblock_dev(mcmd->mcmd_tgt_dev->dev);
+ break;
+
+ case SCST_TARGET_RESET:
+ {
+ struct scst_acg *acg = mcmd->sess->acg;
+ struct scst_acg_dev *acg_dev;
+
+ mutex_lock(&scst_mutex);
+ list_for_each_entry(acg_dev, &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ dev = acg_dev->dev;
+ scst_unblock_dev(dev);
+ }
+ mutex_unlock(&scst_mutex);
+ break;
+ }
+
+ default:
+ BUG();
+ break;
+ }
+ }
+
+ mcmd->tgt_priv = NULL;
+ return;
+}
+
+/* Returns >0, if cmd should be requeued */
+static int scst_process_mgmt_cmd(struct scst_mgmt_cmd *mcmd)
+{
+ int res = 0;
+
+ TRACE_DBG("mcmd %p, state %d", mcmd, mcmd->state);
+
+ while (1) {
+ switch (mcmd->state) {
+ case SCST_MCMD_STATE_INIT:
+ res = scst_mgmt_cmd_init(mcmd);
+ if (res)
+ goto out;
+ break;
+
+ case SCST_MCMD_STATE_READY:
+ if (scst_mgmt_cmd_exec(mcmd))
+ goto out;
+ break;
+
+ case SCST_MCMD_STATE_POST_AFFECTED_CMDS_DONE:
+ if (scst_mgmt_affected_cmds_done(mcmd))
+ goto out;
+ break;
+
+ case SCST_MCMD_STATE_DONE:
+ scst_mgmt_cmd_send_done(mcmd);
+ break;
+
+ default:
+ PRINT_ERROR("Unknown state %d of management command",
+ mcmd->state);
+ res = -1;
+ /* go through */
+ case SCST_MCMD_STATE_FINISHED:
+ scst_free_mgmt_cmd(mcmd);
+ goto out;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ case SCST_MCMD_STATE_EXECUTING:
+ BUG();
+#endif
+ }
+ }
+
+out:
+ return res;
+}
+
+static inline int test_mgmt_cmd_list(void)
+{
+ int res = !list_empty(&scst_active_mgmt_cmd_list) ||
+ unlikely(kthread_should_stop());
+ return res;
+}
+
+int scst_tm_thread(void *arg)
+{
+
+ PRINT_INFO("Task management thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ set_user_nice(current, -10);
+
+ spin_lock_irq(&scst_mcmd_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_mgmt_cmd_list()) {
+ add_wait_queue_exclusive(&scst_mgmt_cmd_list_waitQ,
+ &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_mgmt_cmd_list())
+ break;
+ spin_unlock_irq(&scst_mcmd_lock);
+ schedule();
+ spin_lock_irq(&scst_mcmd_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&scst_mgmt_cmd_list_waitQ, &wait);
+ }
+
+ while (!list_empty(&scst_active_mgmt_cmd_list)) {
+ int rc;
+ struct scst_mgmt_cmd *mcmd;
+ mcmd = list_entry(scst_active_mgmt_cmd_list.next,
+ typeof(*mcmd), mgmt_cmd_list_entry);
+ TRACE_MGMT_DBG("Deleting mgmt cmd %p from active cmd "
+ "list", mcmd);
+ list_del(&mcmd->mgmt_cmd_list_entry);
+ spin_unlock_irq(&scst_mcmd_lock);
+ rc = scst_process_mgmt_cmd(mcmd);
+ spin_lock_irq(&scst_mcmd_lock);
+ if (rc > 0) {
+ if (test_bit(SCST_FLAG_SUSPENDED, &scst_flags) &&
+ !test_bit(SCST_FLAG_SUSPENDING,
+ &scst_flags)) {
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to "
+ "head of delayed mgmt cmd list",
+ mcmd);
+ list_add(&mcmd->mgmt_cmd_list_entry,
+ &scst_delayed_mgmt_cmd_list);
+ } else {
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to "
+ "head of active mgmt cmd list",
+ mcmd);
+ list_add(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ }
+ }
+ }
+ }
+ spin_unlock_irq(&scst_mcmd_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so scst_active_mgmt_cmd_list must be empty.
+ */
+ BUG_ON(!list_empty(&scst_active_mgmt_cmd_list));
+
+ PRINT_INFO("Task management thread PID %d finished", current->pid);
+ return 0;
+}
+
+static struct scst_mgmt_cmd *scst_pre_rx_mgmt_cmd(struct scst_session
+ *sess, int fn, int atomic, void *tgt_priv)
+{
+ struct scst_mgmt_cmd *mcmd = NULL;
+
+ if (unlikely(sess->tgt->tgtt->task_mgmt_fn_done == NULL)) {
+ PRINT_ERROR("New mgmt cmd, but task_mgmt_fn_done() is NULL "
+ "(target %s)", sess->tgt->tgtt->name);
+ goto out;
+ }
+
+ mcmd = scst_alloc_mgmt_cmd(atomic ? GFP_ATOMIC : GFP_KERNEL);
+ if (mcmd == NULL) {
+ PRINT_CRIT_ERROR("Lost TM fn %d, initiator %s", fn,
+ sess->initiator_name);
+ goto out;
+ }
+
+ mcmd->sess = sess;
+ mcmd->fn = fn;
+ mcmd->state = SCST_MCMD_STATE_INIT;
+ mcmd->tgt_priv = tgt_priv;
+
+out:
+ return mcmd;
+}
+
+static int scst_post_rx_mgmt_cmd(struct scst_session *sess,
+ struct scst_mgmt_cmd *mcmd)
+{
+ unsigned long flags;
+ int res = 0;
+
+ scst_sess_get(sess);
+
+ if (unlikely(sess->shut_phase != SCST_SESS_SPH_READY)) {
+ PRINT_CRIT_ERROR("New mgmt cmd while shutting down the "
+ "session %p shut_phase %ld", sess, sess->shut_phase);
+ BUG();
+ }
+
+ local_irq_save(flags);
+
+ spin_lock(&sess->sess_list_lock);
+ atomic_inc(&sess->sess_cmd_count);
+
+ if (unlikely(sess->init_phase != SCST_SESS_IPH_READY)) {
+ switch (sess->init_phase) {
+ case SCST_SESS_IPH_INITING:
+ TRACE_DBG("Adding mcmd %p to init deferred mcmd list",
+ mcmd);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry,
+ &sess->init_deferred_mcmd_list);
+ goto out_unlock;
+ case SCST_SESS_IPH_SUCCESS:
+ break;
+ case SCST_SESS_IPH_FAILED:
+ res = -1;
+ goto out_unlock;
+ default:
+ BUG();
+ }
+ }
+
+ spin_unlock(&sess->sess_list_lock);
+
+ TRACE_MGMT_DBG("Adding mgmt cmd %p to active mgmt cmd list", mcmd);
+ spin_lock(&scst_mcmd_lock);
+ list_add_tail(&mcmd->mgmt_cmd_list_entry, &scst_active_mgmt_cmd_list);
+ spin_unlock(&scst_mcmd_lock);
+
+ local_irq_restore(flags);
+
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+
+out:
+ return res;
+
+out_unlock:
+ spin_unlock(&sess->sess_list_lock);
+ local_irq_restore(flags);
+ goto out;
+}
+
+/**
+ * scst_rx_mgmt_fn() - create new management command and send it for execution
+ *
+ * Description:
+ * Creates new management command and sends it for execution.
+ *
+ * Returns 0 for success, error code otherwise.
+ *
+ * Must not be called in parallel with scst_unregister_session() for the
+ * same sess.
+ */
+int scst_rx_mgmt_fn(struct scst_session *sess,
+ const struct scst_rx_mgmt_params *params)
+{
+ int res = -EFAULT;
+ struct scst_mgmt_cmd *mcmd = NULL;
+
+ switch (params->fn) {
+ case SCST_ABORT_TASK:
+ BUG_ON(!params->tag_set);
+ break;
+ case SCST_TARGET_RESET:
+ case SCST_ABORT_ALL_TASKS:
+ case SCST_NEXUS_LOSS:
+ break;
+ default:
+ BUG_ON(!params->lun_set);
+ }
+
+ mcmd = scst_pre_rx_mgmt_cmd(sess, params->fn, params->atomic,
+ params->tgt_priv);
+ if (mcmd == NULL)
+ goto out;
+
+ if (params->lun_set) {
+ mcmd->lun = scst_unpack_lun(params->lun, params->lun_len);
+ if (mcmd->lun == NO_SUCH_LUN)
+ goto out_free;
+ mcmd->lun_set = 1;
+ }
+
+ if (params->tag_set)
+ mcmd->tag = params->tag;
+
+ mcmd->cmd_sn_set = params->cmd_sn_set;
+ mcmd->cmd_sn = params->cmd_sn;
+
+ TRACE(TRACE_MGMT, "TM fn %d", params->fn);
+
+ TRACE_MGMT_DBG("sess=%p, tag_set %d, tag %lld, lun_set %d, "
+ "lun=%lld, cmd_sn_set %d, cmd_sn %d, priv %p", sess,
+ params->tag_set,
+ (long long unsigned int)params->tag,
+ params->lun_set,
+ (long long unsigned int)mcmd->lun,
+ params->cmd_sn_set,
+ params->cmd_sn,
+ params->tgt_priv);
+
+ if (scst_post_rx_mgmt_cmd(sess, mcmd) != 0)
+ goto out_free;
+
+ res = 0;
+
+out:
+ return res;
+
+out_free:
+ scst_free_mgmt_cmd(mcmd);
+ mcmd = NULL;
+ goto out;
+}
+EXPORT_SYMBOL(scst_rx_mgmt_fn);
+
+/*
+ * Returns true if string "string" matches pattern "wild", false otherwise.
+ * Pattern is a regular DOS-type pattern, containing '*' and '?' symbols.
+ * '*' means match all any symbols, '?' means match only any single symbol.
+ *
+ * For instance:
+ * if (wildcmp("bl?h.*", "blah.jpg")) {
+ * // match
+ * } else {
+ * // no match
+ * }
+ *
+ * Written by Jack Handy - jakkhandy@hotmail.com
+ * Taken by Gennadiy Nerubayev <parakie@gmail.com> from
+ * http://www.codeproject.com/KB/string/wildcmp.aspx. No license attached
+ * to it, and it's posted on a free site; assumed to be free for use.
+ */
+static bool wildcmp(const char *wild, const char *string)
+{
+ const char *cp = NULL, *mp = NULL;
+
+ while ((*string) && (*wild != '*')) {
+ if ((*wild != *string) && (*wild != '?'))
+ return false;
+
+ wild++;
+ string++;
+ }
+
+ while (*string) {
+ if (*wild == '*') {
+ if (!*++wild)
+ return true;
+
+ mp = wild;
+ cp = string+1;
+ } else if ((*wild == *string) || (*wild == '?')) {
+ wild++;
+ string++;
+ } else {
+ wild = mp;
+ string = cp++;
+ }
+ }
+
+ while (*wild == '*')
+ wild++;
+
+ return !*wild;
+}
+
+/* scst_mutex supposed to be held */
+static struct scst_acg *scst_find_tgt_acg_by_name_wild(struct scst_tgt *tgt,
+ const char *initiator_name)
+{
+ struct scst_acg *acg, *res = NULL;
+ struct scst_acn *n;
+
+ if (initiator_name == NULL)
+ goto out;
+
+ list_for_each_entry(acg, &tgt->tgt_acg_list, acg_list_entry) {
+ list_for_each_entry(n, &acg->acn_list, acn_list_entry) {
+ if (wildcmp(n->name, initiator_name)) {
+ TRACE_DBG("Access control group %s found",
+ acg->acg_name);
+ res = acg;
+ goto out;
+ }
+ }
+ }
+
+out:
+ return res;
+}
+
+/* Must be called under scst_mutex */
+struct scst_acg *scst_find_acg(const struct scst_session *sess)
+{
+ struct scst_acg *acg = NULL;
+
+ acg = scst_find_tgt_acg_by_name_wild(sess->tgt, sess->initiator_name);
+ if (acg == NULL)
+ acg = sess->tgt->default_acg;
+ return acg;
+}
+
+static int scst_init_session(struct scst_session *sess)
+{
+ int res = 0, rc;
+ struct scst_cmd *cmd;
+ struct scst_mgmt_cmd *mcmd, *tm;
+ int mwake = 0;
+
+ mutex_lock(&scst_mutex);
+
+ sess->acg = scst_find_acg(sess);
+
+ PRINT_INFO("Using security group \"%s\" for initiator \"%s\"",
+ sess->acg->acg_name, sess->initiator_name);
+
+ list_add_tail(&sess->acg_sess_list_entry, &sess->acg->acg_sess_list);
+
+ TRACE_DBG("Adding sess %p to tgt->sess_list", sess);
+ list_add_tail(&sess->sess_list_entry, &sess->tgt->sess_list);
+
+ res = scst_sess_alloc_tgt_devs(sess);
+
+ /* Let's always create session's sysfs to simplify error recovery */
+ rc = scst_create_sess_sysfs(sess);
+ if (res == 0)
+ res = rc;
+
+ mutex_unlock(&scst_mutex);
+
+ if (sess->init_result_fn) {
+ TRACE_DBG("Calling init_result_fn(%p)", sess);
+ sess->init_result_fn(sess, sess->reg_sess_data, res);
+ TRACE_DBG("%s", "init_result_fn() returned");
+ }
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ if (res == 0)
+ sess->init_phase = SCST_SESS_IPH_SUCCESS;
+ else
+ sess->init_phase = SCST_SESS_IPH_FAILED;
+
+restart:
+ list_for_each_entry(cmd, &sess->init_deferred_cmd_list,
+ cmd_list_entry) {
+ TRACE_DBG("Deleting cmd %p from init deferred cmd list", cmd);
+ list_del(&cmd->cmd_list_entry);
+ atomic_dec(&sess->sess_cmd_count);
+ spin_unlock_irq(&sess->sess_list_lock);
+ scst_cmd_init_done(cmd, SCST_CONTEXT_THREAD);
+ spin_lock_irq(&sess->sess_list_lock);
+ goto restart;
+ }
+
+ spin_lock(&scst_mcmd_lock);
+ list_for_each_entry_safe(mcmd, tm, &sess->init_deferred_mcmd_list,
+ mgmt_cmd_list_entry) {
+ TRACE_DBG("Moving mgmt command %p from init deferred mcmd list",
+ mcmd);
+ list_move_tail(&mcmd->mgmt_cmd_list_entry,
+ &scst_active_mgmt_cmd_list);
+ mwake = 1;
+ }
+
+ spin_unlock(&scst_mcmd_lock);
+ /*
+ * In case of an error at this point the caller target driver supposed
+ * to already call this sess's unregistration.
+ */
+ sess->init_phase = SCST_SESS_IPH_READY;
+ spin_unlock_irq(&sess->sess_list_lock);
+
+ if (mwake)
+ wake_up(&scst_mgmt_cmd_list_waitQ);
+
+ scst_sess_put(sess);
+ return res;
+}
+
+/**
+ * scst_register_session() - register session
+ * @tgt: target
+ * @atomic: true, if the function called in the atomic context. If false,
+ * this function will block until the session registration is
+ * completed.
+ * @initiator_name: remote initiator's name, any NULL-terminated string,
+ * e.g. iSCSI name, which used as the key to found appropriate
+ * access control group. Could be NULL, then the default
+ * target's LUNs are used.
+ * @data: any target driver supplied data
+ * @result_fn: pointer to the function that will be asynchronously called
+ * when session initialization finishes.
+ * Can be NULL. Parameters:
+ * - sess - session
+ * - data - target driver supplied to scst_register_session()
+ * data
+ * - result - session initialization result, 0 on success or
+ * appropriate error code otherwise
+ *
+ * Description:
+ * Registers new session. Returns new session on success or NULL otherwise.
+ *
+ * Note: A session creation and initialization is a complex task,
+ * which requires sleeping state, so it can't be fully done
+ * in interrupt context. Therefore the "bottom half" of it, if
+ * scst_register_session() is called from atomic context, will be
+ * done in SCST thread context. In this case scst_register_session()
+ * will return not completely initialized session, but the target
+ * driver can supply commands to this session via scst_rx_cmd().
+ * Those commands processing will be delayed inside SCST until
+ * the session initialization is finished, then their processing
+ * will be restarted. The target driver will be notified about
+ * finish of the session initialization by function result_fn().
+ * On success the target driver could do nothing, but if the
+ * initialization fails, the target driver must ensure that
+ * no more new commands being sent or will be sent to SCST after
+ * result_fn() returns. All already sent to SCST commands for
+ * failed session will be returned in xmit_response() with BUSY status.
+ * In case of failure the driver shall call scst_unregister_session()
+ * inside result_fn(), it will NOT be called automatically.
+ */
+struct scst_session *scst_register_session(struct scst_tgt *tgt, int atomic,
+ const char *initiator_name, void *data,
+ void (*result_fn) (struct scst_session *sess, void *data, int result))
+{
+ struct scst_session *sess;
+ int res;
+ unsigned long flags;
+
+ sess = scst_alloc_session(tgt, atomic ? GFP_ATOMIC : GFP_KERNEL,
+ initiator_name);
+ if (sess == NULL)
+ goto out;
+
+ scst_sess_get(sess); /* one for registered session */
+ scst_sess_get(sess); /* one held until sess is inited */
+
+ if (atomic) {
+ sess->reg_sess_data = data;
+ sess->init_result_fn = result_fn;
+ spin_lock_irqsave(&scst_mgmt_lock, flags);
+ TRACE_DBG("Adding sess %p to scst_sess_init_list", sess);
+ list_add_tail(&sess->sess_init_list_entry,
+ &scst_sess_init_list);
+ spin_unlock_irqrestore(&scst_mgmt_lock, flags);
+ wake_up(&scst_mgmt_waitQ);
+ } else {
+ res = scst_init_session(sess);
+ if (res != 0)
+ goto out_free;
+ }
+
+out:
+ return sess;
+
+out_free:
+ scst_free_session(sess);
+ sess = NULL;
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_register_session);
+
+/**
+ * scst_register_session_simple() - register session (simple version)
+ * @tgt: target
+ * @initiator_name: remote initiator's name, any NULL-terminated string,
+ * e.g. iSCSI name, which used as the key to found appropriate
+ * access control group. Could be NULL, then the default
+ * target's LUNs are used.
+ *
+ * Description:
+ * Registers new session. Returns new session on success or NULL otherwise.
+ */
+struct scst_session *scst_register_session_simple(struct scst_tgt *tgt,
+ const char *initiator_name)
+{
+ return scst_register_session(tgt, 0, initiator_name, NULL, NULL);
+}
+EXPORT_SYMBOL(scst_register_session_simple);
+
+/**
+ * scst_unregister_session() - unregister session
+ * @sess: session to be unregistered
+ * @wait: if true, instructs to wait until all commands, which
+ * currently is being executed and belonged to the session,
+ * finished. Otherwise, target driver should be prepared to
+ * receive xmit_response() for the session's command after
+ * scst_unregister_session() returns.
+ * @unreg_done_fn: pointer to the function that will be asynchronously called
+ * when the last session's command finishes and
+ * the session is about to be completely freed. Can be NULL.
+ * Parameter:
+ * - sess - session
+ *
+ * Unregisters session.
+ *
+ * Notes:
+ * - All outstanding commands will be finished regularly. After
+ * scst_unregister_session() returned, no new commands must be sent to
+ * SCST via scst_rx_cmd().
+ *
+ * - The caller must ensure that no scst_rx_cmd() or scst_rx_mgmt_fn_*() is
+ * called in paralell with scst_unregister_session().
+ *
+ * - Can be called before result_fn() of scst_register_session() called,
+ * i.e. during the session registration/initialization.
+ *
+ * - It is highly recommended to call scst_unregister_session() as soon as it
+ * gets clear that session will be unregistered and not to wait until all
+ * related commands finished. This function provides the wait functionality,
+ * but it also starts recovering stuck commands, if there are any.
+ * Otherwise, your target driver could wait for those commands forever.
+ */
+void scst_unregister_session(struct scst_session *sess, int wait,
+ void (*unreg_done_fn) (struct scst_session *sess))
+{
+ unsigned long flags;
+ DECLARE_COMPLETION_ONSTACK(c);
+ int rc, lun;
+
+ TRACE_MGMT_DBG("Unregistering session %p (wait %d)", sess, wait);
+
+ sess->unreg_done_fn = unreg_done_fn;
+
+ /* Abort all outstanding commands and clear reservation, if necessary */
+ lun = 0;
+ rc = scst_rx_mgmt_fn_lun(sess, SCST_UNREG_SESS_TM,
+ (uint8_t *)&lun, sizeof(lun), SCST_ATOMIC, NULL);
+ if (rc != 0) {
+ PRINT_ERROR("SCST_UNREG_SESS_TM failed %d (sess %p)",
+ rc, sess);
+ }
+
+ sess->shut_phase = SCST_SESS_SPH_SHUTDOWN;
+
+ spin_lock_irqsave(&scst_mgmt_lock, flags);
+
+ if (wait)
+ sess->shutdown_compl = &c;
+
+ spin_unlock_irqrestore(&scst_mgmt_lock, flags);
+
+ scst_sess_put(sess);
+
+ if (wait) {
+ TRACE_DBG("Waiting for session %p to complete", sess);
+ wait_for_completion(&c);
+ }
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_unregister_session);
+
+/**
+ * scst_unregister_session_simple() - unregister session, simple version
+ * @sess: session to be unregistered
+ *
+ * Unregisters session.
+ *
+ * See notes for scst_unregister_session() above.
+ */
+void scst_unregister_session_simple(struct scst_session *sess)
+{
+
+ scst_unregister_session(sess, 1, NULL);
+ return;
+}
+EXPORT_SYMBOL(scst_unregister_session_simple);
+
+static inline int test_mgmt_list(void)
+{
+ int res = !list_empty(&scst_sess_init_list) ||
+ !list_empty(&scst_sess_shut_list) ||
+ unlikely(kthread_should_stop());
+ return res;
+}
+
+int scst_global_mgmt_thread(void *arg)
+{
+ struct scst_session *sess;
+
+ PRINT_INFO("Management thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ set_user_nice(current, -10);
+
+ spin_lock_irq(&scst_mgmt_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_mgmt_list()) {
+ add_wait_queue_exclusive(&scst_mgmt_waitQ, &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_mgmt_list())
+ break;
+ spin_unlock_irq(&scst_mgmt_lock);
+ schedule();
+ spin_lock_irq(&scst_mgmt_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&scst_mgmt_waitQ, &wait);
+ }
+
+ while (!list_empty(&scst_sess_init_list)) {
+ sess = list_entry(scst_sess_init_list.next,
+ typeof(*sess), sess_init_list_entry);
+ TRACE_DBG("Removing sess %p from scst_sess_init_list",
+ sess);
+ list_del(&sess->sess_init_list_entry);
+ spin_unlock_irq(&scst_mgmt_lock);
+
+ if (sess->init_phase == SCST_SESS_IPH_INITING)
+ scst_init_session(sess);
+ else {
+ PRINT_CRIT_ERROR("session %p is in "
+ "scst_sess_init_list, but in unknown "
+ "init phase %x", sess,
+ sess->init_phase);
+ BUG();
+ }
+
+ spin_lock_irq(&scst_mgmt_lock);
+ }
+
+ while (!list_empty(&scst_sess_shut_list)) {
+ sess = list_entry(scst_sess_shut_list.next,
+ typeof(*sess), sess_shut_list_entry);
+ TRACE_DBG("Removing sess %p from scst_sess_shut_list",
+ sess);
+ list_del(&sess->sess_shut_list_entry);
+ spin_unlock_irq(&scst_mgmt_lock);
+
+ switch (sess->shut_phase) {
+ case SCST_SESS_SPH_SHUTDOWN:
+ BUG_ON(atomic_read(&sess->refcnt) != 0);
+ scst_free_session_callback(sess);
+ break;
+ default:
+ PRINT_CRIT_ERROR("session %p is in "
+ "scst_sess_shut_list, but in unknown "
+ "shut phase %lx", sess,
+ sess->shut_phase);
+ BUG();
+ break;
+ }
+
+ spin_lock_irq(&scst_mgmt_lock);
+ }
+ }
+ spin_unlock_irq(&scst_mgmt_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so both lists must be empty.
+ */
+ BUG_ON(!list_empty(&scst_sess_init_list));
+ BUG_ON(!list_empty(&scst_sess_shut_list));
+
+ PRINT_INFO("Management thread PID %d finished", current->pid);
+ return 0;
+}
+
+/* Called under sess->sess_list_lock */
+static struct scst_cmd *__scst_find_cmd_by_tag(struct scst_session *sess,
+ uint64_t tag, bool to_abort)
+{
+ struct scst_cmd *cmd, *res = NULL;
+
+ /* ToDo: hash list */
+
+ TRACE_DBG("%s (sess=%p, tag=%llu)", "Searching in sess cmd list",
+ sess, (long long unsigned int)tag);
+
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if (cmd->tag == tag) {
+ /*
+ * We must not count done commands, because
+ * they were submitted for transmittion.
+ * Otherwise we can have a race, when for
+ * some reason cmd's release delayed
+ * after transmittion and initiator sends
+ * cmd with the same tag => it can be possible
+ * that a wrong cmd will be returned.
+ */
+ if (cmd->done) {
+ if (to_abort) {
+ /*
+ * We should return the latest not
+ * aborted cmd with this tag.
+ */
+ if (res == NULL)
+ res = cmd;
+ else {
+ if (test_bit(SCST_CMD_ABORTED,
+ &res->cmd_flags)) {
+ res = cmd;
+ } else if (!test_bit(SCST_CMD_ABORTED,
+ &cmd->cmd_flags))
+ res = cmd;
+ }
+ }
+ continue;
+ } else {
+ res = cmd;
+ break;
+ }
+ }
+ }
+ return res;
+}
+
+/**
+ * scst_find_cmd() - find command by custom comparison function
+ *
+ * Finds a command based on user supplied data and comparision
+ * callback function, that should return true, if the command is found.
+ * Returns the command on success or NULL otherwise
+ */
+struct scst_cmd *scst_find_cmd(struct scst_session *sess, void *data,
+ int (*cmp_fn) (struct scst_cmd *cmd,
+ void *data))
+{
+ struct scst_cmd *cmd = NULL;
+ unsigned long flags = 0;
+
+ if (cmp_fn == NULL)
+ goto out;
+
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list, sess_cmd_list_entry) {
+ /*
+ * We must not count done commands, because they were
+ * submitted for transmittion. Otherwise we can have a race,
+ * when for some reason cmd's release delayed after
+ * transmittion and initiator sends cmd with the same tag =>
+ * it can be possible that a wrong cmd will be returned.
+ */
+ if (cmd->done)
+ continue;
+ if (cmp_fn(cmd, data))
+ goto out_unlock;
+ }
+
+ cmd = NULL;
+
+out_unlock:
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+
+out:
+ return cmd;
+}
+EXPORT_SYMBOL(scst_find_cmd);
+
+/**
+ * scst_find_cmd_by_tag() - find command by tag
+ *
+ * Finds a command based on the supplied tag comparing it with one
+ * that previously set by scst_cmd_set_tag(). Returns the found command on
+ * success or NULL otherwise
+ */
+struct scst_cmd *scst_find_cmd_by_tag(struct scst_session *sess,
+ uint64_t tag)
+{
+ unsigned long flags;
+ struct scst_cmd *cmd;
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+ cmd = __scst_find_cmd_by_tag(sess, tag, false);
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ return cmd;
+}
+EXPORT_SYMBOL(scst_find_cmd_by_tag);
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 5/12/1/5] SCST core's scst_lib.c
[not found] ` <4BC44D08.4060907@vlnb.net>
` (3 preceding siblings ...)
2010-04-13 13:05 ` [PATCH][RFC 4/12/1/5] SCST core's scst_targ.c Vladislav Bolkhovitin
@ 2010-04-13 13:05 ` Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 6/12/1/5] SCST core's private header Vladislav Bolkhovitin
` (4 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:05 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains file scst_lib.c.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_lib.c | 6337 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 6337 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/scst_lib.c linux-2.6.33/drivers/scst/scst_lib.c
--- orig/linux-2.6.33/drivers/scst/scst_lib.c
+++ linux-2.6.33/drivers/scst/scst_lib.c
@@ -0,0 +1,6337 @@
+/*
+ * scst_lib.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/kthread.h>
+#include <linux/cdrom.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+#include <asm/kmap_types.h>
+#include <linux/ctype.h>
+#include <linux/delay.h>
+
+#include "scst.h"
+#include "scst_priv.h"
+#include "scst_mem.h"
+
+struct scsi_io_context {
+ unsigned int full_cdb_used:1;
+ void *data;
+ void (*done)(void *data, char *sense, int result, int resid);
+ char sense[SCST_SENSE_BUFFERSIZE];
+ unsigned char full_cdb[0];
+};
+static struct kmem_cache *scsi_io_context_cache;
+
+/* get_trans_len_x extract x bytes from cdb as length starting from off */
+static int get_trans_len_1(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_1_256(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_2(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_3(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_4(struct scst_cmd *cmd, uint8_t off);
+
+/* for special commands */
+static int get_trans_len_block_limit(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_read_capacity(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_serv_act_in(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_single(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_none(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_read_pos(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_cdb_len_10(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_prevent_allow_medium_removal(struct scst_cmd *cmd,
+ uint8_t off);
+static int get_trans_len_3_read_elem_stat(struct scst_cmd *cmd, uint8_t off);
+static int get_trans_len_start_stop(struct scst_cmd *cmd, uint8_t off);
+
+/*
++=====================================-============-======-
+| Command name | Operation | Type |
+| | code | |
+|-------------------------------------+------------+------+
+
++=========================================================+
+|Key: M = command implementation is mandatory. |
+| O = command implementation is optional. |
+| V = Vendor-specific |
+| R = Reserved |
+| ' '= DON'T use for this device |
++=========================================================+
+*/
+
+#define SCST_CDB_MANDATORY 'M' /* mandatory */
+#define SCST_CDB_OPTIONAL 'O' /* optional */
+#define SCST_CDB_VENDOR 'V' /* vendor */
+#define SCST_CDB_RESERVED 'R' /* reserved */
+#define SCST_CDB_NOTSUPP ' ' /* don't use */
+
+struct scst_sdbops {
+ uint8_t ops; /* SCSI-2 op codes */
+ uint8_t devkey[16]; /* Key for every device type M,O,V,R
+ * type_disk devkey[0]
+ * type_tape devkey[1]
+ * type_printer devkey[2]
+ * type_proseccor devkey[3]
+ * type_worm devkey[4]
+ * type_cdrom devkey[5]
+ * type_scanner devkey[6]
+ * type_mod devkey[7]
+ * type_changer devkey[8]
+ * type_commdev devkey[9]
+ * type_reserv devkey[A]
+ * type_reserv devkey[B]
+ * type_raid devkey[C]
+ * type_enclosure devkey[D]
+ * type_reserv devkey[E]
+ * type_reserv devkey[F]
+ */
+ const char *op_name; /* SCSI-2 op codes full name */
+ uint8_t direction; /* init --> target: SCST_DATA_WRITE
+ * target --> init: SCST_DATA_READ
+ */
+ uint16_t flags; /* opcode -- various flags */
+ uint8_t off; /* length offset in cdb */
+ int (*get_trans_len)(struct scst_cmd *cmd, uint8_t off)
+ __attribute__ ((aligned));
+} __attribute__((packed));
+
+static int scst_scsi_op_list[256];
+
+#define FLAG_NONE 0
+
+static const struct scst_sdbops scst_scsi_op_table[] = {
+ /*
+ * +-------------------> TYPE_IS_DISK (0)
+ * |
+ * |+------------------> TYPE_IS_TAPE (1)
+ * ||
+ * || +----------------> TYPE_IS_PROCESSOR (3)
+ * || |
+ * || | +--------------> TYPE_IS_CDROM (5)
+ * || | |
+ * || | | +------------> TYPE_IS_MOD (7)
+ * || | | |
+ * || | | |+-----------> TYPE_IS_CHANGER (8)
+ * || | | ||
+ * || | | || +-------> TYPE_IS_RAID (C)
+ * || | | || |
+ * || | | || |
+ * 0123456789ABCDEF ---> TYPE_IS_???? */
+
+ /* 6-bytes length CDB */
+ {0x00, "MMMMMMMMMMMMMMMM", "TEST UNIT READY",
+ /* let's be HQ to don't look dead under high load */
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_IMPLICIT_HQ|
+ SCST_REG_RESERVE_ALLOWED,
+ 0, get_trans_len_none},
+ {0x01, " M ", "REWIND",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x01, "O V OO OO ", "REZERO UNIT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x02, "VVVVVV V ", "REQUEST BLOCK ADDR",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT, 0, get_trans_len_none},
+ {0x03, "MMMMMMMMMMMMMMMM", "REQUEST SENSE",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_SKIP_UA|SCST_LOCAL_CMD|
+ SCST_REG_RESERVE_ALLOWED,
+ 4, get_trans_len_1},
+ {0x04, "M O O ", "FORMAT UNIT",
+ SCST_DATA_WRITE, SCST_LONG_TIMEOUT|SCST_UNKNOWN_LENGTH|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x04, " O ", "FORMAT",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x05, "VMVVVV V ", "READ BLOCK LIMITS",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_REG_RESERVE_ALLOWED,
+ 0, get_trans_len_block_limit},
+ {0x07, " O ", "INITIALIZE ELEMENT STATUS",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x07, "OVV O OV ", "REASSIGN BLOCKS",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x08, "O ", "READ(6)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 4, get_trans_len_1_256},
+ {0x08, " MV OO OV ", "READ(6)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 2, get_trans_len_3},
+ {0x08, " M ", "GET MESSAGE(6)",
+ SCST_DATA_READ, FLAG_NONE, 2, get_trans_len_3},
+ {0x08, " O ", "RECEIVE",
+ SCST_DATA_READ, FLAG_NONE, 2, get_trans_len_3},
+ {0x0A, "O ", "WRITE(6)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 4, get_trans_len_1_256},
+ {0x0A, " M O OV ", "WRITE(6)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 2, get_trans_len_3},
+ {0x0A, " M ", "PRINT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0A, " M ", "SEND MESSAGE(6)",
+ SCST_DATA_WRITE, FLAG_NONE, 2, get_trans_len_3},
+ {0x0A, " M ", "SEND(6)",
+ SCST_DATA_WRITE, FLAG_NONE, 2, get_trans_len_3},
+ {0x0B, "O OO OV ", "SEEK(6)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0B, " ", "TRACK SELECT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0B, " O ", "SLEW AND PRINT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x0C, "VVVVVV V ", "SEEK BLOCK",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x0D, "VVVVVV V ", "PARTITION",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x0F, "VOVVVV V ", "READ REVERSE",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 2, get_trans_len_3},
+ {0x10, "VM V V ", "WRITE FILEMARKS",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x10, " O O ", "SYNCHRONIZE BUFFER",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x11, "VMVVVV ", "SPACE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x12, "MMMMMMMMMMMMMMMM", "INQUIRY",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_IMPLICIT_HQ|SCST_SKIP_UA|
+ SCST_REG_RESERVE_ALLOWED,
+ 4, get_trans_len_1},
+ {0x13, "VOVVVV ", "VERIFY(6)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED,
+ 2, get_trans_len_3},
+ {0x14, "VOOVVV ", "RECOVER BUFFERED DATA",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 2, get_trans_len_3},
+ {0x15, "OMOOOOOOOOOOOOOO", "MODE SELECT(6)",
+ SCST_DATA_WRITE, SCST_LOCAL_CMD, 4, get_trans_len_1},
+ {0x16, "MMMMMMMMMMMMMMMM", "RESERVE",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD,
+ 0, get_trans_len_none},
+ {0x17, "MMMMMMMMMMMMMMMM", "RELEASE",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD|SCST_REG_RESERVE_ALLOWED,
+ 0, get_trans_len_none},
+ {0x18, "OOOOOOOO ", "COPY",
+ SCST_DATA_WRITE, SCST_LONG_TIMEOUT, 2, get_trans_len_3},
+ {0x19, "VMVVVV ", "ERASE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x1A, "OMOOOOOOOOOOOOOO", "MODE SENSE(6)",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT, 4, get_trans_len_1},
+ {0x1B, " O ", "SCAN",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x1B, " O ", "LOAD UNLOAD",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x1B, " O ", "STOP PRINT",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x1B, "O OO O O ", "START STOP UNIT",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_start_stop},
+ {0x1C, "OOOOOOOOOOOOOOOO", "RECEIVE DIAGNOSTIC RESULTS",
+ SCST_DATA_READ, FLAG_NONE, 3, get_trans_len_2},
+ {0x1D, "MMMMMMMMMMMMMMMM", "SEND DIAGNOSTIC",
+ SCST_DATA_WRITE, FLAG_NONE, 4, get_trans_len_1},
+ {0x1E, "OOOOOOOOOOOOOOOO", "PREVENT ALLOW MEDIUM REMOVAL",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0,
+ get_trans_len_prevent_allow_medium_removal},
+ {0x1F, " O ", "PORT STATUS",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+
+ /* 10-bytes length CDB */
+ {0x23, "V VV V ", "READ FORMAT CAPACITY",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x24, "V VVM ", "SET WINDOW",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_3},
+ {0x25, "M MM M ", "READ CAPACITY",
+ SCST_DATA_READ, SCST_IMPLICIT_HQ|SCST_REG_RESERVE_ALLOWED,
+ 0, get_trans_len_read_capacity},
+ {0x25, " O ", "GET WINDOW",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_3},
+ {0x28, "M MMMM ", "READ(10)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 7, get_trans_len_2},
+ {0x28, " O ", "GET MESSAGE(10)",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x29, "V VV O ", "READ GENERATION",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_1},
+ {0x2A, "O MO M ", "WRITE(10)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 7, get_trans_len_2},
+ {0x2A, " O ", "SEND MESSAGE(10)",
+ SCST_DATA_WRITE, FLAG_NONE, 7, get_trans_len_2},
+ {0x2A, " O ", "SEND(10)",
+ SCST_DATA_WRITE, FLAG_NONE, 7, get_trans_len_2},
+ {0x2B, " O ", "LOCATE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x2B, " O ", "POSITION TO ELEMENT",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x2B, "O OO O ", "SEEK(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x2C, "V O O ", "ERASE(10)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x2D, "V O O ", "READ UPDATED BLOCK",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 0, get_trans_len_single},
+ {0x2E, "O OO O ", "WRITE AND VERIFY(10)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 7, get_trans_len_2},
+ {0x2F, "O OO O ", "VERIFY(10)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED,
+ 7, get_trans_len_2},
+ {0x33, "O OO O ", "SET LIMITS(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x34, " O ", "READ POSITION",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT, 7, get_trans_len_read_pos},
+ {0x34, " O ", "GET DATA BUFFER STATUS",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x34, "O OO O ", "PRE-FETCH",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x35, "O OO O ", "SYNCHRONIZE CACHE",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x36, "O OO O ", "LOCK UNLOCK CACHE",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x37, "O O ", "READ DEFECT DATA(10)",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_1},
+ {0x37, " O ", "INIT ELEMENT STATUS WRANGE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x38, " O O ", "MEDIUM SCAN",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_1},
+ {0x39, "OOOOOOOO ", "COMPARE",
+ SCST_DATA_WRITE, FLAG_NONE, 3, get_trans_len_3},
+ {0x3A, "OOOOOOOO ", "COPY AND VERIFY",
+ SCST_DATA_WRITE, FLAG_NONE, 3, get_trans_len_3},
+ {0x3B, "OOOOOOOOOOOOOOOO", "WRITE BUFFER",
+ SCST_DATA_WRITE, SCST_SMALL_TIMEOUT, 6, get_trans_len_3},
+ {0x3C, "OOOOOOOOOOOOOOOO", "READ BUFFER",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT, 6, get_trans_len_3},
+ {0x3D, " O O ", "UPDATE BLOCK",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED,
+ 0, get_trans_len_single},
+ {0x3E, "O OO O ", "READ LONG",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x3F, "O O O ", "WRITE LONG",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 7, get_trans_len_2},
+ {0x40, "OOOOOOOOOO ", "CHANGE DEFINITION",
+ SCST_DATA_WRITE, SCST_SMALL_TIMEOUT, 8, get_trans_len_1},
+ {0x41, "O O ", "WRITE SAME",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_single},
+ {0x42, " O ", "READ SUB-CHANNEL",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x43, " O ", "READ TOC/PMA/ATIP",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x44, " M ", "REPORT DENSITY SUPPORT",
+ SCST_DATA_READ, SCST_REG_RESERVE_ALLOWED, 7, get_trans_len_2},
+ {0x44, " O ", "READ HEADER",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x45, " O ", "PLAY AUDIO(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x46, " O ", "GET CONFIGURATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x47, " O ", "PLAY AUDIO MSF",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x48, " O ", "PLAY AUDIO TRACK INDEX",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x49, " O ", "PLAY TRACK RELATIVE(10)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x4A, " O ", "GET EVENT STATUS NOTIFICATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x4B, " O ", "PAUSE/RESUME",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x4C, "OOOOOOOOOOOOOOOO", "LOG SELECT",
+ SCST_DATA_WRITE, SCST_SMALL_TIMEOUT, 7, get_trans_len_2},
+ {0x4D, "OOOOOOOOOOOOOOOO", "LOG SENSE",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_REG_RESERVE_ALLOWED,
+ 7, get_trans_len_2},
+ {0x4E, " O ", "STOP PLAY/SCAN",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x50, " ", "XDWRITE",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x51, " O ", "READ DISC INFORMATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x51, " ", "XPWRITE",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x52, " O ", "READ TRACK INFORMATION",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x53, " O ", "RESERVE TRACK",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x54, " O ", "SEND OPC INFORMATION",
+ SCST_DATA_WRITE, FLAG_NONE, 7, get_trans_len_2},
+ {0x55, "OOOOOOOOOOOOOOOO", "MODE SELECT(10)",
+ SCST_DATA_WRITE, SCST_LOCAL_CMD, 7, get_trans_len_2},
+ {0x56, "OOOOOOOOOOOOOOOO", "RESERVE(10)",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD,
+ 0, get_trans_len_none},
+ {0x57, "OOOOOOOOOOOOOOOO", "RELEASE(10)",
+ SCST_DATA_NONE, SCST_SMALL_TIMEOUT|SCST_LOCAL_CMD|SCST_REG_RESERVE_ALLOWED,
+ 0, get_trans_len_none},
+ {0x58, " O ", "REPAIR TRACK",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x5A, "OOOOOOOOOOOOOOOO", "MODE SENSE(10)",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT, 7, get_trans_len_2},
+ {0x5B, " O ", "CLOSE TRACK/SESSION",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x5C, " O ", "READ BUFFER CAPACITY",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_2},
+ {0x5D, " O ", "SEND CUE SHEET",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_3},
+ {0x5E, "OOOOO OOOO ", "PERSISTENT RESERV IN",
+ SCST_DATA_READ, FLAG_NONE, 5, get_trans_len_4},
+ {0x5F, "OOOOO OOOO ", "PERSISTENT RESERV OUT",
+ SCST_DATA_WRITE, FLAG_NONE, 5, get_trans_len_4},
+
+ /* 16-bytes length CDB */
+ {0x80, "O OO O ", "XDWRITE EXTENDED",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x80, " M ", "WRITE FILEMARKS",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0x81, "O OO O ", "REBUILD",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x82, "O OO O ", "REGENERATE",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x83, "OOOOOOOOOOOOOOOO", "EXTENDED COPY",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x84, "OOOOOOOOOOOOOOOO", "RECEIVE COPY RESULT",
+ SCST_DATA_WRITE, FLAG_NONE, 10, get_trans_len_4},
+ {0x86, "OOOOOOOOOO ", "ACCESS CONTROL IN",
+ SCST_DATA_NONE, SCST_REG_RESERVE_ALLOWED, 0, get_trans_len_none},
+ {0x87, "OOOOOOOOOO ", "ACCESS CONTROL OUT",
+ SCST_DATA_NONE, SCST_REG_RESERVE_ALLOWED, 0, get_trans_len_none},
+ {0x88, "M MMMM ", "READ(16)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 10, get_trans_len_4},
+ {0x8A, "O OO O ", "WRITE(16)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 10, get_trans_len_4},
+ {0x8C, "OOOOOOOOOO ", "READ ATTRIBUTE",
+ SCST_DATA_READ, FLAG_NONE, 10, get_trans_len_4},
+ {0x8D, "OOOOOOOOOO ", "WRITE ATTRIBUTE",
+ SCST_DATA_WRITE, SCST_WRITE_MEDIUM, 10, get_trans_len_4},
+ {0x8E, "O OO O ", "WRITE AND VERIFY(16)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 10, get_trans_len_4},
+ {0x8F, "O OO O ", "VERIFY(16)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED,
+ 10, get_trans_len_4},
+ {0x90, "O OO O ", "PRE-FETCH(16)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x91, "O OO O ", "SYNCHRONIZE CACHE(16)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x91, " M ", "SPACE(16)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x92, "O OO O ", "LOCK UNLOCK CACHE(16)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0x92, " O ", "LOCATE(16)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0x93, "O O ", "WRITE SAME(16)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 10, get_trans_len_4},
+ {0x93, " M ", "ERASE(16)",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT|SCST_WRITE_MEDIUM,
+ 0, get_trans_len_none},
+ {0x9E, "O ", "SERVICE ACTION IN",
+ SCST_DATA_READ, FLAG_NONE, 0, get_trans_len_serv_act_in},
+
+ /* 12-bytes length CDB */
+ {0xA0, "VVVVVVVVVV M ", "REPORT LUNS",
+ SCST_DATA_READ, SCST_SMALL_TIMEOUT|SCST_IMPLICIT_HQ|SCST_SKIP_UA|
+ SCST_FULLY_LOCAL_CMD|SCST_LOCAL_CMD|
+ SCST_REG_RESERVE_ALLOWED,
+ 6, get_trans_len_4},
+ {0xA1, " O ", "BLANK",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xA3, " O ", "SEND KEY",
+ SCST_DATA_WRITE, FLAG_NONE, 8, get_trans_len_2},
+ {0xA3, "OOOOO OOOO ", "REPORT DEVICE IDENTIDIER",
+ SCST_DATA_READ, SCST_REG_RESERVE_ALLOWED, 6, get_trans_len_4},
+ {0xA3, " M ", "MAINTENANCE(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xA4, " O ", "REPORT KEY",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_2},
+ {0xA4, " O ", "MAINTENANCE(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xA5, " M ", "MOVE MEDIUM",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xA5, " O ", "PLAY AUDIO(12)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xA6, " O O ", "EXCHANGE/LOAD/UNLOAD MEDIUM",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xA7, " O ", "SET READ AHEAD",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xA8, " O ", "GET MESSAGE(12)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xA8, "O OO O ", "READ(12)",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 6, get_trans_len_4},
+ {0xA9, " O ", "PLAY TRACK RELATIVE(12)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xAA, "O OO O ", "WRITE(12)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 6, get_trans_len_4},
+ {0xAA, " O ", "SEND MESSAGE(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xAC, " O ", "ERASE(12)",
+ SCST_DATA_NONE, SCST_WRITE_MEDIUM, 0, get_trans_len_none},
+ {0xAC, " M ", "GET PERFORMANCE",
+ SCST_DATA_READ, SCST_UNKNOWN_LENGTH, 0, get_trans_len_none},
+ {0xAD, " O ", "READ DVD STRUCTURE",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_2},
+ {0xAE, "O OO O ", "WRITE AND VERIFY(12)",
+ SCST_DATA_WRITE, SCST_TRANSFER_LEN_TYPE_FIXED|SCST_WRITE_MEDIUM,
+ 6, get_trans_len_4},
+ {0xAF, "O OO O ", "VERIFY(12)",
+ SCST_DATA_NONE, SCST_TRANSFER_LEN_TYPE_FIXED|
+ SCST_VERIFY_BYTCHK_MISMATCH_ALLOWED,
+ 6, get_trans_len_4},
+#if 0 /* No need to support at all */
+ {0xB0, " OO O ", "SEARCH DATA HIGH(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+ {0xB1, " OO O ", "SEARCH DATA EQUAL(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+ {0xB2, " OO O ", "SEARCH DATA LOW(12)",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+#endif
+ {0xB3, " OO O ", "SET LIMITS(12)",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xB5, " O ", "REQUEST VOLUME ELEMENT ADDRESS",
+ SCST_DATA_READ, FLAG_NONE, 9, get_trans_len_1},
+ {0xB6, " O ", "SEND VOLUME TAG",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_1},
+ {0xB6, " M ", "SET STREAMING",
+ SCST_DATA_WRITE, FLAG_NONE, 9, get_trans_len_2},
+ {0xB7, " O ", "READ DEFECT DATA(12)",
+ SCST_DATA_READ, FLAG_NONE, 9, get_trans_len_1},
+ {0xB8, " O ", "READ ELEMENT STATUS",
+ SCST_DATA_READ, FLAG_NONE, 7, get_trans_len_3_read_elem_stat},
+ {0xB9, " O ", "READ CD MSF",
+ SCST_DATA_READ, SCST_UNKNOWN_LENGTH, 0, get_trans_len_none},
+ {0xBA, " O ", "SCAN",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_len_none},
+ {0xBA, " O ", "REDUNDANCY GROUP(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xBB, " O ", "SET SPEED",
+ SCST_DATA_NONE, FLAG_NONE, 0, get_trans_len_none},
+ {0xBB, " O ", "REDUNDANCY GROUP(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xBC, " O ", "SPARE(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xBD, " O ", "MECHANISM STATUS",
+ SCST_DATA_READ, FLAG_NONE, 8, get_trans_len_2},
+ {0xBD, " O ", "SPARE(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xBE, " O ", "READ CD",
+ SCST_DATA_READ, SCST_TRANSFER_LEN_TYPE_FIXED, 6, get_trans_len_3},
+ {0xBE, " O ", "VOLUME SET(IN)",
+ SCST_DATA_READ, FLAG_NONE, 6, get_trans_len_4},
+ {0xBF, " O ", "SEND DVD STRUCTUE",
+ SCST_DATA_WRITE, FLAG_NONE, 8, get_trans_len_2},
+ {0xBF, " O ", "VOLUME SET(OUT)",
+ SCST_DATA_WRITE, FLAG_NONE, 6, get_trans_len_4},
+ {0xE7, " V ", "INIT ELEMENT STATUS WRANGE",
+ SCST_DATA_NONE, SCST_LONG_TIMEOUT, 0, get_trans_cdb_len_10}
+};
+
+#define SCST_CDB_TBL_SIZE ((int)ARRAY_SIZE(scst_scsi_op_table))
+
+static void scst_free_tgt_dev(struct scst_tgt_dev *tgt_dev);
+static void scst_check_internal_sense(struct scst_device *dev, int result,
+ uint8_t *sense, int sense_len);
+static void scst_queue_report_luns_changed_UA(struct scst_session *sess,
+ int flags);
+static void __scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags);
+static void scst_alloc_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags);
+static void scst_free_all_UA(struct scst_tgt_dev *tgt_dev);
+static void scst_release_space(struct scst_cmd *cmd);
+static void scst_unblock_cmds(struct scst_device *dev);
+static void scst_clear_reservation(struct scst_tgt_dev *tgt_dev);
+static struct scst_tgt_dev *scst_alloc_add_tgt_dev(struct scst_session *sess,
+ struct scst_acg_dev *acg_dev);
+static void scst_tgt_retry_timer_fn(unsigned long arg);
+
+#ifdef CONFIG_SCST_DEBUG_TM
+static void tm_dbg_init_tgt_dev(struct scst_tgt_dev *tgt_dev);
+static void tm_dbg_deinit_tgt_dev(struct scst_tgt_dev *tgt_dev);
+#else
+static inline void tm_dbg_init_tgt_dev(struct scst_tgt_dev *tgt_dev) {}
+static inline void tm_dbg_deinit_tgt_dev(struct scst_tgt_dev *tgt_dev) {}
+#endif /* CONFIG_SCST_DEBUG_TM */
+
+/**
+ * scst_alloc_sense() - allocate sense buffer for command
+ *
+ * Allocates, if necessary, sense buffer for command. Returns 0 on success
+ * and error code othrwise. Parameter "atomic" should be non-0 if the
+ * function called in atomic context.
+ */
+int scst_alloc_sense(struct scst_cmd *cmd, int atomic)
+{
+ int res = 0;
+ gfp_t gfp_mask = atomic ? GFP_ATOMIC : (GFP_KERNEL|__GFP_NOFAIL);
+
+ if (cmd->sense != NULL)
+ goto memzero;
+
+ cmd->sense = mempool_alloc(scst_sense_mempool, gfp_mask);
+ if (cmd->sense == NULL) {
+ PRINT_CRIT_ERROR("Sense memory allocation failed (op %x). "
+ "The sense data will be lost!!", cmd->cdb[0]);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ cmd->sense_buflen = SCST_SENSE_BUFFERSIZE;
+
+memzero:
+ cmd->sense_valid_len = 0;
+ memset(cmd->sense, 0, cmd->sense_buflen);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_alloc_sense);
+
+/**
+ * scst_alloc_set_sense() - allocate and fill sense buffer for command
+ *
+ * Allocates, if necessary, sense buffer for command and copies in
+ * it data from the supplied sense buffer. Returns 0 on success
+ * and error code othrwise.
+ */
+int scst_alloc_set_sense(struct scst_cmd *cmd, int atomic,
+ const uint8_t *sense, unsigned int len)
+{
+ int res;
+
+ /*
+ * We don't check here if the existing sense is valid or not, because
+ * we suppose the caller did it based on cmd->status.
+ */
+
+ res = scst_alloc_sense(cmd, atomic);
+ if (res != 0) {
+ PRINT_BUFFER("Lost sense", sense, len);
+ goto out;
+ }
+
+ cmd->sense_valid_len = len;
+ if (cmd->sense_buflen < len) {
+ PRINT_WARNING("Sense truncated (needed %d), shall you increase "
+ "SCST_SENSE_BUFFERSIZE? Op: %x", len, cmd->cdb[0]);
+ cmd->sense_valid_len = cmd->sense_buflen;
+ }
+
+ memcpy(cmd->sense, sense, cmd->sense_valid_len);
+ TRACE_BUFFER("Sense set", cmd->sense, cmd->sense_valid_len);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_alloc_set_sense);
+
+/**
+ * scst_set_cmd_error_status() - set error SCSI status
+ * @cmd: SCST command
+ * @status: SCSI status to set
+ *
+ * Description:
+ * Sets error SCSI status in the command and prepares it for returning it.
+ * Returns 0 on success, error code otherwise.
+ */
+int scst_set_cmd_error_status(struct scst_cmd *cmd, int status)
+{
+ int res = 0;
+
+ if (cmd->status != 0) {
+ TRACE_MGMT_DBG("cmd %p already has status %x set", cmd,
+ cmd->status);
+ res = -EEXIST;
+ goto out;
+ }
+
+ cmd->status = status;
+ cmd->host_status = DID_OK;
+
+ cmd->dbl_ua_orig_resp_data_len = cmd->resp_data_len;
+ cmd->dbl_ua_orig_data_direction = cmd->data_direction;
+
+ cmd->data_direction = SCST_DATA_NONE;
+ cmd->resp_data_len = 0;
+ cmd->is_send_status = 1;
+
+ cmd->completed = 1;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_set_cmd_error_status);
+
+static int scst_set_lun_not_supported_request_sense(struct scst_cmd *cmd,
+ int key, int asc, int ascq)
+{
+ int res;
+ int sense_len;
+
+ if (cmd->status != 0) {
+ TRACE_MGMT_DBG("cmd %p already has status %x set", cmd,
+ cmd->status);
+ res = -EEXIST;
+ goto out;
+ }
+
+ if ((cmd->sg != NULL) && SCST_SENSE_VALID(sg_virt(cmd->sg))) {
+ TRACE_MGMT_DBG("cmd %p already has sense set", cmd);
+ res = -EEXIST;
+ goto out;
+ }
+
+ if (cmd->sg == NULL) {
+ if (cmd->bufflen == 0)
+ cmd->bufflen = cmd->cdb[4];
+
+ cmd->sg = scst_alloc(cmd->bufflen, GFP_ATOMIC, &cmd->sg_cnt);
+ if (cmd->sg == NULL) {
+ PRINT_ERROR("Unable to alloc sg for REQUEST SENSE"
+ "(sense %x/%x/%x)", key, asc, ascq);
+ res = 1;
+ goto out;
+ }
+ }
+
+ TRACE_MEM("sg %p alloced for sense for cmd %p (cnt %d, "
+ "len %d)", cmd->sg, cmd, cmd->sg_cnt, cmd->bufflen);
+
+ sense_len = scst_set_sense(sg_virt(cmd->sg),
+ cmd->bufflen, cmd->cdb[1] & 1, key, asc, ascq);
+ scst_set_resp_data_len(cmd, sense_len);
+
+ TRACE_BUFFER("Sense set", sg_virt(cmd->sg), sense_len);
+
+ res = 0;
+ cmd->completed = 1;
+
+out:
+ return res;
+}
+
+static int scst_set_lun_not_supported_inquiry(struct scst_cmd *cmd)
+{
+ int res;
+ uint8_t *buf;
+ int len;
+
+ if (cmd->status != 0) {
+ TRACE_MGMT_DBG("cmd %p already has status %x set", cmd,
+ cmd->status);
+ res = -EEXIST;
+ goto out;
+ }
+
+ if (cmd->sg == NULL) {
+ if (cmd->bufflen == 0)
+ cmd->bufflen = min_t(int, 36, (cmd->cdb[3] << 8) | cmd->cdb[4]);
+
+ cmd->sg = scst_alloc(cmd->bufflen, GFP_ATOMIC, &cmd->sg_cnt);
+ if (cmd->sg == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc sg for INQUIRY "
+ "for not supported LUN");
+ res = 1;
+ goto out;
+ }
+ }
+
+ TRACE_MEM("sg %p alloced INQUIRY for cmd %p (cnt %d, len %d)",
+ cmd->sg, cmd, cmd->sg_cnt, cmd->bufflen);
+
+ buf = sg_virt(cmd->sg);
+ len = min_t(int, 36, cmd->bufflen);
+
+ memset(buf, 0, len);
+ buf[0] = 0x7F; /* Peripheral qualifier 011b, Peripheral device type 1Fh */
+
+ TRACE_BUFFER("INQUIRY for not supported LUN set", buf, len);
+
+ res = 0;
+ cmd->completed = 1;
+
+out:
+ return res;
+}
+
+/**
+ * scst_set_cmd_error() - set error in the command and fill the sense buffer.
+ *
+ * Sets error in the command and fill the sense buffer. Returns 0 on success,
+ * error code otherwise.
+ */
+int scst_set_cmd_error(struct scst_cmd *cmd, int key, int asc, int ascq)
+{
+ int res;
+
+ /*
+ * We need for LOGICAL UNIT NOT SUPPORTED special handling for
+ * REQUEST SENSE and INQUIRY.
+ */
+ if ((key == ILLEGAL_REQUEST) && (asc == 0x25) && (ascq == 0)) {
+ if (cmd->cdb[0] == REQUEST_SENSE)
+ res = scst_set_lun_not_supported_request_sense(cmd,
+ key, asc, ascq);
+ else if (cmd->cdb[0] == INQUIRY)
+ res = scst_set_lun_not_supported_inquiry(cmd);
+ else
+ goto do_sense;
+
+ if (res > 0)
+ goto do_sense;
+ else
+ goto out;
+ }
+
+do_sense:
+ res = scst_set_cmd_error_status(cmd, SAM_STAT_CHECK_CONDITION);
+ if (res != 0)
+ goto out;
+
+ res = scst_alloc_sense(cmd, 1);
+ if (res != 0) {
+ PRINT_ERROR("Lost sense data (key %x, asc %x, ascq %x)",
+ key, asc, ascq);
+ goto out;
+ }
+
+ cmd->sense_valid_len = scst_set_sense(cmd->sense, cmd->sense_buflen,
+ scst_get_cmd_dev_d_sense(cmd), key, asc, ascq);
+ TRACE_BUFFER("Sense set", cmd->sense, cmd->sense_valid_len);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_set_cmd_error);
+
+/**
+ * scst_set_sense() - set sense from KEY/ASC/ASCQ numbers
+ *
+ * Sets the corresponding fields in the sense buffer taking sense type
+ * into account. Returns resulting sense length.
+ */
+int scst_set_sense(uint8_t *buffer, int len, bool d_sense,
+ int key, int asc, int ascq)
+{
+ int res;
+
+ BUG_ON(len == 0);
+
+ memset(buffer, 0, len);
+
+ if (d_sense) {
+ /* Descriptor format */
+ if (len < 8) {
+ PRINT_ERROR("Length %d of sense buffer too small to "
+ "fit sense %x:%x:%x", len, key, asc, ascq);
+ }
+
+ buffer[0] = 0x72; /* Response Code */
+ if (len > 1)
+ buffer[1] = key; /* Sense Key */
+ if (len > 2)
+ buffer[2] = asc; /* ASC */
+ if (len > 3)
+ buffer[3] = ascq; /* ASCQ */
+ res = 8;
+ } else {
+ /* Fixed format */
+ if (len < 18) {
+ PRINT_ERROR("Length %d of sense buffer too small to "
+ "fit sense %x:%x:%x", len, key, asc, ascq);
+ }
+
+ buffer[0] = 0x70; /* Response Code */
+ if (len > 2)
+ buffer[2] = key; /* Sense Key */
+ if (len > 7)
+ buffer[7] = 0x0a; /* Additional Sense Length */
+ if (len > 12)
+ buffer[12] = asc; /* ASC */
+ if (len > 13)
+ buffer[13] = ascq; /* ASCQ */
+ res = 18;
+ }
+
+ TRACE_BUFFER("Sense set", buffer, res);
+ return res;
+}
+EXPORT_SYMBOL(scst_set_sense);
+
+/**
+ * scst_analyze_sense() - analyze sense
+ *
+ * Returns true if sense matches to (key, asc, ascq) and false otherwise.
+ * Valid_mask is one or several SCST_SENSE_*_VALID constants setting valid
+ * (key, asc, ascq) values.
+ */
+bool scst_analyze_sense(const uint8_t *sense, int len, unsigned int valid_mask,
+ int key, int asc, int ascq)
+{
+ bool res = false;
+
+ /* Response Code */
+ if ((sense[0] == 0x70) || (sense[0] == 0x71)) {
+ /* Fixed format */
+
+ /* Sense Key */
+ if (valid_mask & SCST_SENSE_KEY_VALID) {
+ if (len < 3)
+ goto out;
+ if (sense[2] != key)
+ goto out;
+ }
+
+ /* ASC */
+ if (valid_mask & SCST_SENSE_ASC_VALID) {
+ if (len < 13)
+ goto out;
+ if (sense[12] != asc)
+ goto out;
+ }
+
+ /* ASCQ */
+ if (valid_mask & SCST_SENSE_ASCQ_VALID) {
+ if (len < 14)
+ goto out;
+ if (sense[13] != ascq)
+ goto out;
+ }
+ } else if ((sense[0] == 0x72) || (sense[0] == 0x73)) {
+ /* Descriptor format */
+
+ /* Sense Key */
+ if (valid_mask & SCST_SENSE_KEY_VALID) {
+ if (len < 2)
+ goto out;
+ if (sense[1] != key)
+ goto out;
+ }
+
+ /* ASC */
+ if (valid_mask & SCST_SENSE_ASC_VALID) {
+ if (len < 3)
+ goto out;
+ if (sense[2] != asc)
+ goto out;
+ }
+
+ /* ASCQ */
+ if (valid_mask & SCST_SENSE_ASCQ_VALID) {
+ if (len < 4)
+ goto out;
+ if (sense[3] != ascq)
+ goto out;
+ }
+ } else
+ goto out;
+
+ res = true;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL(scst_analyze_sense);
+
+/**
+ * scst_is_ua_sense() - determine if the sense is UA sense
+ *
+ * Returns true if the sense is valid and carrying a Unit
+ * Attention or false otherwise.
+ */
+bool scst_is_ua_sense(const uint8_t *sense, int len)
+{
+ if (SCST_SENSE_VALID(sense))
+ return scst_analyze_sense(sense, len,
+ SCST_SENSE_KEY_VALID, UNIT_ATTENTION, 0, 0);
+ else
+ return false;
+}
+EXPORT_SYMBOL(scst_is_ua_sense);
+
+bool scst_is_ua_global(const uint8_t *sense, int len)
+{
+ bool res;
+
+ /* Changing it don't forget to change scst_requeue_ua() as well!! */
+
+ if (scst_analyze_sense(sense, len, SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed)))
+ res = true;
+ else
+ res = false;
+
+ return res;
+}
+
+/**
+ * scst_check_convert_sense() - check sense type and convert it if needed
+ *
+ * Checks if sense in the sense buffer, if any, is in the correct format.
+ * If not, converts it in the correct format.
+ */
+void scst_check_convert_sense(struct scst_cmd *cmd)
+{
+ bool d_sense;
+
+ if ((cmd->sense == NULL) || (cmd->status != SAM_STAT_CHECK_CONDITION))
+ goto out;
+
+ d_sense = scst_get_cmd_dev_d_sense(cmd);
+ if (d_sense && ((cmd->sense[0] == 0x70) || (cmd->sense[0] == 0x71))) {
+ TRACE_MGMT_DBG("Converting fixed sense to descriptor (cmd %p)",
+ cmd);
+ if ((cmd->sense_valid_len < 18)) {
+ PRINT_ERROR("Sense too small to convert (%d, "
+ "type: fixed)", cmd->sense_buflen);
+ goto out;
+ }
+ cmd->sense_valid_len = scst_set_sense(cmd->sense, cmd->sense_buflen,
+ d_sense, cmd->sense[2], cmd->sense[12], cmd->sense[13]);
+ } else if (!d_sense && ((cmd->sense[0] == 0x72) ||
+ (cmd->sense[0] == 0x73))) {
+ TRACE_MGMT_DBG("Converting descriptor sense to fixed (cmd %p)",
+ cmd);
+ if ((cmd->sense_buflen < 18) || (cmd->sense_valid_len < 8)) {
+ PRINT_ERROR("Sense too small to convert (%d, "
+ "type: descryptor, valid %d)",
+ cmd->sense_buflen, cmd->sense_valid_len);
+ goto out;
+ }
+ cmd->sense_valid_len = scst_set_sense(cmd->sense,
+ cmd->sense_buflen, d_sense,
+ cmd->sense[1], cmd->sense[2], cmd->sense[3]);
+ }
+
+out:
+ return;
+}
+EXPORT_SYMBOL(scst_check_convert_sense);
+
+static int scst_set_cmd_error_sense(struct scst_cmd *cmd, uint8_t *sense,
+ unsigned int len)
+{
+ int res;
+
+ res = scst_set_cmd_error_status(cmd, SAM_STAT_CHECK_CONDITION);
+ if (res != 0)
+ goto out;
+
+ res = scst_alloc_set_sense(cmd, 1, sense, len);
+
+out:
+ return res;
+}
+
+/**
+ * scst_set_busy() - set BUSY or TASK QUEUE FULL status
+ *
+ * Sets BUSY or TASK QUEUE FULL status depending on if this session has other
+ * outstanding commands or not.
+ */
+void scst_set_busy(struct scst_cmd *cmd)
+{
+ int c = atomic_read(&cmd->sess->sess_cmd_count);
+
+ if ((c <= 1) || (cmd->sess->init_phase != SCST_SESS_IPH_READY)) {
+ scst_set_cmd_error_status(cmd, SAM_STAT_BUSY);
+ TRACE(TRACE_FLOW_CONTROL, "Sending BUSY status to initiator %s "
+ "(cmds count %d, queue_type %x, sess->init_phase %d)",
+ cmd->sess->initiator_name, c,
+ cmd->queue_type, cmd->sess->init_phase);
+ } else {
+ scst_set_cmd_error_status(cmd, SAM_STAT_TASK_SET_FULL);
+ TRACE(TRACE_FLOW_CONTROL, "Sending QUEUE_FULL status to "
+ "initiator %s (cmds count %d, queue_type %x, "
+ "sess->init_phase %d)", cmd->sess->initiator_name, c,
+ cmd->queue_type, cmd->sess->init_phase);
+ }
+ return;
+}
+EXPORT_SYMBOL(scst_set_busy);
+
+/**
+ * scst_set_initial_UA() - set initial Unit Attention
+ *
+ * Sets initial Unit Attention on all devices of the session,
+ * replacing default scst_sense_reset_UA
+ */
+void scst_set_initial_UA(struct scst_session *sess, int key, int asc, int ascq)
+{
+ int i;
+
+ TRACE_MGMT_DBG("Setting for sess %p initial UA %x/%x/%x", sess, key,
+ asc, ascq);
+
+ /* Protect sess_tgt_dev_list_hash */
+ mutex_lock(&scst_mutex);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ if (!list_empty(&tgt_dev->UA_list)) {
+ struct scst_tgt_dev_UA *ua;
+
+ ua = list_entry(tgt_dev->UA_list.next,
+ typeof(*ua), UA_list_entry);
+ if (scst_analyze_sense(ua->UA_sense_buffer,
+ ua->UA_valid_sense_len,
+ SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reset_UA))) {
+ ua->UA_valid_sense_len = scst_set_sense(
+ ua->UA_sense_buffer,
+ sizeof(ua->UA_sense_buffer),
+ tgt_dev->dev->d_sense,
+ key, asc, ascq);
+ } else
+ PRINT_ERROR("%s",
+ "The first UA isn't RESET UA");
+ } else
+ PRINT_ERROR("%s", "There's no RESET UA to "
+ "replace");
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ mutex_unlock(&scst_mutex);
+ return;
+}
+EXPORT_SYMBOL(scst_set_initial_UA);
+
+static struct scst_aen *scst_alloc_aen(struct scst_session *sess,
+ uint64_t unpacked_lun)
+{
+ struct scst_aen *aen;
+
+ aen = mempool_alloc(scst_aen_mempool, GFP_KERNEL);
+ if (aen == NULL) {
+ PRINT_ERROR("AEN memory allocation failed. Corresponding "
+ "event notification will not be performed (initiator "
+ "%s)", sess->initiator_name);
+ goto out;
+ }
+ memset(aen, 0, sizeof(*aen));
+
+ aen->sess = sess;
+ scst_sess_get(sess);
+
+ aen->lun = scst_pack_lun(unpacked_lun, sess->acg->addr_method);
+
+out:
+ return aen;
+};
+
+static void scst_free_aen(struct scst_aen *aen)
+{
+
+ scst_sess_put(aen->sess);
+ mempool_free(aen, scst_aen_mempool);
+ return;
+};
+
+/* Must be called under scst_mutex */
+void scst_gen_aen_or_ua(struct scst_tgt_dev *tgt_dev,
+ int key, int asc, int ascq)
+{
+ struct scst_tgt_template *tgtt = tgt_dev->sess->tgt->tgtt;
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl;
+
+ if (tgtt->report_aen != NULL) {
+ struct scst_aen *aen;
+ int rc;
+
+ aen = scst_alloc_aen(tgt_dev->sess, tgt_dev->lun);
+ if (aen == NULL)
+ goto queue_ua;
+
+ aen->event_fn = SCST_AEN_SCSI;
+ aen->aen_sense_len = scst_set_sense(aen->aen_sense,
+ sizeof(aen->aen_sense), tgt_dev->dev->d_sense,
+ key, asc, ascq);
+
+ TRACE_DBG("Calling target's %s report_aen(%p)",
+ tgtt->name, aen);
+ rc = tgtt->report_aen(aen);
+ TRACE_DBG("Target's %s report_aen(%p) returned %d",
+ tgtt->name, aen, rc);
+ if (rc == SCST_AEN_RES_SUCCESS)
+ goto out;
+
+ scst_free_aen(aen);
+ }
+
+queue_ua:
+ TRACE_MGMT_DBG("AEN not supported, queuing plain UA (tgt_dev %p)",
+ tgt_dev);
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ tgt_dev->dev->d_sense, key, asc, ascq);
+ scst_check_set_UA(tgt_dev, sense_buffer, sl, 0);
+
+out:
+ return;
+}
+
+/**
+ * scst_capacity_data_changed() - notify SCST about device capacity change
+ *
+ * Notifies SCST core that dev has changed its capacity. Called under no locks.
+ */
+void scst_capacity_data_changed(struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev;
+
+ if (dev->type != TYPE_DISK) {
+ TRACE_MGMT_DBG("Device type %d isn't for CAPACITY DATA "
+ "CHANGED UA", dev->type);
+ goto out;
+ }
+
+ TRACE_MGMT_DBG("CAPACITY DATA CHANGED (dev %p)", dev);
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_gen_aen_or_ua(tgt_dev,
+ SCST_LOAD_SENSE(scst_sense_capacity_data_changed));
+ }
+
+ mutex_unlock(&scst_mutex);
+
+out:
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_capacity_data_changed);
+
+static inline bool scst_is_report_luns_changed_type(int type)
+{
+ switch (type) {
+ case TYPE_DISK:
+ case TYPE_TAPE:
+ case TYPE_PRINTER:
+ case TYPE_PROCESSOR:
+ case TYPE_WORM:
+ case TYPE_ROM:
+ case TYPE_SCANNER:
+ case TYPE_MOD:
+ case TYPE_MEDIUM_CHANGER:
+ case TYPE_RAID:
+ case TYPE_ENCLOSURE:
+ return true;
+ default:
+ return false;
+ }
+}
+
+/* scst_mutex supposed to be held */
+static void scst_queue_report_luns_changed_UA(struct scst_session *sess,
+ int flags)
+{
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+ int i;
+
+ TRACE_MGMT_DBG("Queuing REPORTED LUNS DATA CHANGED UA "
+ "(sess %p)", sess);
+
+ local_bh_disable();
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ /* Lockdep triggers here a false positive.. */
+ spin_lock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ int sl;
+
+ if (!scst_is_report_luns_changed_type(
+ tgt_dev->dev->type))
+ continue;
+
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ tgt_dev->dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed));
+
+ __scst_check_set_UA(tgt_dev, sense_buffer,
+ sl, flags | SCST_SET_UA_FLAG_GLOBAL);
+ }
+ }
+
+ for (i = TGT_DEV_HASH_SIZE-1; i >= 0; i--) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry_reverse(tgt_dev,
+ shead, sess_tgt_dev_list_entry) {
+ spin_unlock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ local_bh_enable();
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+static void scst_report_luns_changed_sess(struct scst_session *sess)
+{
+ int i;
+ struct scst_tgt_template *tgtt = sess->tgt->tgtt;
+ int d_sense = 0;
+ uint64_t lun = 0;
+
+ TRACE_DBG("REPORTED LUNS DATA CHANGED (sess %p)", sess);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if (scst_is_report_luns_changed_type(
+ tgt_dev->dev->type)) {
+ lun = tgt_dev->lun;
+ d_sense = tgt_dev->dev->d_sense;
+ goto found;
+ }
+ }
+ }
+
+found:
+ if (tgtt->report_aen != NULL) {
+ struct scst_aen *aen;
+ int rc;
+
+ aen = scst_alloc_aen(sess, lun);
+ if (aen == NULL)
+ goto queue_ua;
+
+ aen->event_fn = SCST_AEN_SCSI;
+ aen->aen_sense_len = scst_set_sense(aen->aen_sense,
+ sizeof(aen->aen_sense), d_sense,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed));
+
+ TRACE_DBG("Calling target's %s report_aen(%p)",
+ tgtt->name, aen);
+ rc = tgtt->report_aen(aen);
+ TRACE_DBG("Target's %s report_aen(%p) returned %d",
+ tgtt->name, aen, rc);
+ if (rc == SCST_AEN_RES_SUCCESS)
+ goto out;
+
+ scst_free_aen(aen);
+ }
+
+queue_ua:
+ scst_queue_report_luns_changed_UA(sess, 0);
+
+out:
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_report_luns_changed(struct scst_acg *acg)
+{
+ struct scst_session *sess;
+
+ TRACE_MGMT_DBG("REPORTED LUNS DATA CHANGED (acg %s)", acg->acg_name);
+
+ list_for_each_entry(sess, &acg->acg_sess_list, acg_sess_list_entry) {
+ scst_report_luns_changed_sess(sess);
+ }
+ return;
+}
+
+/**
+ * scst_aen_done() - AEN processing done
+ *
+ * Notifies SCST that the driver has sent the AEN and it
+ * can be freed now. Don't forget to set the delivery status, if it
+ * isn't success, using scst_set_aen_delivery_status() before calling
+ * this function.
+ */
+void scst_aen_done(struct scst_aen *aen)
+{
+
+ TRACE_MGMT_DBG("AEN %p (fn %d) done (initiator %s)", aen,
+ aen->event_fn, aen->sess->initiator_name);
+
+ if (aen->delivery_status == SCST_AEN_RES_SUCCESS)
+ goto out_free;
+
+ if (aen->event_fn != SCST_AEN_SCSI)
+ goto out_free;
+
+ TRACE_MGMT_DBG("Delivery of SCSI AEN failed (initiator %s)",
+ aen->sess->initiator_name);
+
+ if (scst_analyze_sense(aen->aen_sense, aen->aen_sense_len,
+ SCST_SENSE_ALL_VALID, SCST_LOAD_SENSE(
+ scst_sense_reported_luns_data_changed))) {
+ mutex_lock(&scst_mutex);
+ scst_queue_report_luns_changed_UA(aen->sess,
+ SCST_SET_UA_FLAG_AT_HEAD);
+ mutex_unlock(&scst_mutex);
+ } else {
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+ uint64_t lun;
+
+ lun = scst_unpack_lun((uint8_t *)&aen->lun, sizeof(aen->lun));
+
+ mutex_lock(&scst_mutex);
+
+ /* tgt_dev might get dead, so we need to reseek it */
+ shead = &aen->sess->sess_tgt_dev_list_hash[HASH_VAL(lun)];
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->lun == lun) {
+ TRACE_MGMT_DBG("Requeuing failed AEN UA for "
+ "tgt_dev %p", tgt_dev);
+ scst_check_set_UA(tgt_dev, aen->aen_sense,
+ aen->aen_sense_len,
+ SCST_SET_UA_FLAG_AT_HEAD);
+ break;
+ }
+ }
+
+ mutex_unlock(&scst_mutex);
+ }
+
+out_free:
+ scst_free_aen(aen);
+ return;
+}
+EXPORT_SYMBOL(scst_aen_done);
+
+void scst_requeue_ua(struct scst_cmd *cmd)
+{
+
+ if (scst_analyze_sense(cmd->sense, cmd->sense_valid_len,
+ SCST_SENSE_ALL_VALID,
+ SCST_LOAD_SENSE(scst_sense_reported_luns_data_changed))) {
+ TRACE_MGMT_DBG("Requeuing REPORTED LUNS DATA CHANGED UA "
+ "for delivery failed cmd %p", cmd);
+ mutex_lock(&scst_mutex);
+ scst_queue_report_luns_changed_UA(cmd->sess,
+ SCST_SET_UA_FLAG_AT_HEAD);
+ mutex_unlock(&scst_mutex);
+ } else {
+ TRACE_MGMT_DBG("Requeuing UA for delivery failed cmd %p", cmd);
+ scst_check_set_UA(cmd->tgt_dev, cmd->sense,
+ cmd->sense_valid_len, SCST_SET_UA_FLAG_AT_HEAD);
+ }
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+static void scst_check_reassign_sess(struct scst_session *sess)
+{
+ struct scst_acg *acg, *old_acg;
+ struct scst_acg_dev *acg_dev;
+ int i;
+ struct list_head *shead;
+ struct scst_tgt_dev *tgt_dev;
+ bool luns_changed = false;
+ bool add_failed, something_freed, not_needed_freed = false;
+
+ TRACE_MGMT_DBG("Checking reassignment for sess %p (initiator %s)",
+ sess, sess->initiator_name);
+
+ acg = scst_find_acg(sess);
+ if (acg == sess->acg) {
+ TRACE_MGMT_DBG("No reassignment for sess %p", sess);
+ goto out;
+ }
+
+ TRACE_MGMT_DBG("sess %p will be reassigned from acg %s to acg %s",
+ sess, sess->acg->acg_name, acg->acg_name);
+
+ old_acg = sess->acg;
+ sess->acg = NULL; /* to catch implicit dependencies earlier */
+
+retry_add:
+ add_failed = false;
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ unsigned int inq_changed_ua_needed = 0;
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if ((tgt_dev->dev == acg_dev->dev) &&
+ (tgt_dev->lun == acg_dev->lun) &&
+ (tgt_dev->acg_dev->rd_only == acg_dev->rd_only)) {
+ TRACE_MGMT_DBG("sess %p: tgt_dev %p for "
+ "LUN %lld stays the same",
+ sess, tgt_dev,
+ (unsigned long long)tgt_dev->lun);
+ tgt_dev->acg_dev = acg_dev;
+ goto next;
+ } else if (tgt_dev->lun == acg_dev->lun)
+ inq_changed_ua_needed = 1;
+ }
+ }
+
+ luns_changed = true;
+
+ TRACE_MGMT_DBG("sess %p: Allocing new tgt_dev for LUN %lld",
+ sess, (unsigned long long)acg_dev->lun);
+
+ tgt_dev = scst_alloc_add_tgt_dev(sess, acg_dev);
+ if (tgt_dev == NULL) {
+ add_failed = true;
+ break;
+ }
+
+ tgt_dev->inq_changed_ua_needed = inq_changed_ua_needed ||
+ not_needed_freed;
+next:
+ continue;
+ }
+
+ something_freed = false;
+ not_needed_freed = true;
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct scst_tgt_dev *t;
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry_safe(tgt_dev, t, shead,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->acg_dev->acg != acg) {
+ TRACE_MGMT_DBG("sess %p: Deleting not used "
+ "tgt_dev %p for LUN %lld",
+ sess, tgt_dev,
+ (unsigned long long)tgt_dev->lun);
+ luns_changed = true;
+ something_freed = true;
+ scst_free_tgt_dev(tgt_dev);
+ }
+ }
+ }
+
+ if (add_failed && something_freed) {
+ TRACE_MGMT_DBG("sess %p: Retrying adding new tgt_devs", sess);
+ goto retry_add;
+ }
+
+ sess->acg = acg;
+
+ TRACE_DBG("Moving sess %p from acg %s to acg %s", sess,
+ old_acg->acg_name, acg->acg_name);
+ list_move_tail(&sess->acg_sess_list_entry, &acg->acg_sess_list);
+
+ if (luns_changed) {
+ scst_report_luns_changed_sess(sess);
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ shead = &sess->sess_tgt_dev_list_hash[i];
+
+ list_for_each_entry(tgt_dev, shead,
+ sess_tgt_dev_list_entry) {
+ if (tgt_dev->inq_changed_ua_needed) {
+ TRACE_MGMT_DBG("sess %p: Setting "
+ "INQUIRY DATA HAS CHANGED UA "
+ "(tgt_dev %p)", sess, tgt_dev);
+
+ tgt_dev->inq_changed_ua_needed = 0;
+
+ scst_gen_aen_or_ua(tgt_dev,
+ SCST_LOAD_SENSE(scst_sense_inquery_data_changed));
+ }
+ }
+ }
+ }
+
+out:
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_check_reassign_sessions(void)
+{
+ struct scst_tgt_template *tgtt;
+
+ list_for_each_entry(tgtt, &scst_template_list, scst_template_list_entry) {
+ struct scst_tgt *tgt;
+ list_for_each_entry(tgt, &tgtt->tgt_list, tgt_list_entry) {
+ struct scst_session *sess;
+ list_for_each_entry(sess, &tgt->sess_list,
+ sess_list_entry) {
+ scst_check_reassign_sess(sess);
+ }
+ }
+ }
+ return;
+}
+
+/**
+ * scst_get_cmd_abnormal_done_state() - get command's next abnormal done state
+ *
+ * Returns the next state of the SCSI target state machine in case if command's
+ * completed abnormally.
+ */
+int scst_get_cmd_abnormal_done_state(const struct scst_cmd *cmd)
+{
+ int res;
+
+ switch (cmd->state) {
+ case SCST_CMD_STATE_INIT_WAIT:
+ case SCST_CMD_STATE_INIT:
+ case SCST_CMD_STATE_PRE_PARSE:
+ case SCST_CMD_STATE_DEV_PARSE:
+ if (cmd->preprocessing_only) {
+ res = SCST_CMD_STATE_PREPROCESSING_DONE;
+ break;
+ } /* else go through */
+ case SCST_CMD_STATE_DEV_DONE:
+ if (cmd->internal)
+ res = SCST_CMD_STATE_FINISHED_INTERNAL;
+ else
+ res = SCST_CMD_STATE_PRE_XMIT_RESP;
+ break;
+
+ case SCST_CMD_STATE_PRE_DEV_DONE:
+ case SCST_CMD_STATE_MODE_SELECT_CHECKS:
+ res = SCST_CMD_STATE_DEV_DONE;
+ break;
+
+ case SCST_CMD_STATE_PRE_XMIT_RESP:
+ res = SCST_CMD_STATE_XMIT_RESP;
+ break;
+
+ case SCST_CMD_STATE_PREPROCESSING_DONE:
+ case SCST_CMD_STATE_PREPROCESSING_DONE_CALLED:
+ if (cmd->tgt_dev == NULL)
+ res = SCST_CMD_STATE_PRE_XMIT_RESP;
+ else
+ res = SCST_CMD_STATE_PRE_DEV_DONE;
+ break;
+
+ case SCST_CMD_STATE_PREPARE_SPACE:
+ if (cmd->preprocessing_only) {
+ res = SCST_CMD_STATE_PREPROCESSING_DONE;
+ break;
+ } /* else go through */
+ case SCST_CMD_STATE_RDY_TO_XFER:
+ case SCST_CMD_STATE_DATA_WAIT:
+ case SCST_CMD_STATE_TGT_PRE_EXEC:
+ case SCST_CMD_STATE_SEND_FOR_EXEC:
+ case SCST_CMD_STATE_LOCAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXEC:
+ case SCST_CMD_STATE_REAL_EXECUTING:
+ res = SCST_CMD_STATE_PRE_DEV_DONE;
+ break;
+
+ default:
+ PRINT_CRIT_ERROR("Wrong cmd state %d (cmd %p, op %x)",
+ cmd->state, cmd, cmd->cdb[0]);
+ BUG();
+ /* Invalid state to supress compiler's warning */
+ res = SCST_CMD_STATE_LAST_ACTIVE;
+ }
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_get_cmd_abnormal_done_state);
+
+/**
+ * scst_set_cmd_abnormal_done_state() - set command's next abnormal done state
+ *
+ * Sets state of the SCSI target state machine in case if command's completed
+ * abnormally.
+ */
+void scst_set_cmd_abnormal_done_state(struct scst_cmd *cmd)
+{
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ switch (cmd->state) {
+ case SCST_CMD_STATE_XMIT_RESP:
+ case SCST_CMD_STATE_FINISHED:
+ case SCST_CMD_STATE_FINISHED_INTERNAL:
+ case SCST_CMD_STATE_XMIT_WAIT:
+ PRINT_CRIT_ERROR("Wrong cmd state %d (cmd %p, op %x)",
+ cmd->state, cmd, cmd->cdb[0]);
+ BUG();
+ }
+#endif
+
+ cmd->state = scst_get_cmd_abnormal_done_state(cmd);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (((cmd->state != SCST_CMD_STATE_PRE_XMIT_RESP) &&
+ (cmd->state != SCST_CMD_STATE_PREPROCESSING_DONE)) &&
+ (cmd->tgt_dev == NULL) && !cmd->internal) {
+ PRINT_CRIT_ERROR("Wrong not inited cmd state %d (cmd %p, "
+ "op %x)", cmd->state, cmd, cmd->cdb[0]);
+ BUG();
+ }
+#endif
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_set_cmd_abnormal_done_state);
+
+/**
+ * scst_set_resp_data_len() - set response data length
+ *
+ * Sets response data length for cmd and truncates its SG vector accordingly.
+ *
+ * The cmd->resp_data_len must not be set directly, it must be set only
+ * using this function. Value of resp_data_len must be <= cmd->bufflen.
+ */
+void scst_set_resp_data_len(struct scst_cmd *cmd, int resp_data_len)
+{
+ int i, l;
+
+ scst_check_restore_sg_buff(cmd);
+ cmd->resp_data_len = resp_data_len;
+
+ if (resp_data_len == cmd->bufflen)
+ goto out;
+
+ l = 0;
+ for (i = 0; i < cmd->sg_cnt; i++) {
+ l += cmd->sg[i].length;
+ if (l >= resp_data_len) {
+ int left = resp_data_len - (l - cmd->sg[i].length);
+#ifdef CONFIG_SCST_DEBUG
+ TRACE(TRACE_SG_OP|TRACE_MEMORY, "cmd %p (tag %llu), "
+ "resp_data_len %d, i %d, cmd->sg[i].length %d, "
+ "left %d",
+ cmd, (long long unsigned int)cmd->tag,
+ resp_data_len, i,
+ cmd->sg[i].length, left);
+#endif
+ cmd->orig_sg_cnt = cmd->sg_cnt;
+ cmd->orig_sg_entry = i;
+ cmd->orig_entry_len = cmd->sg[i].length;
+ cmd->sg_cnt = (left > 0) ? i+1 : i;
+ cmd->sg[i].length = left;
+ cmd->sg_buff_modified = 1;
+ break;
+ }
+ }
+
+out:
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_set_resp_data_len);
+
+/* No locks */
+int scst_queue_retry_cmd(struct scst_cmd *cmd, int finished_cmds)
+{
+ struct scst_tgt *tgt = cmd->tgt;
+ int res = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&tgt->tgt_lock, flags);
+ tgt->retry_cmds++;
+ /*
+ * Memory barrier is needed here, because we need the exact order
+ * between the read and write between retry_cmds and finished_cmds to
+ * not miss the case when a command finished while we queuing it for
+ * retry after the finished_cmds check.
+ */
+ smp_mb();
+ TRACE_RETRY("TGT QUEUE FULL: incrementing retry_cmds %d",
+ tgt->retry_cmds);
+ if (finished_cmds != atomic_read(&tgt->finished_cmds)) {
+ /* At least one cmd finished, so try again */
+ tgt->retry_cmds--;
+ TRACE_RETRY("Some command(s) finished, direct retry "
+ "(finished_cmds=%d, tgt->finished_cmds=%d, "
+ "retry_cmds=%d)", finished_cmds,
+ atomic_read(&tgt->finished_cmds), tgt->retry_cmds);
+ res = -1;
+ goto out_unlock_tgt;
+ }
+
+ TRACE_RETRY("Adding cmd %p to retry cmd list", cmd);
+ list_add_tail(&cmd->cmd_list_entry, &tgt->retry_cmd_list);
+
+ if (!tgt->retry_timer_active) {
+ tgt->retry_timer.expires = jiffies + SCST_TGT_RETRY_TIMEOUT;
+ add_timer(&tgt->retry_timer);
+ tgt->retry_timer_active = 1;
+ }
+
+out_unlock_tgt:
+ spin_unlock_irqrestore(&tgt->tgt_lock, flags);
+ return res;
+}
+
+/* Returns 0 to continue, >0 to restart, <0 to break */
+static int scst_check_hw_pending_cmd(struct scst_cmd *cmd,
+ unsigned long cur_time, unsigned long max_time,
+ struct scst_session *sess, unsigned long *flags,
+ struct scst_tgt_template *tgtt)
+{
+ int res = -1; /* break */
+
+ TRACE_DBG("cmd %p, hw_pending %d, proc time %ld, "
+ "pending time %ld", cmd, cmd->cmd_hw_pending,
+ (long)(cur_time - cmd->start_time) / HZ,
+ (long)(cur_time - cmd->hw_pending_start) / HZ);
+
+ if (time_before_eq(cur_time, cmd->start_time + max_time)) {
+ /* Cmds are ordered, so no need to check more */
+ goto out;
+ }
+
+ if (!cmd->cmd_hw_pending) {
+ res = 0; /* continue */
+ goto out;
+ }
+
+ if (time_before(cur_time, cmd->hw_pending_start + max_time)) {
+ /* Cmds are ordered, so no need to check more */
+ goto out;
+ }
+
+ TRACE_MGMT_DBG("Cmd %p HW pending for too long %ld (state %x)",
+ cmd, (cur_time - cmd->hw_pending_start) / HZ,
+ cmd->state);
+
+ cmd->cmd_hw_pending = 0;
+
+ spin_unlock_irqrestore(&sess->sess_list_lock, *flags);
+ tgtt->on_hw_pending_cmd_timeout(cmd);
+ spin_lock_irqsave(&sess->sess_list_lock, *flags);
+
+ res = 1; /* restart */
+
+out:
+ return res;
+}
+
+static void scst_hw_pending_work_fn(struct delayed_work *work)
+{
+ struct scst_session *sess = container_of(work, struct scst_session,
+ hw_pending_work);
+ struct scst_tgt_template *tgtt = sess->tgt->tgtt;
+ struct scst_cmd *cmd;
+ unsigned long cur_time = jiffies;
+ unsigned long flags;
+ unsigned long max_time = tgtt->max_hw_pending_time * HZ;
+
+ TRACE_DBG("HW pending work (sess %p, max time %ld)", sess, max_time/HZ);
+
+ clear_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags);
+
+ spin_lock_irqsave(&sess->sess_list_lock, flags);
+
+restart:
+ list_for_each_entry(cmd, &sess->sess_cmd_list, sess_cmd_list_entry) {
+ int rc;
+
+ rc = scst_check_hw_pending_cmd(cmd, cur_time, max_time, sess,
+ &flags, tgtt);
+ if (rc < 0)
+ break;
+ else if (rc == 0)
+ continue;
+ else
+ goto restart;
+ }
+
+ if (!list_empty(&sess->sess_cmd_list)) {
+ /*
+ * For stuck cmds if there is no activity we might need to have
+ * one more run to release them, so reschedule once again.
+ */
+ TRACE_DBG("Sched HW pending work for sess %p (max time %d)",
+ sess, tgtt->max_hw_pending_time);
+ set_bit(SCST_SESS_HW_PENDING_WORK_SCHEDULED, &sess->sess_aflags);
+ schedule_delayed_work(&sess->hw_pending_work,
+ tgtt->max_hw_pending_time * HZ);
+ }
+
+ spin_unlock_irqrestore(&sess->sess_list_lock, flags);
+ return;
+}
+
+bool scst_is_relative_target_port_id_unique(uint16_t id, struct scst_tgt *t)
+{
+ bool res = true;
+ struct scst_tgt_template *tgtt;
+
+ mutex_lock(&scst_mutex);
+ list_for_each_entry(tgtt, &scst_template_list,
+ scst_template_list_entry) {
+ struct scst_tgt *tgt;
+ list_for_each_entry(tgt, &tgtt->tgt_list, tgt_list_entry) {
+ if (tgt == t)
+ continue;
+ if (id == tgt->rel_tgt_id) {
+ res = false;
+ break;
+ }
+ }
+ }
+ mutex_unlock(&scst_mutex);
+ return res;
+}
+
+int gen_relative_target_port_id(uint16_t *id)
+{
+ int res = -EOVERFLOW;
+ static unsigned long rti = SCST_MIN_REL_TGT_ID, rti_prev;
+
+ rti_prev = rti;
+ do {
+ if (scst_is_relative_target_port_id_unique(rti, NULL)) {
+ *id = (uint16_t)rti++;
+ res = 0;
+ goto out;
+ }
+ rti++;
+ if (rti > SCST_MAX_REL_TGT_ID)
+ rti = SCST_MIN_REL_TGT_ID;
+ } while (rti != rti_prev);
+
+ PRINT_ERROR("%s", "Unable to create unique relative target port id");
+
+out:
+ return res;
+}
+
+int scst_alloc_tgt(struct scst_tgt_template *tgtt, struct scst_tgt **tgt)
+{
+ struct scst_tgt *t;
+ int res = 0;
+
+ t = kzalloc(sizeof(*t), GFP_KERNEL);
+ if (t == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of tgt failed");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ INIT_LIST_HEAD(&t->sess_list);
+ init_waitqueue_head(&t->unreg_waitQ);
+ t->tgtt = tgtt;
+ t->sg_tablesize = tgtt->sg_tablesize;
+ spin_lock_init(&t->tgt_lock);
+ INIT_LIST_HEAD(&t->retry_cmd_list);
+ atomic_set(&t->finished_cmds, 0);
+ init_timer(&t->retry_timer);
+ t->retry_timer.data = (unsigned long)t;
+ t->retry_timer.function = scst_tgt_retry_timer_fn;
+
+ *tgt = t;
+
+out:
+ return res;
+}
+
+void scst_free_tgt(struct scst_tgt *tgt)
+{
+
+ if (tgt->default_acg != NULL)
+ scst_free_acg(tgt->default_acg);
+
+ kfree(tgt->tgt_name);
+
+ kfree(tgt);
+ return;
+}
+
+/* Called under scst_mutex and suspended activity */
+int scst_alloc_device(gfp_t gfp_mask, struct scst_device **out_dev)
+{
+ struct scst_device *dev;
+ int res = 0;
+
+ dev = kzalloc(sizeof(*dev), gfp_mask);
+ if (dev == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Allocation of scst_device failed");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ dev->handler = &scst_null_devtype;
+ atomic_set(&dev->dev_cmd_count, 0);
+ atomic_set(&dev->write_cmd_count, 0);
+ scst_init_mem_lim(&dev->dev_mem_lim);
+ spin_lock_init(&dev->dev_lock);
+ atomic_set(&dev->on_dev_count, 0);
+ INIT_LIST_HEAD(&dev->blocked_cmd_list);
+ INIT_LIST_HEAD(&dev->dev_tgt_dev_list);
+ INIT_LIST_HEAD(&dev->dev_acg_dev_list);
+ init_waitqueue_head(&dev->on_dev_waitQ);
+ dev->dev_double_ua_possible = 1;
+ dev->queue_alg = SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER;
+
+ scst_init_threads(&dev->dev_cmd_threads);
+
+ *out_dev = dev;
+
+out:
+ return res;
+}
+
+void scst_free_device(struct scst_device *dev)
+{
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (!list_empty(&dev->dev_tgt_dev_list) ||
+ !list_empty(&dev->dev_acg_dev_list)) {
+ PRINT_CRIT_ERROR("%s: dev_tgt_dev_list or dev_acg_dev_list "
+ "is not empty!", __func__);
+ BUG();
+ }
+#endif
+
+ scst_deinit_threads(&dev->dev_cmd_threads);
+
+ kfree(dev->virt_name);
+ kfree(dev);
+ return;
+}
+
+/**
+ * scst_init_mem_lim - initialize memory limits structure
+ *
+ * Initializes memory limits structure mem_lim according to
+ * the current system configuration. This structure should be latter used
+ * to track and limit allocated by one or more SGV pools memory.
+ */
+void scst_init_mem_lim(struct scst_mem_lim *mem_lim)
+{
+ atomic_set(&mem_lim->alloced_pages, 0);
+ mem_lim->max_allowed_pages =
+ ((uint64_t)scst_max_dev_cmd_mem << 10) >> (PAGE_SHIFT - 10);
+}
+EXPORT_SYMBOL_GPL(scst_init_mem_lim);
+
+static struct scst_acg_dev *scst_alloc_acg_dev(struct scst_acg *acg,
+ struct scst_device *dev, uint64_t lun)
+{
+ struct scst_acg_dev *res;
+
+ res = kmem_cache_zalloc(scst_acgd_cachep, GFP_KERNEL);
+ if (res == NULL) {
+ TRACE(TRACE_OUT_OF_MEM,
+ "%s", "Allocation of scst_acg_dev failed");
+ goto out;
+ }
+
+ res->dev = dev;
+ res->acg = acg;
+ res->lun = lun;
+
+out:
+ return res;
+}
+
+void scst_acg_dev_destroy(struct scst_acg_dev *acg_dev)
+{
+
+ if ((acg_dev->dev != NULL) && acg_dev->dev->dev_kobj_initialized)
+ kobject_put(&acg_dev->dev->dev_kobj);
+
+ kmem_cache_free(scst_acgd_cachep, acg_dev);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+static void scst_free_acg_dev(struct scst_acg_dev *acg_dev)
+{
+
+ TRACE_DBG("Removing acg_dev %p from acg_dev_list and dev_acg_dev_list",
+ acg_dev);
+ list_del(&acg_dev->acg_dev_list_entry);
+ list_del(&acg_dev->dev_acg_dev_list_entry);
+
+ if (acg_dev->acg_dev_kobj_initialized) {
+ kobject_del(&acg_dev->acg_dev_kobj);
+ kobject_put(&acg_dev->acg_dev_kobj);
+ } else
+ scst_acg_dev_destroy(acg_dev);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+struct scst_acg *scst_alloc_add_acg(struct scst_tgt *tgt,
+ const char *acg_name)
+{
+ struct scst_acg *acg;
+
+ acg = kzalloc(sizeof(*acg), GFP_KERNEL);
+ if (acg == NULL) {
+ PRINT_ERROR("%s", "Allocation of acg failed");
+ goto out;
+ }
+
+ INIT_LIST_HEAD(&acg->acg_dev_list);
+ INIT_LIST_HEAD(&acg->acg_sess_list);
+ INIT_LIST_HEAD(&acg->acn_list);
+ acg->acg_name = kstrdup(acg_name, GFP_KERNEL);
+ if (acg->acg_name == NULL) {
+ PRINT_ERROR("%s", "Allocation of acg_name failed");
+ goto out_free;
+ }
+
+ acg->addr_method = SCST_LUN_ADDR_METHOD_PERIPHERAL;
+
+ if (tgt != NULL) {
+ TRACE_DBG("Adding acg '%s' to device '%s' acg_list", acg_name,
+ tgt->tgt_name);
+ list_add_tail(&acg->acg_list_entry, &tgt->tgt_acg_list);
+ acg->in_tgt_acg_list = 1;
+ }
+
+out:
+ return acg;
+
+out_free:
+ kfree(acg);
+ acg = NULL;
+ goto out;
+}
+
+void scst_free_acg(struct scst_acg *acg)
+{
+
+ BUG_ON(!list_empty(&acg->acg_sess_list));
+ BUG_ON(!list_empty(&acg->acg_dev_list));
+ BUG_ON(!list_empty(&acg->acn_list));
+
+ kfree(acg->acg_name);
+ kfree(acg);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_clear_acg(struct scst_acg *acg)
+{
+ struct scst_acn *n, *nn;
+ struct scst_acg_dev *acg_dev, *acg_dev_tmp;
+
+ TRACE_DBG("Clearing acg %s from list", acg->acg_name);
+
+ BUG_ON(!list_empty(&acg->acg_sess_list));
+ if (acg->in_tgt_acg_list) {
+ TRACE_DBG("Removing acg %s from list", acg->acg_name);
+ list_del(&acg->acg_list_entry);
+ acg->in_tgt_acg_list = 0;
+ }
+ /* Freeing acg_devs */
+ list_for_each_entry_safe(acg_dev, acg_dev_tmp, &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ struct scst_tgt_dev *tgt_dev, *tt;
+ list_for_each_entry_safe(tgt_dev, tt,
+ &acg_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (tgt_dev->acg_dev == acg_dev)
+ scst_free_tgt_dev(tgt_dev);
+ }
+ scst_free_acg_dev(acg_dev);
+ }
+
+ /* Freeing names */
+ list_for_each_entry_safe(n, nn, &acg->acn_list,
+ acn_list_entry) {
+ scst_acn_sysfs_del(acg, n,
+ list_is_last(&n->acn_list_entry, &acg->acn_list));
+ }
+ INIT_LIST_HEAD(&acg->acn_list);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+void scst_destroy_acg(struct scst_acg *acg)
+{
+
+ scst_clear_acg(acg);
+ scst_free_acg(acg);
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+struct scst_acg *scst_tgt_find_acg(struct scst_tgt *tgt, const char *name)
+{
+ struct scst_acg *acg, *acg_ret = NULL;
+
+ list_for_each_entry(acg, &tgt->tgt_acg_list, acg_list_entry) {
+ if (strcmp(acg->acg_name, name) == 0) {
+ acg_ret = acg;
+ break;
+ }
+ }
+ return acg_ret;
+}
+
+/* scst_mutex supposed to be held */
+static struct scst_tgt_dev *scst_find_shared_io_tgt_dev(
+ struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_tgt_dev *res = NULL;
+ struct scst_acg *acg = tgt_dev->acg_dev->acg;
+ struct scst_tgt_dev *t;
+
+ TRACE_DBG("tgt_dev %s (acg %p, io_grouping_type %d)",
+ tgt_dev->sess->initiator_name, acg, acg->acg_io_grouping_type);
+
+ switch (acg->acg_io_grouping_type) {
+ case SCST_IO_GROUPING_AUTO:
+ if (tgt_dev->sess->initiator_name == NULL)
+ goto out;
+
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((t == tgt_dev) ||
+ (t->sess->initiator_name == NULL) ||
+ (t->active_cmd_threads == NULL))
+ continue;
+
+ TRACE_DBG("t %s", t->sess->initiator_name);
+
+ /* We check other ACG's as well */
+
+ if (strcmp(t->sess->initiator_name,
+ tgt_dev->sess->initiator_name) == 0)
+ goto found;
+ }
+ break;
+
+ case SCST_IO_GROUPING_THIS_GROUP_ONLY:
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((t == tgt_dev) || (t->active_cmd_threads == NULL))
+ continue;
+
+ TRACE_DBG("t %s (acg %p)", t->sess->initiator_name,
+ t->acg_dev->acg);
+
+ if (t->acg_dev->acg == acg)
+ goto found;
+ }
+ break;
+
+ case SCST_IO_GROUPING_NEVER:
+ goto out;
+
+ default:
+ list_for_each_entry(t, &tgt_dev->dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((t == tgt_dev) || (t->active_cmd_threads == NULL))
+ continue;
+
+ TRACE_DBG("t %s (acg %p, io_grouping_type %d)",
+ t->sess->initiator_name, t->acg_dev->acg,
+ t->acg_dev->acg->acg_io_grouping_type);
+
+ if (t->acg_dev->acg->acg_io_grouping_type ==
+ acg->acg_io_grouping_type)
+ goto found;
+ }
+ break;
+ }
+
+out:
+ return res;
+
+found:
+ if (t->active_cmd_threads == &scst_main_cmd_threads) {
+ res = t;
+ TRACE_MGMT_DBG("Going to share async IO context %p (res %p, "
+ "ini %s, dev %s, grouping type %d)",
+ t->aic_keeper->aic, res, t->sess->initiator_name,
+ t->dev->virt_name,
+ t->acg_dev->acg->acg_io_grouping_type);
+ } else {
+ res = t;
+ if (res->active_cmd_threads->io_context == NULL) {
+ TRACE_MGMT_DBG("IO context for t %p not yet "
+ "initialized, waiting...", t);
+ msleep(100);
+ barrier();
+ goto found;
+ }
+ TRACE_MGMT_DBG("Going to share IO context %p (res %p, ini %s, "
+ "dev %s, cmd_threads %p, grouping type %d)",
+ res->active_cmd_threads->io_context, res,
+ t->sess->initiator_name, t->dev->virt_name,
+ t->active_cmd_threads,
+ t->acg_dev->acg->acg_io_grouping_type);
+ }
+ goto out;
+}
+
+enum scst_dev_type_threads_pool_type scst_parse_threads_pool_type(const char *p,
+ int len)
+{
+ enum scst_dev_type_threads_pool_type res;
+
+ if (strncasecmp(p, SCST_THREADS_POOL_PER_INITIATOR_STR,
+ min_t(int, strlen(SCST_THREADS_POOL_PER_INITIATOR_STR),
+ len)) == 0)
+ res = SCST_THREADS_POOL_PER_INITIATOR;
+ else if (strncasecmp(p, SCST_THREADS_POOL_SHARED_STR,
+ min_t(int, strlen(SCST_THREADS_POOL_SHARED_STR),
+ len)) == 0)
+ res = SCST_THREADS_POOL_SHARED;
+ else {
+ PRINT_ERROR("Unknown threads pool type %s", p);
+ res = SCST_THREADS_POOL_TYPE_INVALID;
+ }
+
+ return res;
+}
+
+static int scst_ioc_keeper_thread(void *arg)
+{
+ struct scst_async_io_context_keeper *aic_keeper =
+ (struct scst_async_io_context_keeper *)arg;
+
+ TRACE_MGMT_DBG("AIC %p keeper thread %s (PID %d) started", aic_keeper,
+ current->comm, current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ BUG_ON(aic_keeper->aic != NULL);
+
+ aic_keeper->aic = get_io_context(GFP_KERNEL, -1);
+ TRACE_MGMT_DBG("Alloced new async IO context %p (aic %p)",
+ aic_keeper->aic, aic_keeper);
+
+ /* We have our own ref counting */
+ put_io_context(aic_keeper->aic);
+
+ /* We are ready */
+ wake_up_all(&aic_keeper->aic_keeper_waitQ);
+
+ wait_event_interruptible(aic_keeper->aic_keeper_waitQ,
+ kthread_should_stop());
+
+ TRACE_MGMT_DBG("AIC %p keeper thread %s (PID %d) finished", aic_keeper,
+ current->comm, current->pid);
+ return 0;
+}
+
+/* scst_mutex supposed to be held */
+int scst_tgt_dev_setup_threads(struct scst_tgt_dev *tgt_dev)
+{
+ int res = 0;
+ struct scst_device *dev = tgt_dev->dev;
+ struct scst_async_io_context_keeper *aic_keeper;
+
+ if (dev->threads_num <= 0) {
+ tgt_dev->active_cmd_threads = &scst_main_cmd_threads;
+
+ if (dev->threads_num == 0) {
+ struct scst_tgt_dev *shared_io_tgt_dev;
+
+ shared_io_tgt_dev = scst_find_shared_io_tgt_dev(tgt_dev);
+ if (shared_io_tgt_dev != NULL) {
+ aic_keeper = shared_io_tgt_dev->aic_keeper;
+ kref_get(&aic_keeper->aic_keeper_kref);
+
+ TRACE_MGMT_DBG("Linking async io context %p "
+ "for shared tgt_dev %p (dev %s)",
+ aic_keeper->aic, tgt_dev,
+ tgt_dev->dev->virt_name);
+ } else {
+ /* Create new context */
+ aic_keeper = kzalloc(sizeof(*aic_keeper),
+ GFP_KERNEL);
+ if (aic_keeper == NULL) {
+ PRINT_ERROR("Unable to alloc aic_keeper "
+ "(size %zd)", sizeof(*aic_keeper));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ kref_init(&aic_keeper->aic_keeper_kref);
+ init_waitqueue_head(&aic_keeper->aic_keeper_waitQ);
+
+ aic_keeper->aic_keeper_thr =
+ kthread_run(scst_ioc_keeper_thread,
+ aic_keeper, "aic_keeper");
+ if (IS_ERR(aic_keeper->aic_keeper_thr)) {
+ PRINT_ERROR("Error running ioc_keeper "
+ "thread (tgt_dev %p)", tgt_dev);
+ res = PTR_ERR(aic_keeper->aic_keeper_thr);
+ goto out_free_keeper;
+ }
+
+ wait_event(aic_keeper->aic_keeper_waitQ,
+ aic_keeper->aic != NULL);
+
+ TRACE_MGMT_DBG("Created async io context %p "
+ "for not shared tgt_dev %p (dev %s)",
+ aic_keeper->aic, tgt_dev,
+ tgt_dev->dev->virt_name);
+ }
+
+ tgt_dev->async_io_context = aic_keeper->aic;
+ tgt_dev->aic_keeper = aic_keeper;
+ }
+
+ res = scst_add_threads(tgt_dev->active_cmd_threads, NULL, NULL,
+ tgt_dev->sess->tgt->tgtt->threads_num);
+ goto out;
+ }
+
+ switch (dev->threads_pool_type) {
+ case SCST_THREADS_POOL_PER_INITIATOR:
+ {
+ struct scst_tgt_dev *shared_io_tgt_dev;
+
+ scst_init_threads(&tgt_dev->tgt_dev_cmd_threads);
+
+ tgt_dev->active_cmd_threads = &tgt_dev->tgt_dev_cmd_threads;
+
+ shared_io_tgt_dev = scst_find_shared_io_tgt_dev(tgt_dev);
+ if (shared_io_tgt_dev != NULL) {
+ TRACE_MGMT_DBG("Linking io context %p for "
+ "shared tgt_dev %p (cmd_threads %p)",
+ shared_io_tgt_dev->active_cmd_threads->io_context,
+ tgt_dev, tgt_dev->active_cmd_threads);
+ /* It's ref counted via threads */
+ tgt_dev->active_cmd_threads->io_context =
+ shared_io_tgt_dev->active_cmd_threads->io_context;
+ }
+
+ res = scst_add_threads(tgt_dev->active_cmd_threads, NULL,
+ tgt_dev,
+ dev->threads_num + tgt_dev->sess->tgt->tgtt->threads_num);
+ if (res != 0) {
+ /* Let's clear here, because no threads could be run */
+ tgt_dev->active_cmd_threads->io_context = NULL;
+ }
+ break;
+ }
+ case SCST_THREADS_POOL_SHARED:
+ tgt_dev->active_cmd_threads = &dev->dev_cmd_threads;
+
+ res = scst_add_threads(tgt_dev->active_cmd_threads, dev, NULL,
+ tgt_dev->sess->tgt->tgtt->threads_num);
+ break;
+ default:
+ PRINT_CRIT_ERROR("Unknown threads pool type %d (dev %s)",
+ dev->threads_pool_type, dev->virt_name);
+ BUG();
+ break;
+ }
+
+out:
+ if (res == 0)
+ tm_dbg_init_tgt_dev(tgt_dev);
+ return res;
+
+out_free_keeper:
+ kfree(aic_keeper);
+ goto out;
+}
+
+static void scst_aic_keeper_release(struct kref *kref)
+{
+ struct scst_async_io_context_keeper *aic_keeper;
+
+ aic_keeper = container_of(kref, struct scst_async_io_context_keeper,
+ aic_keeper_kref);
+
+ kthread_stop(aic_keeper->aic_keeper_thr);
+
+ kfree(aic_keeper);
+ return;
+}
+
+/* scst_mutex supposed to be held */
+void scst_tgt_dev_stop_threads(struct scst_tgt_dev *tgt_dev)
+{
+
+ if (tgt_dev->active_cmd_threads == &scst_main_cmd_threads) {
+ /* Global async threads */
+ kref_put(&tgt_dev->aic_keeper->aic_keeper_kref,
+ scst_aic_keeper_release);
+ tgt_dev->async_io_context = NULL;
+ tgt_dev->aic_keeper = NULL;
+ } else if (tgt_dev->active_cmd_threads == &tgt_dev->dev->dev_cmd_threads) {
+ /* Per device shared threads */
+ scst_del_threads(tgt_dev->active_cmd_threads,
+ tgt_dev->sess->tgt->tgtt->threads_num);
+ } else if (tgt_dev->active_cmd_threads == &tgt_dev->tgt_dev_cmd_threads) {
+ /* Per tgt_dev threads */
+ scst_del_threads(tgt_dev->active_cmd_threads, -1);
+ scst_deinit_threads(&tgt_dev->tgt_dev_cmd_threads);
+ } /* else no threads (not yet initialized, e.g.) */
+
+ tm_dbg_deinit_tgt_dev(tgt_dev);
+ tgt_dev->active_cmd_threads = NULL;
+ return;
+}
+
+/*
+ * scst_mutex supposed to be held, there must not be parallel activity in this
+ * session.
+ */
+static struct scst_tgt_dev *scst_alloc_add_tgt_dev(struct scst_session *sess,
+ struct scst_acg_dev *acg_dev)
+{
+ int ini_sg, ini_unchecked_isa_dma, ini_use_clustering;
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_device *dev = acg_dev->dev;
+ struct list_head *sess_tgt_dev_list_head;
+ int rc, i, sl;
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+
+ tgt_dev = kmem_cache_zalloc(scst_tgtd_cachep, GFP_KERNEL);
+ if (tgt_dev == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Allocation of scst_tgt_dev failed");
+ goto out;
+ }
+
+ tgt_dev->dev = dev;
+ tgt_dev->lun = acg_dev->lun;
+ tgt_dev->acg_dev = acg_dev;
+ tgt_dev->sess = sess;
+ atomic_set(&tgt_dev->tgt_dev_cmd_count, 0);
+
+ scst_sgv_pool_use_norm(tgt_dev);
+
+ if (dev->scsi_dev != NULL) {
+ ini_sg = dev->scsi_dev->host->sg_tablesize;
+ ini_unchecked_isa_dma = dev->scsi_dev->host->unchecked_isa_dma;
+ ini_use_clustering = (dev->scsi_dev->host->use_clustering ==
+ ENABLE_CLUSTERING);
+ } else {
+ ini_sg = (1 << 15) /* infinite */;
+ ini_unchecked_isa_dma = 0;
+ ini_use_clustering = 0;
+ }
+ tgt_dev->max_sg_cnt = min(ini_sg, sess->tgt->sg_tablesize);
+
+ if ((sess->tgt->tgtt->use_clustering || ini_use_clustering) &&
+ !sess->tgt->tgtt->no_clustering)
+ scst_sgv_pool_use_norm_clust(tgt_dev);
+
+ if (sess->tgt->tgtt->unchecked_isa_dma || ini_unchecked_isa_dma)
+ scst_sgv_pool_use_dma(tgt_dev);
+
+ TRACE_MGMT_DBG("Device %s on SCST lun=%lld",
+ dev->virt_name, (long long unsigned int)tgt_dev->lun);
+
+ spin_lock_init(&tgt_dev->tgt_dev_lock);
+ INIT_LIST_HEAD(&tgt_dev->UA_list);
+ spin_lock_init(&tgt_dev->thr_data_lock);
+ INIT_LIST_HEAD(&tgt_dev->thr_data_list);
+ spin_lock_init(&tgt_dev->sn_lock);
+ INIT_LIST_HEAD(&tgt_dev->deferred_cmd_list);
+ INIT_LIST_HEAD(&tgt_dev->skipped_sn_list);
+ tgt_dev->curr_sn = (typeof(tgt_dev->curr_sn))(-300);
+ tgt_dev->expected_sn = tgt_dev->curr_sn + 1;
+ tgt_dev->num_free_sn_slots = ARRAY_SIZE(tgt_dev->sn_slots)-1;
+ tgt_dev->cur_sn_slot = &tgt_dev->sn_slots[0];
+ for (i = 0; i < (int)ARRAY_SIZE(tgt_dev->sn_slots); i++)
+ atomic_set(&tgt_dev->sn_slots[i], 0);
+
+ if (dev->handler->parse_atomic &&
+ (sess->tgt->tgtt->preprocessing_done == NULL)) {
+ if (sess->tgt->tgtt->rdy_to_xfer_atomic)
+ __set_bit(SCST_TGT_DEV_AFTER_INIT_WR_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ if (dev->handler->exec_atomic)
+ __set_bit(SCST_TGT_DEV_AFTER_INIT_OTH_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ }
+ if (dev->handler->exec_atomic) {
+ if (sess->tgt->tgtt->rdy_to_xfer_atomic)
+ __set_bit(SCST_TGT_DEV_AFTER_RESTART_WR_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ __set_bit(SCST_TGT_DEV_AFTER_RESTART_OTH_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ __set_bit(SCST_TGT_DEV_AFTER_RX_DATA_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ }
+ if (dev->handler->dev_done_atomic &&
+ sess->tgt->tgtt->xmit_response_atomic) {
+ __set_bit(SCST_TGT_DEV_AFTER_EXEC_ATOMIC,
+ &tgt_dev->tgt_dev_flags);
+ }
+
+ sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ dev->d_sense, SCST_LOAD_SENSE(scst_sense_reset_UA));
+ scst_alloc_set_UA(tgt_dev, sense_buffer, sl, 0);
+
+ rc = scst_tgt_dev_setup_threads(tgt_dev);
+ if (rc != 0)
+ goto out_free;
+
+ if (dev->handler && dev->handler->attach_tgt) {
+ TRACE_DBG("Calling dev handler's attach_tgt(%p)", tgt_dev);
+ rc = dev->handler->attach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's attach_tgt() returned");
+ if (rc != 0) {
+ PRINT_ERROR("Device handler's %s attach_tgt() "
+ "failed: %d", dev->handler->name, rc);
+ goto out_stop_threads;
+ }
+ }
+
+ spin_lock_bh(&dev->dev_lock);
+ list_add_tail(&tgt_dev->dev_tgt_dev_list_entry, &dev->dev_tgt_dev_list);
+ if (dev->dev_reserved)
+ __set_bit(SCST_TGT_DEV_RESERVED, &tgt_dev->tgt_dev_flags);
+ spin_unlock_bh(&dev->dev_lock);
+
+ sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[HASH_VAL(tgt_dev->lun)];
+ list_add_tail(&tgt_dev->sess_tgt_dev_list_entry,
+ sess_tgt_dev_list_head);
+
+out:
+ return tgt_dev;
+
+out_stop_threads:
+ scst_tgt_dev_stop_threads(tgt_dev);
+
+out_free:
+ scst_free_all_UA(tgt_dev);
+
+ kmem_cache_free(scst_tgtd_cachep, tgt_dev);
+ tgt_dev = NULL;
+ goto out;
+}
+
+/* No locks supposed to be held, scst_mutex - held */
+void scst_nexus_loss(struct scst_tgt_dev *tgt_dev, bool queue_UA)
+{
+
+ scst_clear_reservation(tgt_dev);
+
+ /* With activity suspended the lock isn't needed, but let's be safe */
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ scst_free_all_UA(tgt_dev);
+ memset(tgt_dev->tgt_dev_sense, 0, sizeof(tgt_dev->tgt_dev_sense));
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+
+ if (queue_UA) {
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ tgt_dev->dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_nexus_loss_UA));
+ scst_check_set_UA(tgt_dev, sense_buffer, sl, 0);
+ }
+ return;
+}
+
+/*
+ * scst_mutex supposed to be held, there must not be parallel activity in this
+ * session.
+ */
+static void scst_free_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_device *dev = tgt_dev->dev;
+
+ spin_lock_bh(&dev->dev_lock);
+ list_del(&tgt_dev->dev_tgt_dev_list_entry);
+ spin_unlock_bh(&dev->dev_lock);
+
+ list_del(&tgt_dev->sess_tgt_dev_list_entry);
+
+ scst_clear_reservation(tgt_dev);
+ scst_free_all_UA(tgt_dev);
+
+ if (dev->handler && dev->handler->detach_tgt) {
+ TRACE_DBG("Calling dev handler's detach_tgt(%p)",
+ tgt_dev);
+ dev->handler->detach_tgt(tgt_dev);
+ TRACE_DBG("%s", "Dev handler's detach_tgt() returned");
+ }
+
+ scst_tgt_dev_stop_threads(tgt_dev);
+
+ BUG_ON(!list_empty(&tgt_dev->thr_data_list));
+
+ kmem_cache_free(scst_tgtd_cachep, tgt_dev);
+ return;
+}
+
+/* scst_mutex supposed to be held */
+int scst_sess_alloc_tgt_devs(struct scst_session *sess)
+{
+ int res = 0;
+ struct scst_acg_dev *acg_dev;
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(acg_dev, &sess->acg->acg_dev_list,
+ acg_dev_list_entry) {
+ tgt_dev = scst_alloc_add_tgt_dev(sess, acg_dev);
+ if (tgt_dev == NULL) {
+ res = -ENOMEM;
+ goto out_free;
+ }
+ }
+
+out:
+ return res;
+
+out_free:
+ scst_sess_free_tgt_devs(sess);
+ goto out;
+}
+
+/*
+ * scst_mutex supposed to be held, there must not be parallel activity in this
+ * session.
+ */
+void scst_sess_free_tgt_devs(struct scst_session *sess)
+{
+ int i;
+ struct scst_tgt_dev *tgt_dev, *t;
+
+ /* The session is going down, no users, so no locks */
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ list_for_each_entry_safe(tgt_dev, t, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ scst_free_tgt_dev(tgt_dev);
+ }
+ INIT_LIST_HEAD(sess_tgt_dev_list_head);
+ }
+ return;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_acg_add_dev(struct scst_acg *acg, struct scst_device *dev,
+ uint64_t lun, int read_only, bool gen_scst_report_luns_changed)
+{
+ int res = 0;
+ struct scst_acg_dev *acg_dev;
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_session *sess;
+ LIST_HEAD(tmp_tgt_dev_list);
+
+ INIT_LIST_HEAD(&tmp_tgt_dev_list);
+
+ acg_dev = scst_alloc_acg_dev(acg, dev, lun);
+ if (acg_dev == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ acg_dev->rd_only = read_only;
+
+ TRACE_DBG("Adding acg_dev %p to acg_dev_list and dev_acg_dev_list",
+ acg_dev);
+ list_add_tail(&acg_dev->acg_dev_list_entry, &acg->acg_dev_list);
+ list_add_tail(&acg_dev->dev_acg_dev_list_entry, &dev->dev_acg_dev_list);
+
+ list_for_each_entry(sess, &acg->acg_sess_list, acg_sess_list_entry) {
+ tgt_dev = scst_alloc_add_tgt_dev(sess, acg_dev);
+ if (tgt_dev == NULL) {
+ res = -ENOMEM;
+ goto out_free;
+ }
+ list_add_tail(&tgt_dev->extra_tgt_dev_list_entry,
+ &tmp_tgt_dev_list);
+ }
+
+ if (gen_scst_report_luns_changed)
+ scst_report_luns_changed(acg);
+
+ PRINT_INFO("Added device %s to group %s (LUN %lld, "
+ "rd_only %d)", dev->virt_name, acg->acg_name,
+ (long long unsigned int)lun, read_only);
+
+out:
+ return res;
+
+out_free:
+ list_for_each_entry(tgt_dev, &tmp_tgt_dev_list,
+ extra_tgt_dev_list_entry) {
+ scst_free_tgt_dev(tgt_dev);
+ }
+ scst_free_acg_dev(acg_dev);
+ goto out;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_acg_remove_dev(struct scst_acg *acg, struct scst_device *dev,
+ bool gen_scst_report_luns_changed)
+{
+ int res = 0;
+ struct scst_acg_dev *acg_dev = NULL, *a;
+ struct scst_tgt_dev *tgt_dev, *tt;
+
+ list_for_each_entry(a, &acg->acg_dev_list, acg_dev_list_entry) {
+ if (a->dev == dev) {
+ acg_dev = a;
+ break;
+ }
+ }
+
+ if (acg_dev == NULL) {
+ PRINT_ERROR("Device is not found in group %s", acg->acg_name);
+ res = -EINVAL;
+ goto out;
+ }
+
+ list_for_each_entry_safe(tgt_dev, tt, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (tgt_dev->acg_dev == acg_dev)
+ scst_free_tgt_dev(tgt_dev);
+ }
+ scst_free_acg_dev(acg_dev);
+
+ if (gen_scst_report_luns_changed)
+ scst_report_luns_changed(acg);
+
+ PRINT_INFO("Removed device %s from group %s", dev->virt_name,
+ acg->acg_name);
+
+out:
+ return res;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+int scst_acg_add_name(struct scst_acg *acg, const char *name)
+{
+ int res = 0;
+ struct scst_acn *n;
+ int len;
+ char *nm;
+
+ list_for_each_entry(n, &acg->acn_list, acn_list_entry) {
+ if (strcmp(n->name, name) == 0) {
+ PRINT_ERROR("Name %s already exists in group %s",
+ name, acg->acg_name);
+ res = -EEXIST;
+ goto out;
+ }
+ }
+
+ n = kmalloc(sizeof(*n), GFP_KERNEL);
+ if (n == NULL) {
+ PRINT_ERROR("%s", "Unable to allocate scst_acn");
+ res = -ENOMEM;
+ goto out;
+ }
+
+ len = strlen(name);
+ nm = kmalloc(len + 1, GFP_KERNEL);
+ if (nm == NULL) {
+ PRINT_ERROR("%s", "Unable to allocate scst_acn->name");
+ res = -ENOMEM;
+ goto out_free;
+ }
+
+ strcpy(nm, name);
+ n->name = nm;
+
+ res = scst_create_acn_sysfs(acg, n);
+ if (res != 0)
+ goto out_free_nm;
+
+ list_add_tail(&n->acn_list_entry, &acg->acn_list);
+
+out:
+ if (res == 0) {
+ PRINT_INFO("Added name '%s' to group '%s'", name, acg->acg_name);
+ scst_check_reassign_sessions();
+ }
+ return res;
+
+out_free_nm:
+ kfree(nm);
+
+out_free:
+ kfree(n);
+ goto out;
+}
+
+/* The activity supposed to be suspended and scst_mutex held */
+struct scst_acn *scst_acg_find_name(struct scst_acg *acg, const char *name)
+{
+ struct scst_acn *n;
+
+ TRACE_DBG("Trying to find name '%s'", name);
+
+ list_for_each_entry(n, &acg->acn_list, acn_list_entry) {
+ if (strcmp(n->name, name) == 0) {
+ TRACE_DBG("%s", "Found");
+ goto out;
+ }
+ }
+ n = NULL;
+out:
+ return n;
+}
+
+/* scst_mutex supposed to be held */
+void scst_acg_remove_acn(struct scst_acn *acn)
+{
+
+ list_del(&acn->acn_list_entry);
+ kfree(acn->name);
+ kfree(acn);
+ return;
+}
+
+static struct scst_cmd *scst_create_prepare_internal_cmd(
+ struct scst_cmd *orig_cmd, int bufsize)
+{
+ struct scst_cmd *res;
+ gfp_t gfp_mask = scst_cmd_atomic(orig_cmd) ? GFP_ATOMIC : GFP_KERNEL;
+
+ res = scst_alloc_cmd(gfp_mask);
+ if (res == NULL)
+ goto out;
+
+ res->cmd_threads = orig_cmd->cmd_threads;
+ res->sess = orig_cmd->sess;
+ res->atomic = scst_cmd_atomic(orig_cmd);
+ res->internal = 1;
+ res->tgtt = orig_cmd->tgtt;
+ res->tgt = orig_cmd->tgt;
+ res->dev = orig_cmd->dev;
+ res->tgt_dev = orig_cmd->tgt_dev;
+ res->lun = orig_cmd->lun;
+ res->queue_type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ res->data_direction = SCST_DATA_UNKNOWN;
+ res->orig_cmd = orig_cmd;
+ res->bufflen = bufsize;
+
+ scst_sess_get(res->sess);
+ if (res->tgt_dev != NULL)
+ __scst_get(0);
+
+ res->state = SCST_CMD_STATE_PRE_PARSE;
+
+out:
+ return res;
+}
+
+int scst_prepare_request_sense(struct scst_cmd *orig_cmd)
+{
+ int res = 0;
+ static const uint8_t request_sense[6] = {
+ REQUEST_SENSE, 0, 0, 0, SCST_SENSE_BUFFERSIZE, 0
+ };
+ struct scst_cmd *rs_cmd;
+
+ if (orig_cmd->sense != NULL) {
+ TRACE_MEM("Releasing sense %p (orig_cmd %p)",
+ orig_cmd->sense, orig_cmd);
+ mempool_free(orig_cmd->sense, scst_sense_mempool);
+ orig_cmd->sense = NULL;
+ }
+
+ rs_cmd = scst_create_prepare_internal_cmd(orig_cmd,
+ SCST_SENSE_BUFFERSIZE);
+ if (rs_cmd == NULL)
+ goto out_error;
+
+ memcpy(rs_cmd->cdb, request_sense, sizeof(request_sense));
+ rs_cmd->cdb[1] |= scst_get_cmd_dev_d_sense(orig_cmd);
+ rs_cmd->cdb_len = sizeof(request_sense);
+ rs_cmd->data_direction = SCST_DATA_READ;
+ rs_cmd->expected_data_direction = rs_cmd->data_direction;
+ rs_cmd->expected_transfer_len = SCST_SENSE_BUFFERSIZE;
+ rs_cmd->expected_values_set = 1;
+
+ TRACE_MGMT_DBG("Adding REQUEST SENSE cmd %p to head of active "
+ "cmd list", rs_cmd);
+ spin_lock_irq(&rs_cmd->cmd_threads->cmd_list_lock);
+ list_add(&rs_cmd->cmd_list_entry, &rs_cmd->cmd_threads->active_cmd_list);
+ wake_up(&rs_cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&rs_cmd->cmd_threads->cmd_list_lock);
+
+out:
+ return res;
+
+out_error:
+ res = -1;
+ goto out;
+}
+
+static void scst_complete_request_sense(struct scst_cmd *req_cmd)
+{
+ struct scst_cmd *orig_cmd = req_cmd->orig_cmd;
+ uint8_t *buf;
+ int len;
+
+ BUG_ON(orig_cmd == NULL);
+
+ len = scst_get_buf_first(req_cmd, &buf);
+
+ if (scsi_status_is_good(req_cmd->status) && (len > 0) &&
+ SCST_SENSE_VALID(buf) && (!SCST_NO_SENSE(buf))) {
+ PRINT_BUFF_FLAG(TRACE_SCSI, "REQUEST SENSE returned",
+ buf, len);
+ scst_alloc_set_sense(orig_cmd, scst_cmd_atomic(req_cmd), buf,
+ len);
+ } else {
+ PRINT_ERROR("%s", "Unable to get the sense via "
+ "REQUEST SENSE, returning HARDWARE ERROR");
+ scst_set_cmd_error(orig_cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ }
+
+ if (len > 0)
+ scst_put_buf(req_cmd, buf);
+
+ TRACE_MGMT_DBG("Adding orig cmd %p to head of active "
+ "cmd list", orig_cmd);
+ spin_lock_irq(&orig_cmd->cmd_threads->cmd_list_lock);
+ list_add(&orig_cmd->cmd_list_entry, &orig_cmd->cmd_threads->active_cmd_list);
+ wake_up(&orig_cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&orig_cmd->cmd_threads->cmd_list_lock);
+ return;
+}
+
+int scst_finish_internal_cmd(struct scst_cmd *cmd)
+{
+ int res;
+
+ BUG_ON(!cmd->internal);
+
+ if (cmd->cdb[0] == REQUEST_SENSE)
+ scst_complete_request_sense(cmd);
+
+ __scst_cmd_put(cmd);
+
+ res = SCST_CMD_STATE_RES_CONT_NEXT;
+ return res;
+}
+
+static void scst_send_release(struct scst_device *dev)
+{
+ struct scsi_device *scsi_dev;
+ unsigned char cdb[6];
+ uint8_t sense[SCSI_SENSE_BUFFERSIZE];
+ int rc, i;
+
+ if (dev->scsi_dev == NULL)
+ goto out;
+
+ scsi_dev = dev->scsi_dev;
+
+ for (i = 0; i < 5; i++) {
+ memset(cdb, 0, sizeof(cdb));
+ cdb[0] = RELEASE;
+ cdb[1] = (scsi_dev->scsi_level <= SCSI_2) ?
+ ((scsi_dev->lun << 5) & 0xe0) : 0;
+
+ memset(sense, 0, sizeof(sense));
+
+ TRACE(TRACE_DEBUG | TRACE_SCSI, "%s", "Sending RELEASE req to "
+ "SCSI mid-level");
+ rc = scsi_execute(scsi_dev, cdb, SCST_DATA_NONE, NULL, 0,
+ sense, 15, 0, 0
+ , NULL
+ );
+ TRACE_DBG("MODE_SENSE done: %x", rc);
+
+ if (scsi_status_is_good(rc)) {
+ break;
+ } else {
+ PRINT_ERROR("RELEASE failed: %d", rc);
+ PRINT_BUFFER("RELEASE sense", sense, sizeof(sense));
+ scst_check_internal_sense(dev, rc, sense,
+ sizeof(sense));
+ }
+ }
+
+out:
+ return;
+}
+
+/* scst_mutex supposed to be held */
+static void scst_clear_reservation(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_device *dev = tgt_dev->dev;
+ int release = 0;
+
+ spin_lock_bh(&dev->dev_lock);
+ if (dev->dev_reserved &&
+ !test_bit(SCST_TGT_DEV_RESERVED, &tgt_dev->tgt_dev_flags)) {
+ /* This is one who holds the reservation */
+ struct scst_tgt_dev *tgt_dev_tmp;
+ list_for_each_entry(tgt_dev_tmp, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev_tmp->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ release = 1;
+ }
+ spin_unlock_bh(&dev->dev_lock);
+
+ if (release)
+ scst_send_release(dev);
+ return;
+}
+
+struct scst_session *scst_alloc_session(struct scst_tgt *tgt, gfp_t gfp_mask,
+ const char *initiator_name)
+{
+ struct scst_session *sess;
+ int i;
+ int len;
+ char *nm;
+
+ sess = kmem_cache_zalloc(scst_sess_cachep, gfp_mask);
+ if (sess == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s",
+ "Allocation of scst_session failed");
+ goto out;
+ }
+
+ sess->init_phase = SCST_SESS_IPH_INITING;
+ sess->shut_phase = SCST_SESS_SPH_READY;
+ atomic_set(&sess->refcnt, 0);
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ INIT_LIST_HEAD(sess_tgt_dev_list_head);
+ }
+ spin_lock_init(&sess->sess_list_lock);
+ INIT_LIST_HEAD(&sess->sess_cmd_list);
+ sess->tgt = tgt;
+ INIT_LIST_HEAD(&sess->init_deferred_cmd_list);
+ INIT_LIST_HEAD(&sess->init_deferred_mcmd_list);
+ INIT_DELAYED_WORK(&sess->hw_pending_work,
+ (void (*)(struct work_struct *))scst_hw_pending_work_fn);
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+ spin_lock_init(&sess->lat_lock);
+#endif
+
+ len = strlen(initiator_name);
+ nm = kmalloc(len + 1, gfp_mask);
+ if (nm == NULL) {
+ PRINT_ERROR("%s", "Unable to allocate sess->initiator_name");
+ goto out_free;
+ }
+
+ strcpy(nm, initiator_name);
+ sess->initiator_name = nm;
+
+out:
+ return sess;
+
+out_free:
+ kmem_cache_free(scst_sess_cachep, sess);
+ sess = NULL;
+ goto out;
+}
+
+void scst_free_session(struct scst_session *sess)
+{
+
+ mutex_lock(&scst_mutex);
+
+ TRACE_DBG("Removing sess %p from the list", sess);
+ list_del(&sess->sess_list_entry);
+ TRACE_DBG("Removing session %p from acg %s", sess, sess->acg->acg_name);
+ list_del(&sess->acg_sess_list_entry);
+
+ scst_sess_free_tgt_devs(sess);
+
+ /* Called under lock to protect from too early tgt release */
+ wake_up_all(&sess->tgt->unreg_waitQ);
+
+ mutex_unlock(&scst_mutex);
+
+ scst_sess_sysfs_put(sess); /* must not be called under scst_mutex */
+ return;
+}
+
+void scst_release_session(struct scst_session *sess)
+{
+
+ kfree(sess->initiator_name);
+ kmem_cache_free(scst_sess_cachep, sess);
+ return;
+}
+
+void scst_free_session_callback(struct scst_session *sess)
+{
+ struct completion *c;
+
+ TRACE_DBG("Freeing session %p", sess);
+
+ cancel_delayed_work_sync(&sess->hw_pending_work);
+
+ c = sess->shutdown_compl;
+
+ if (sess->unreg_done_fn) {
+ TRACE_DBG("Calling unreg_done_fn(%p)", sess);
+ sess->unreg_done_fn(sess);
+ TRACE_DBG("%s", "unreg_done_fn() returned");
+ }
+ scst_free_session(sess);
+
+ if (c)
+ complete_all(c);
+ return;
+}
+
+void scst_sched_session_free(struct scst_session *sess)
+{
+ unsigned long flags;
+
+ if (sess->shut_phase != SCST_SESS_SPH_SHUTDOWN) {
+ PRINT_CRIT_ERROR("session %p is going to shutdown with unknown "
+ "shut phase %lx", sess, sess->shut_phase);
+ BUG();
+ }
+
+ spin_lock_irqsave(&scst_mgmt_lock, flags);
+ TRACE_DBG("Adding sess %p to scst_sess_shut_list", sess);
+ list_add_tail(&sess->sess_shut_list_entry, &scst_sess_shut_list);
+ spin_unlock_irqrestore(&scst_mgmt_lock, flags);
+
+ wake_up(&scst_mgmt_waitQ);
+ return;
+}
+
+/**
+ * scst_cmd_get() - increase command's reference counter
+ */
+void scst_cmd_get(struct scst_cmd *cmd)
+{
+ __scst_cmd_get(cmd);
+}
+EXPORT_SYMBOL_GPL(scst_cmd_get);
+
+/**
+ * scst_cmd_put() - decrease command's reference counter
+ */
+void scst_cmd_put(struct scst_cmd *cmd)
+{
+ __scst_cmd_put(cmd);
+}
+EXPORT_SYMBOL_GPL(scst_cmd_put);
+
+struct scst_cmd *scst_alloc_cmd(gfp_t gfp_mask)
+{
+ struct scst_cmd *cmd;
+
+ cmd = kmem_cache_zalloc(scst_cmd_cachep, gfp_mask);
+ if (cmd == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of scst_cmd failed");
+ goto out;
+ }
+
+ cmd->state = SCST_CMD_STATE_INIT_WAIT;
+ cmd->start_time = jiffies;
+ atomic_set(&cmd->cmd_ref, 1);
+ cmd->cmd_threads = &scst_main_cmd_threads;
+ INIT_LIST_HEAD(&cmd->mgmt_cmd_list);
+ cmd->queue_type = SCST_CMD_QUEUE_SIMPLE;
+ cmd->timeout = SCST_DEFAULT_TIMEOUT;
+ cmd->retries = 0;
+ cmd->data_len = -1;
+ cmd->is_send_status = 1;
+ cmd->resp_data_len = -1;
+
+ cmd->dbl_ua_orig_data_direction = SCST_DATA_UNKNOWN;
+ cmd->dbl_ua_orig_resp_data_len = -1;
+
+out:
+ return cmd;
+}
+
+static void scst_destroy_put_cmd(struct scst_cmd *cmd)
+{
+ scst_sess_put(cmd->sess);
+
+ /*
+ * At this point tgt_dev can be dead, but the pointer remains non-NULL
+ */
+ if (likely(cmd->tgt_dev != NULL))
+ __scst_put();
+
+ scst_destroy_cmd(cmd);
+ return;
+}
+
+/* No locks supposed to be held */
+void scst_free_cmd(struct scst_cmd *cmd)
+{
+ int destroy = 1;
+
+ TRACE_DBG("Freeing cmd %p (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags))) {
+ TRACE_MGMT_DBG("Freeing aborted cmd %p (scst_cmd_count %d)",
+ cmd, atomic_read(&scst_cmd_count));
+ }
+
+ BUG_ON(cmd->inc_blocking || cmd->needs_unblocking ||
+ cmd->dec_on_dev_needed);
+
+ /*
+ * Target driver can already free sg buffer before calling
+ * scst_tgt_cmd_done(). E.g., scst_local has to do that.
+ */
+ if (!cmd->tgt_data_buf_alloced)
+ scst_check_restore_sg_buff(cmd);
+
+ if ((cmd->tgtt->on_free_cmd != NULL) && likely(!cmd->internal)) {
+ TRACE_DBG("Calling target's on_free_cmd(%p)", cmd);
+ scst_set_cur_start(cmd);
+ cmd->tgtt->on_free_cmd(cmd);
+ scst_set_tgt_on_free_time(cmd);
+ TRACE_DBG("%s", "Target's on_free_cmd() returned");
+ }
+
+ if (likely(cmd->dev != NULL)) {
+ struct scst_dev_type *handler = cmd->dev->handler;
+ if (handler->on_free_cmd != NULL) {
+ TRACE_DBG("Calling dev handler %s on_free_cmd(%p)",
+ handler->name, cmd);
+ scst_set_cur_start(cmd);
+ handler->on_free_cmd(cmd);
+ scst_set_dev_on_free_time(cmd);
+ TRACE_DBG("Dev handler %s on_free_cmd() returned",
+ handler->name);
+ }
+ }
+
+ scst_release_space(cmd);
+
+ if (unlikely(cmd->sense != NULL)) {
+ TRACE_MEM("Releasing sense %p (cmd %p)", cmd->sense, cmd);
+ mempool_free(cmd->sense, scst_sense_mempool);
+ cmd->sense = NULL;
+ }
+
+ if (likely(cmd->tgt_dev != NULL)) {
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(!cmd->sent_for_exec) && !cmd->internal) {
+ PRINT_ERROR("Finishing not executed cmd %p (opcode "
+ "%d, target %s, LUN %lld, sn %d, expected_sn %d)",
+ cmd, cmd->cdb[0], cmd->tgtt->name,
+ (long long unsigned int)cmd->lun,
+ cmd->sn, cmd->tgt_dev->expected_sn);
+ scst_unblock_deferred(cmd->tgt_dev, cmd);
+ }
+#endif
+
+ if (unlikely(cmd->out_of_sn)) {
+ TRACE_SN("Out of SN cmd %p (tag %llu, sn %d), "
+ "destroy=%d", cmd,
+ (long long unsigned int)cmd->tag,
+ cmd->sn, destroy);
+ destroy = test_and_set_bit(SCST_CMD_CAN_BE_DESTROYED,
+ &cmd->cmd_flags);
+ }
+ }
+
+ if (likely(destroy))
+ scst_destroy_put_cmd(cmd);
+ return;
+}
+
+/* No locks supposed to be held. */
+void scst_check_retries(struct scst_tgt *tgt)
+{
+ int need_wake_up = 0;
+
+ /*
+ * We don't worry about overflow of finished_cmds, because we check
+ * only for its change.
+ */
+ atomic_inc(&tgt->finished_cmds);
+ /* See comment in scst_queue_retry_cmd() */
+ smp_mb__after_atomic_inc();
+ if (unlikely(tgt->retry_cmds > 0)) {
+ struct scst_cmd *c, *tc;
+ unsigned long flags;
+
+ TRACE_RETRY("Checking retry cmd list (retry_cmds %d)",
+ tgt->retry_cmds);
+
+ spin_lock_irqsave(&tgt->tgt_lock, flags);
+ list_for_each_entry_safe(c, tc, &tgt->retry_cmd_list,
+ cmd_list_entry) {
+ tgt->retry_cmds--;
+
+ TRACE_RETRY("Moving retry cmd %p to head of active "
+ "cmd list (retry_cmds left %d)",
+ c, tgt->retry_cmds);
+ spin_lock(&c->cmd_threads->cmd_list_lock);
+ list_move(&c->cmd_list_entry,
+ &c->cmd_threads->active_cmd_list);
+ wake_up(&c->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&c->cmd_threads->cmd_list_lock);
+
+ need_wake_up++;
+ if (need_wake_up >= 2) /* "slow start" */
+ break;
+ }
+ spin_unlock_irqrestore(&tgt->tgt_lock, flags);
+ }
+ return;
+}
+
+static void scst_tgt_retry_timer_fn(unsigned long arg)
+{
+ struct scst_tgt *tgt = (struct scst_tgt *)arg;
+ unsigned long flags;
+
+ TRACE_RETRY("Retry timer expired (retry_cmds %d)", tgt->retry_cmds);
+
+ spin_lock_irqsave(&tgt->tgt_lock, flags);
+ tgt->retry_timer_active = 0;
+ spin_unlock_irqrestore(&tgt->tgt_lock, flags);
+
+ scst_check_retries(tgt);
+ return;
+}
+
+struct scst_mgmt_cmd *scst_alloc_mgmt_cmd(gfp_t gfp_mask)
+{
+ struct scst_mgmt_cmd *mcmd;
+
+ mcmd = mempool_alloc(scst_mgmt_mempool, gfp_mask);
+ if (mcmd == NULL) {
+ PRINT_CRIT_ERROR("%s", "Allocation of management command "
+ "failed, some commands and their data could leak");
+ goto out;
+ }
+ memset(mcmd, 0, sizeof(*mcmd));
+
+out:
+ return mcmd;
+}
+
+void scst_free_mgmt_cmd(struct scst_mgmt_cmd *mcmd)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&mcmd->sess->sess_list_lock, flags);
+ atomic_dec(&mcmd->sess->sess_cmd_count);
+ spin_unlock_irqrestore(&mcmd->sess->sess_list_lock, flags);
+
+ scst_sess_put(mcmd->sess);
+
+ if (mcmd->mcmd_tgt_dev != NULL)
+ __scst_put();
+
+ mempool_free(mcmd, scst_mgmt_mempool);
+ return;
+}
+
+static bool is_report_sg_limitation(void)
+{
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ return (trace_flag & TRACE_OUT_OF_MEM) != 0;
+#else
+ return false;
+#endif
+}
+
+int scst_alloc_space(struct scst_cmd *cmd)
+{
+ gfp_t gfp_mask;
+ int res = -ENOMEM;
+ int atomic = scst_cmd_atomic(cmd);
+ int flags;
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+ static int ll;
+
+ gfp_mask = tgt_dev->gfp_mask | (atomic ? GFP_ATOMIC : GFP_KERNEL);
+
+ flags = atomic ? SGV_POOL_NO_ALLOC_ON_CACHE_MISS : 0;
+ if (cmd->no_sgv)
+ flags |= SGV_POOL_ALLOC_NO_CACHED;
+
+ cmd->sg = sgv_pool_alloc(tgt_dev->pool, cmd->bufflen, gfp_mask, flags,
+ &cmd->sg_cnt, &cmd->sgv, &cmd->dev->dev_mem_lim, NULL);
+ if (cmd->sg == NULL)
+ goto out;
+
+ if (unlikely(cmd->sg_cnt > tgt_dev->max_sg_cnt)) {
+ if ((ll < 10) || is_report_sg_limitation()) {
+ PRINT_INFO("Unable to complete command due to "
+ "SG IO count limitation (requested %d, "
+ "available %d, tgt lim %d)", cmd->sg_cnt,
+ tgt_dev->max_sg_cnt, cmd->tgt->sg_tablesize);
+ ll++;
+ }
+ goto out_sg_free;
+ }
+
+ if (cmd->data_direction != SCST_DATA_BIDI)
+ goto success;
+
+ cmd->in_sg = sgv_pool_alloc(tgt_dev->pool, cmd->in_bufflen, gfp_mask,
+ flags, &cmd->in_sg_cnt, &cmd->in_sgv,
+ &cmd->dev->dev_mem_lim, NULL);
+ if (cmd->in_sg == NULL)
+ goto out_sg_free;
+
+ if (unlikely(cmd->in_sg_cnt > tgt_dev->max_sg_cnt)) {
+ if ((ll < 10) || is_report_sg_limitation()) {
+ PRINT_INFO("Unable to complete command due to "
+ "SG IO count limitation (IN buffer, requested "
+ "%d, available %d, tgt lim %d)", cmd->in_sg_cnt,
+ tgt_dev->max_sg_cnt, cmd->tgt->sg_tablesize);
+ ll++;
+ }
+ goto out_in_sg_free;
+ }
+
+success:
+ res = 0;
+
+out:
+ return res;
+
+out_in_sg_free:
+ sgv_pool_free(cmd->in_sgv, &cmd->dev->dev_mem_lim);
+ cmd->in_sgv = NULL;
+ cmd->in_sg = NULL;
+ cmd->in_sg_cnt = 0;
+
+out_sg_free:
+ sgv_pool_free(cmd->sgv, &cmd->dev->dev_mem_lim);
+ cmd->sgv = NULL;
+ cmd->sg = NULL;
+ cmd->sg_cnt = 0;
+ goto out;
+}
+
+static void scst_release_space(struct scst_cmd *cmd)
+{
+
+ if (cmd->sgv == NULL) {
+ if ((cmd->sg != NULL) &&
+ !(cmd->tgt_data_buf_alloced || cmd->dh_data_buf_alloced)) {
+ TRACE_MEM("Freeing sg %p for cmd %p (cnt %d)", cmd->sg,
+ cmd, cmd->sg_cnt);
+ scst_free(cmd->sg, cmd->sg_cnt);
+ goto out_zero;
+ } else
+ goto out;
+ }
+
+ if (cmd->tgt_data_buf_alloced || cmd->dh_data_buf_alloced) {
+ TRACE_MEM("%s", "*data_buf_alloced set, returning");
+ goto out;
+ }
+
+ if (cmd->in_sgv != NULL) {
+ sgv_pool_free(cmd->in_sgv, &cmd->dev->dev_mem_lim);
+ cmd->in_sgv = NULL;
+ cmd->in_sg_cnt = 0;
+ cmd->in_sg = NULL;
+ cmd->in_bufflen = 0;
+ }
+
+ sgv_pool_free(cmd->sgv, &cmd->dev->dev_mem_lim);
+
+out_zero:
+ cmd->sgv = NULL;
+ cmd->sg_cnt = 0;
+ cmd->sg = NULL;
+ cmd->bufflen = 0;
+ cmd->data_len = 0;
+
+out:
+ return;
+}
+
+static void scsi_end_async(struct request *req, int error)
+{
+ struct scsi_io_context *sioc = req->end_io_data;
+
+ TRACE_DBG("sioc %p, cmd %p", sioc, sioc->data);
+
+ if (sioc->done)
+ sioc->done(sioc->data, sioc->sense, req->errors, req->resid_len);
+
+ if (!sioc->full_cdb_used)
+ kmem_cache_free(scsi_io_context_cache, sioc);
+ else
+ kfree(sioc);
+
+ __blk_put_request(req->q, req);
+ return;
+}
+
+/**
+ * scst_scsi_exec_async - executes a SCSI command in pass-through mode
+ * @cmd: scst command
+ * @done: callback function when done
+ */
+int scst_scsi_exec_async(struct scst_cmd *cmd,
+ void (*done)(void *, char *, int, int))
+{
+ int res = 0;
+ struct request_queue *q = cmd->dev->scsi_dev->request_queue;
+ struct request *rq;
+ struct scsi_io_context *sioc;
+ int write = (cmd->data_direction & SCST_DATA_WRITE) ? WRITE : READ;
+ gfp_t gfp = scst_cmd_atomic(cmd) ? GFP_ATOMIC : GFP_KERNEL;
+ int cmd_len = cmd->cdb_len;
+
+ if (cmd->ext_cdb_len == 0) {
+ TRACE_DBG("Simple CDB (cmd_len %d)", cmd_len);
+ sioc = kmem_cache_zalloc(scsi_io_context_cache, gfp);
+ if (sioc == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+ } else {
+ cmd_len += cmd->ext_cdb_len;
+
+ TRACE_DBG("Extended CDB (cmd_len %d)", cmd_len);
+
+ sioc = kzalloc(sizeof(*sioc) + cmd_len, gfp);
+ if (sioc == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ sioc->full_cdb_used = 1;
+
+ memcpy(sioc->full_cdb, cmd->cdb, cmd->cdb_len);
+ memcpy(&sioc->full_cdb[cmd->cdb_len], cmd->ext_cdb,
+ cmd->ext_cdb_len);
+ }
+
+ rq = blk_get_request(q, write, gfp);
+ if (rq == NULL) {
+ res = -ENOMEM;
+ goto out_free_sioc;
+ }
+
+ rq->cmd_type = REQ_TYPE_BLOCK_PC;
+ rq->cmd_flags |= REQ_QUIET;
+
+ if (cmd->sg != NULL) {
+ res = blk_rq_map_kern_sg(rq, cmd->sg, cmd->sg_cnt, gfp);
+ if (res) {
+ TRACE_DBG("blk_rq_map_kern_sg() failed: %d", res);
+ goto out_free_rq;
+ }
+ }
+
+ if (cmd->data_direction == SCST_DATA_BIDI) {
+ struct request *next_rq;
+
+ if (!test_bit(QUEUE_FLAG_BIDI, &q->queue_flags)) {
+ res = -EOPNOTSUPP;
+ goto out_free_unmap;
+ }
+
+ next_rq = blk_get_request(q, READ, gfp);
+ if (next_rq == NULL) {
+ res = -ENOMEM;
+ goto out_free_unmap;
+ }
+ rq->next_rq = next_rq;
+ next_rq->cmd_type = rq->cmd_type;
+
+ res = blk_rq_map_kern_sg(next_rq, cmd->in_sg,
+ cmd->in_sg_cnt, gfp);
+ if (res != 0)
+ goto out_free_unmap;
+ }
+
+ TRACE_DBG("sioc %p, cmd %p", sioc, cmd);
+
+ sioc->data = cmd;
+ sioc->done = done;
+
+ rq->cmd_len = cmd_len;
+ if (cmd->ext_cdb_len == 0) {
+ memset(rq->cmd, 0, BLK_MAX_CDB); /* ATAPI hates garbage after CDB */
+ memcpy(rq->cmd, cmd->cdb, cmd->cdb_len);
+ } else
+ rq->cmd = sioc->full_cdb;
+
+ rq->sense = sioc->sense;
+ rq->sense_len = sizeof(sioc->sense);
+ rq->timeout = cmd->timeout;
+ rq->retries = cmd->retries;
+ rq->end_io_data = sioc;
+
+ blk_execute_rq_nowait(rq->q, NULL, rq,
+ (cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE), scsi_end_async);
+out:
+ return res;
+
+out_free_unmap:
+ if (rq->next_rq != NULL) {
+ blk_put_request(rq->next_rq);
+ rq->next_rq = NULL;
+ }
+ blk_rq_unmap_kern_sg(rq, res);
+
+out_free_rq:
+ blk_put_request(rq);
+
+out_free_sioc:
+ if (!sioc->full_cdb_used)
+ kmem_cache_free(scsi_io_context_cache, sioc);
+ else
+ kfree(sioc);
+ goto out;
+}
+
+/**
+ * scst_copy_sg() - copy data between the command's SGs
+ *
+ * Copies data between cmd->tgt_sg and cmd->sg in direction defined by
+ * copy_dir parameter.
+ */
+void scst_copy_sg(struct scst_cmd *cmd, enum scst_sg_copy_dir copy_dir)
+{
+ struct scatterlist *src_sg, *dst_sg;
+ unsigned int to_copy;
+ int atomic = scst_cmd_atomic(cmd);
+
+ if (copy_dir == SCST_SG_COPY_FROM_TARGET) {
+ if (cmd->data_direction != SCST_DATA_BIDI) {
+ src_sg = cmd->tgt_sg;
+ dst_sg = cmd->sg;
+ to_copy = cmd->bufflen;
+ } else {
+ TRACE_MEM("BIDI cmd %p", cmd);
+ src_sg = cmd->tgt_in_sg;
+ dst_sg = cmd->in_sg;
+ to_copy = cmd->in_bufflen;
+ }
+ } else {
+ src_sg = cmd->sg;
+ dst_sg = cmd->tgt_sg;
+ to_copy = cmd->resp_data_len;
+ }
+
+ TRACE_MEM("cmd %p, copy_dir %d, src_sg %p, dst_sg %p, to_copy %lld",
+ cmd, copy_dir, src_sg, dst_sg, (long long)to_copy);
+
+ if (unlikely(src_sg == NULL) || unlikely(dst_sg == NULL)) {
+ /*
+ * It can happened, e.g., with scst_user for cmd with delay
+ * alloc, which failed with Check Condition.
+ */
+ goto out;
+ }
+
+ sg_copy(dst_sg, src_sg, 0, to_copy,
+ atomic ? KM_SOFTIRQ0 : KM_USER0,
+ atomic ? KM_SOFTIRQ1 : KM_USER1);
+
+out:
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_copy_sg);
+
+static const int SCST_CDB_LENGTH[8] = { 6, 10, 10, -1, 16, 12, -1, -1 };
+
+#define SCST_CDB_GROUP(opcode) ((opcode >> 5) & 0x7)
+#define SCST_GET_CDB_LEN(opcode) SCST_CDB_LENGTH[SCST_CDB_GROUP(opcode)]
+
+int scst_get_cdb_len(const uint8_t *cdb)
+{
+ return SCST_GET_CDB_LEN(cdb[0]);
+}
+
+/* get_trans_len_x extract x bytes from cdb as length starting from off */
+
+static int get_trans_cdb_len_10(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->cdb_len = 10;
+ cmd->bufflen = 0;
+ return 0;
+}
+
+static int get_trans_len_block_limit(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = 6;
+ return 0;
+}
+
+static int get_trans_len_read_capacity(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = READ_CAP_LEN;
+ return 0;
+}
+
+static int get_trans_len_serv_act_in(struct scst_cmd *cmd, uint8_t off)
+{
+ int res = 0;
+
+ if ((cmd->cdb[1] & 0x1f) == SAI_READ_CAPACITY_16) {
+ cmd->op_name = "READ CAPACITY(16)";
+ cmd->bufflen = READ_CAP16_LEN;
+ cmd->op_flags |= SCST_IMPLICIT_HQ|SCST_REG_RESERVE_ALLOWED;
+ } else
+ cmd->op_flags |= SCST_UNKNOWN_LENGTH;
+ return res;
+}
+
+static int get_trans_len_single(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = 1;
+ return 0;
+}
+
+static int get_trans_len_read_pos(struct scst_cmd *cmd, uint8_t off)
+{
+ uint8_t *p = (uint8_t *)cmd->cdb + off;
+ int res = 0;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 8;
+ cmd->bufflen |= ((u32)p[1]);
+
+ switch (cmd->cdb[1] & 0x1f) {
+ case 0:
+ case 1:
+ case 6:
+ if (cmd->bufflen != 0) {
+ PRINT_ERROR("READ POSITION: Invalid non-zero (%d) "
+ "allocation length for service action %x",
+ cmd->bufflen, cmd->cdb[1] & 0x1f);
+ goto out_inval;
+ }
+ break;
+ }
+
+ switch (cmd->cdb[1] & 0x1f) {
+ case 0:
+ case 1:
+ cmd->bufflen = 20;
+ break;
+ case 6:
+ cmd->bufflen = 32;
+ break;
+ case 8:
+ cmd->bufflen = max(28, cmd->bufflen);
+ break;
+ default:
+ PRINT_ERROR("READ POSITION: Invalid service action %x",
+ cmd->cdb[1] & 0x1f);
+ goto out_inval;
+ }
+
+out:
+ return res;
+
+out_inval:
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(scst_sense_invalid_field_in_cdb));
+ res = 1;
+ goto out;
+}
+
+static int get_trans_len_prevent_allow_medium_removal(struct scst_cmd *cmd,
+ uint8_t off)
+{
+ if ((cmd->cdb[4] & 3) == 0)
+ cmd->op_flags |= SCST_REG_RESERVE_ALLOWED;
+ return 0;
+}
+
+static int get_trans_len_start_stop(struct scst_cmd *cmd, uint8_t off)
+{
+ if ((cmd->cdb[4] & 0xF1) == 0x1)
+ cmd->op_flags |= SCST_REG_RESERVE_ALLOWED;
+ return 0;
+}
+
+static int get_trans_len_3_read_elem_stat(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 16;
+ cmd->bufflen |= ((u32)p[1]) << 8;
+ cmd->bufflen |= ((u32)p[2]);
+
+ if ((cmd->cdb[6] & 0x2) == 0x2)
+ cmd->op_flags |= SCST_REG_RESERVE_ALLOWED;
+
+ return 0;
+}
+
+static int get_trans_len_1(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = (u32)cmd->cdb[off];
+ return 0;
+}
+
+static int get_trans_len_1_256(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = (u32)cmd->cdb[off];
+ if (cmd->bufflen == 0)
+ cmd->bufflen = 256;
+ return 0;
+}
+
+static int get_trans_len_2(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 8;
+ cmd->bufflen |= ((u32)p[1]);
+
+ return 0;
+}
+
+static int get_trans_len_3(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 16;
+ cmd->bufflen |= ((u32)p[1]) << 8;
+ cmd->bufflen |= ((u32)p[2]);
+
+ return 0;
+}
+
+static int get_trans_len_4(struct scst_cmd *cmd, uint8_t off)
+{
+ const uint8_t *p = cmd->cdb + off;
+
+ cmd->bufflen = 0;
+ cmd->bufflen |= ((u32)p[0]) << 24;
+ cmd->bufflen |= ((u32)p[1]) << 16;
+ cmd->bufflen |= ((u32)p[2]) << 8;
+ cmd->bufflen |= ((u32)p[3]);
+
+ return 0;
+}
+
+static int get_trans_len_none(struct scst_cmd *cmd, uint8_t off)
+{
+ cmd->bufflen = 0;
+ return 0;
+}
+
+/**
+ * scst_get_cdb_info() - fill various info about the command's CDB
+ *
+ * Description:
+ * Fills various info about the command's CDB in the corresponding fields
+ * in the command.
+ *
+ * Returns: 0 on success, <0 if command is unknown, >0 if command
+ * is invalid.
+ */
+int scst_get_cdb_info(struct scst_cmd *cmd)
+{
+ int dev_type = cmd->dev->type;
+ int i, res = 0;
+ uint8_t op;
+ const struct scst_sdbops *ptr = NULL;
+
+ op = cmd->cdb[0]; /* get clear opcode */
+
+ TRACE_DBG("opcode=%02x, cdblen=%d bytes, tblsize=%d, "
+ "dev_type=%d", op, SCST_GET_CDB_LEN(op), SCST_CDB_TBL_SIZE,
+ dev_type);
+
+ i = scst_scsi_op_list[op];
+ while (i < SCST_CDB_TBL_SIZE && scst_scsi_op_table[i].ops == op) {
+ if (scst_scsi_op_table[i].devkey[dev_type] != SCST_CDB_NOTSUPP) {
+ ptr = &scst_scsi_op_table[i];
+ TRACE_DBG("op = 0x%02x+'%c%c%c%c%c%c%c%c%c%c'+<%s>",
+ ptr->ops, ptr->devkey[0], /* disk */
+ ptr->devkey[1], /* tape */
+ ptr->devkey[2], /* printer */
+ ptr->devkey[3], /* cpu */
+ ptr->devkey[4], /* cdr */
+ ptr->devkey[5], /* cdrom */
+ ptr->devkey[6], /* scanner */
+ ptr->devkey[7], /* worm */
+ ptr->devkey[8], /* changer */
+ ptr->devkey[9], /* commdev */
+ ptr->op_name);
+ TRACE_DBG("direction=%d flags=%d off=%d",
+ ptr->direction,
+ ptr->flags,
+ ptr->off);
+ break;
+ }
+ i++;
+ }
+
+ if (unlikely(ptr == NULL)) {
+ /* opcode not found or now not used */
+ TRACE(TRACE_MINOR, "Unknown opcode 0x%x for type %d", op,
+ dev_type);
+ res = -1;
+ goto out;
+ }
+
+ cmd->cdb_len = SCST_GET_CDB_LEN(op);
+ cmd->op_name = ptr->op_name;
+ cmd->data_direction = ptr->direction;
+ cmd->op_flags = ptr->flags | SCST_INFO_VALID;
+ res = (*ptr->get_trans_len)(cmd, ptr->off);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_get_cdb_info);
+
+/* Packs SCST LUN back to SCSI form */
+uint64_t scst_pack_lun(const uint64_t lun, unsigned int addr_method)
+{
+
+ uint64_t res = 0;
+ uint16_t *p = (uint16_t *)&res;
+
+ res = lun;
+
+ if ((addr_method == SCST_LUN_ADDR_METHOD_FLAT) && (lun != 0)) {
+ /*
+ * Flat space: luns other than 0 should use flat space
+ * addressing method.
+ */
+ *p = 0x7fff & *p;
+ *p = 0x4000 | *p;
+ }
+ /* Default is to use peripheral device addressing mode */
+
+ *p = cpu_to_be16(*p);
+ return res;
+}
+
+/*
+ * Routine to extract a lun number from an 8-byte LUN structure
+ * in network byte order (BE).
+ * (see SAM-2, Section 4.12.3 page 40)
+ * Supports 2 types of lun unpacking: peripheral and logical unit.
+ */
+uint64_t scst_unpack_lun(const uint8_t *lun, int len)
+{
+ uint64_t res = NO_SUCH_LUN;
+ int address_method;
+
+ TRACE_BUFF_FLAG(TRACE_DEBUG, "Raw LUN", lun, len);
+
+ if (unlikely(len < 2)) {
+ PRINT_ERROR("Illegal lun length %d, expected 2 bytes or "
+ "more", len);
+ goto out;
+ }
+
+ if (len > 2) {
+ switch (len) {
+ case 8:
+ if ((*((uint64_t *)lun) &
+ __constant_cpu_to_be64(0x0000FFFFFFFFFFFFLL)) != 0)
+ goto out_err;
+ break;
+ case 4:
+ if (*((uint16_t *)&lun[2]) != 0)
+ goto out_err;
+ break;
+ case 6:
+ if (*((uint32_t *)&lun[2]) != 0)
+ goto out_err;
+ break;
+ default:
+ goto out_err;
+ }
+ }
+
+ address_method = (*lun) >> 6; /* high 2 bits of byte 0 */
+ switch (address_method) {
+ case 0: /* peripheral device addressing method */
+#if 0
+ if (*lun) {
+ PRINT_ERROR("Illegal BUS INDENTIFIER in LUN "
+ "peripheral device addressing method 0x%02x, "
+ "expected 0", *lun);
+ break;
+ }
+ res = *(lun + 1);
+ break;
+#else
+ /*
+ * Looks like it's legal to use it as flat space addressing
+ * method as well
+ */
+
+ /* go through */
+#endif
+
+ case 1: /* flat space addressing method */
+ res = *(lun + 1) | (((*lun) & 0x3f) << 8);
+ break;
+
+ case 2: /* logical unit addressing method */
+ if (*lun & 0x3f) {
+ PRINT_ERROR("Illegal BUS NUMBER in LUN logical unit "
+ "addressing method 0x%02x, expected 0",
+ *lun & 0x3f);
+ break;
+ }
+ if (*(lun + 1) & 0xe0) {
+ PRINT_ERROR("Illegal TARGET in LUN logical unit "
+ "addressing method 0x%02x, expected 0",
+ (*(lun + 1) & 0xf8) >> 5);
+ break;
+ }
+ res = *(lun + 1) & 0x1f;
+ break;
+
+ case 3: /* extended logical unit addressing method */
+ default:
+ PRINT_ERROR("Unimplemented LUN addressing method %u",
+ address_method);
+ break;
+ }
+
+out:
+ return res;
+
+out_err:
+ PRINT_ERROR("%s", "Multi-level LUN unimplemented");
+ goto out;
+}
+
+/**
+ ** Generic parse() support routines.
+ ** Done via pointer on functions to avoid unneeded dereferences on
+ ** the fast path.
+ **/
+
+/**
+ * scst_calc_block_shift() - calculate block shift
+ *
+ * Calculates and returns block shift for the given sector size
+ */
+int scst_calc_block_shift(int sector_size)
+{
+ int block_shift = 0;
+ int t;
+
+ if (sector_size == 0)
+ sector_size = 512;
+
+ t = sector_size;
+ while (1) {
+ if ((t & 1) != 0)
+ break;
+ t >>= 1;
+ block_shift++;
+ }
+ if (block_shift < 9) {
+ PRINT_ERROR("Wrong sector size %d", sector_size);
+ block_shift = -1;
+ }
+ return block_shift;
+}
+EXPORT_SYMBOL_GPL(scst_calc_block_shift);
+
+/**
+ * scst_sbc_generic_parse() - generic SBC parsing
+ *
+ * Generic parse() for SBC (disk) devices
+ */
+int scst_sbc_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ switch (cmd->cdb[0]) {
+ case VERIFY_6:
+ case VERIFY:
+ case VERIFY_12:
+ case VERIFY_16:
+ if ((cmd->cdb[1] & BYTCHK) == 0) {
+ cmd->data_len = cmd->bufflen << get_block_shift(cmd);
+ cmd->bufflen = 0;
+ goto set_timeout;
+ } else
+ cmd->data_len = 0;
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED) {
+ /*
+ * No need for locks here, since *_detach() can not be
+ * called, when there are existing commands.
+ */
+ cmd->bufflen = cmd->bufflen << get_block_shift(cmd);
+ }
+
+set_timeout:
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_DISK_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_DISK_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_DISK_LONG_TIMEOUT;
+
+ TRACE_DBG("res %d, bufflen %d, data_len %d, direct %d",
+ res, cmd->bufflen, cmd->data_len, cmd->data_direction);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_sbc_generic_parse);
+
+/**
+ * scst_cdrom_generic_parse() - generic MMC parse
+ *
+ * Generic parse() for MMC (cdrom) devices
+ */
+int scst_cdrom_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ cmd->cdb[1] &= 0x1f;
+
+ switch (cmd->cdb[0]) {
+ case VERIFY_6:
+ case VERIFY:
+ case VERIFY_12:
+ case VERIFY_16:
+ if ((cmd->cdb[1] & BYTCHK) == 0) {
+ cmd->data_len = cmd->bufflen << get_block_shift(cmd);
+ cmd->bufflen = 0;
+ goto set_timeout;
+ }
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED)
+ cmd->bufflen = cmd->bufflen << get_block_shift(cmd);
+
+set_timeout:
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_CDROM_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_CDROM_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_CDROM_LONG_TIMEOUT;
+
+ TRACE_DBG("res=%d, bufflen=%d, direct=%d", res, cmd->bufflen,
+ cmd->data_direction);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_cdrom_generic_parse);
+
+/**
+ * scst_modisk_generic_parse() - generic MO parse
+ *
+ * Generic parse() for MO disk devices
+ */
+int scst_modisk_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_shift)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ cmd->cdb[1] &= 0x1f;
+
+ switch (cmd->cdb[0]) {
+ case VERIFY_6:
+ case VERIFY:
+ case VERIFY_12:
+ case VERIFY_16:
+ if ((cmd->cdb[1] & BYTCHK) == 0) {
+ cmd->data_len = cmd->bufflen << get_block_shift(cmd);
+ cmd->bufflen = 0;
+ goto set_timeout;
+ }
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED)
+ cmd->bufflen = cmd->bufflen << get_block_shift(cmd);
+
+set_timeout:
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_MODISK_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_MODISK_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_MODISK_LONG_TIMEOUT;
+
+ TRACE_DBG("res=%d, bufflen=%d, direct=%d", res, cmd->bufflen,
+ cmd->data_direction);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_modisk_generic_parse);
+
+/**
+ * scst_tape_generic_parse() - generic tape parse
+ *
+ * Generic parse() for tape devices
+ */
+int scst_tape_generic_parse(struct scst_cmd *cmd,
+ int (*get_block_size)(struct scst_cmd *cmd))
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+
+ if (cmd->cdb[0] == READ_POSITION) {
+ int tclp = cmd->cdb[1] & 4;
+ int long_bit = cmd->cdb[1] & 2;
+ int bt = cmd->cdb[1] & 1;
+
+ if ((tclp == long_bit) && (!bt || !long_bit)) {
+ cmd->bufflen =
+ tclp ? POSITION_LEN_LONG : POSITION_LEN_SHORT;
+ cmd->data_direction = SCST_DATA_READ;
+ } else {
+ cmd->bufflen = 0;
+ cmd->data_direction = SCST_DATA_NONE;
+ }
+ }
+
+ if (cmd->op_flags & SCST_TRANSFER_LEN_TYPE_FIXED & cmd->cdb[1])
+ cmd->bufflen = cmd->bufflen * get_block_size(cmd);
+
+ if ((cmd->op_flags & (SCST_SMALL_TIMEOUT | SCST_LONG_TIMEOUT)) == 0)
+ cmd->timeout = SCST_GENERIC_TAPE_REG_TIMEOUT;
+ else if (cmd->op_flags & SCST_SMALL_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_TAPE_SMALL_TIMEOUT;
+ else if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_TAPE_LONG_TIMEOUT;
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_tape_generic_parse);
+
+static int scst_null_parse(struct scst_cmd *cmd)
+{
+ int res = 0;
+
+ /*
+ * SCST sets good defaults for cmd->data_direction and cmd->bufflen,
+ * therefore change them only if necessary
+ */
+
+ TRACE_DBG("op_name <%s> direct %d flags %d transfer_len %d",
+ cmd->op_name, cmd->data_direction, cmd->op_flags, cmd->bufflen);
+#if 0
+ switch (cmd->cdb[0]) {
+ default:
+ /* It's all good */
+ break;
+ }
+#endif
+ TRACE_DBG("res %d bufflen %d direct %d",
+ res, cmd->bufflen, cmd->data_direction);
+ return res;
+}
+
+/**
+ * scst_changer_generic_parse() - generic changer parse
+ *
+ * Generic parse() for changer devices
+ */
+int scst_changer_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd))
+{
+ int res = scst_null_parse(cmd);
+
+ if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_CHANGER_LONG_TIMEOUT;
+ else
+ cmd->timeout = SCST_GENERIC_CHANGER_TIMEOUT;
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_changer_generic_parse);
+
+/**
+ * scst_processor_generic_parse - generic SCSI processor parse
+ *
+ * Generic parse() for SCSI processor devices
+ */
+int scst_processor_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd))
+{
+ int res = scst_null_parse(cmd);
+
+ if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_PROCESSOR_LONG_TIMEOUT;
+ else
+ cmd->timeout = SCST_GENERIC_PROCESSOR_TIMEOUT;
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_processor_generic_parse);
+
+/**
+ * scst_raid_generic_parse() - generic RAID parse
+ *
+ * Generic parse() for RAID devices
+ */
+int scst_raid_generic_parse(struct scst_cmd *cmd,
+ int (*nothing)(struct scst_cmd *cmd))
+{
+ int res = scst_null_parse(cmd);
+
+ if (cmd->op_flags & SCST_LONG_TIMEOUT)
+ cmd->timeout = SCST_GENERIC_RAID_LONG_TIMEOUT;
+ else
+ cmd->timeout = SCST_GENERIC_RAID_TIMEOUT;
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_raid_generic_parse);
+
+/**
+ ** Generic dev_done() support routines.
+ ** Done via pointer on functions to avoid unneeded dereferences on
+ ** the fast path.
+ **/
+
+/**
+ * scst_block_generic_dev_done() - generic SBC dev_done
+ *
+ * Generic dev_done() for block (SBC) devices
+ */
+int scst_block_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_shift)(struct scst_cmd *cmd, int block_shift))
+{
+ int opcode = cmd->cdb[0];
+ int status = cmd->status;
+ int res = SCST_CMD_STATE_DEFAULT;
+
+ /*
+ * SCST sets good defaults for cmd->is_send_status and
+ * cmd->resp_data_len based on cmd->status and cmd->data_direction,
+ * therefore change them only if necessary
+ */
+
+ if ((status == SAM_STAT_GOOD) || (status == SAM_STAT_CONDITION_MET)) {
+ switch (opcode) {
+ case READ_CAPACITY:
+ {
+ /* Always keep track of disk capacity */
+ int buffer_size, sector_size, sh;
+ uint8_t *buffer;
+
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size <= 0)) {
+ if (buffer_size < 0) {
+ PRINT_ERROR("%s: Unable to get the"
+ " buffer (%d)", __func__, buffer_size);
+ }
+ goto out;
+ }
+
+ sector_size =
+ ((buffer[4] << 24) | (buffer[5] << 16) |
+ (buffer[6] << 8) | (buffer[7] << 0));
+ scst_put_buf(cmd, buffer);
+ if (sector_size != 0)
+ sh = scst_calc_block_shift(sector_size);
+ else
+ sh = 0;
+ set_block_shift(cmd, sh);
+ TRACE_DBG("block_shift %d", sh);
+ break;
+ }
+ default:
+ /* It's all good */
+ break;
+ }
+ }
+
+ TRACE_DBG("cmd->is_send_status=%x, cmd->resp_data_len=%d, "
+ "res=%d", cmd->is_send_status, cmd->resp_data_len, res);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_block_generic_dev_done);
+
+/**
+ * scst_tape_generic_dev_done() - generic tape dev done
+ *
+ * Generic dev_done() for tape devices
+ */
+int scst_tape_generic_dev_done(struct scst_cmd *cmd,
+ void (*set_block_size)(struct scst_cmd *cmd, int block_shift))
+{
+ int opcode = cmd->cdb[0];
+ int res = SCST_CMD_STATE_DEFAULT;
+ int buffer_size, bs;
+ uint8_t *buffer = NULL;
+
+ /*
+ * SCST sets good defaults for cmd->is_send_status and
+ * cmd->resp_data_len based on cmd->status and cmd->data_direction,
+ * therefore change them only if necessary
+ */
+
+ switch (opcode) {
+ case MODE_SENSE:
+ case MODE_SELECT:
+ buffer_size = scst_get_buf_first(cmd, &buffer);
+ if (unlikely(buffer_size <= 0)) {
+ if (buffer_size < 0) {
+ PRINT_ERROR("%s: Unable to get the buffer (%d)",
+ __func__, buffer_size);
+ }
+ goto out;
+ }
+ break;
+ }
+
+ switch (opcode) {
+ case MODE_SENSE:
+ TRACE_DBG("%s", "MODE_SENSE");
+ if ((cmd->cdb[2] & 0xC0) == 0) {
+ if (buffer[3] == 8) {
+ bs = (buffer[9] << 16) |
+ (buffer[10] << 8) | buffer[11];
+ set_block_size(cmd, bs);
+ }
+ }
+ break;
+ case MODE_SELECT:
+ TRACE_DBG("%s", "MODE_SELECT");
+ if (buffer[3] == 8) {
+ bs = (buffer[9] << 16) | (buffer[10] << 8) |
+ (buffer[11]);
+ set_block_size(cmd, bs);
+ }
+ break;
+ default:
+ /* It's all good */
+ break;
+ }
+
+ switch (opcode) {
+ case MODE_SENSE:
+ case MODE_SELECT:
+ scst_put_buf(cmd, buffer);
+ break;
+ }
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_tape_generic_dev_done);
+
+static void scst_check_internal_sense(struct scst_device *dev, int result,
+ uint8_t *sense, int sense_len)
+{
+
+ if (host_byte(result) == DID_RESET) {
+ int sl;
+ TRACE(TRACE_MGMT, "DID_RESET received for device %s, "
+ "triggering reset UA", dev->virt_name);
+ sl = scst_set_sense(sense, sense_len, dev->d_sense,
+ SCST_LOAD_SENSE(scst_sense_reset_UA));
+ scst_dev_check_set_UA(dev, NULL, sense, sl);
+ } else if ((status_byte(result) == CHECK_CONDITION) &&
+ scst_is_ua_sense(sense, sense_len))
+ scst_dev_check_set_UA(dev, NULL, sense, sense_len);
+ return;
+}
+
+/**
+ * scst_to_dma_dir() - translate SCST's data direction to DMA direction
+ *
+ * Translates SCST's data direction to DMA one from backend storage
+ * perspective.
+ */
+enum dma_data_direction scst_to_dma_dir(int scst_dir)
+{
+ static const enum dma_data_direction tr_tbl[] = { DMA_NONE,
+ DMA_TO_DEVICE, DMA_FROM_DEVICE, DMA_BIDIRECTIONAL, DMA_NONE };
+
+ return tr_tbl[scst_dir];
+}
+EXPORT_SYMBOL(scst_to_dma_dir);
+
+/*
+ * scst_to_tgt_dma_dir() - translate SCST data direction to DMA direction
+ *
+ * Translates SCST data direction to DMA data direction from the perspective
+ * of the target device.
+ */
+enum dma_data_direction scst_to_tgt_dma_dir(int scst_dir)
+{
+ static const enum dma_data_direction tr_tbl[] = { DMA_NONE,
+ DMA_FROM_DEVICE, DMA_TO_DEVICE, DMA_BIDIRECTIONAL, DMA_NONE };
+
+ return tr_tbl[scst_dir];
+}
+EXPORT_SYMBOL(scst_to_tgt_dma_dir);
+
+/**
+ * scst_obtain_device_parameters() - obtain device control parameters
+ *
+ * Issues a MODE SENSE for control mode page data and sets the corresponding
+ * dev's parameter from it. Returns 0 on success and not 0 otherwise.
+ */
+int scst_obtain_device_parameters(struct scst_device *dev)
+{
+ int rc, i;
+ uint8_t cmd[16];
+ uint8_t buffer[4+0x0A];
+ uint8_t sense_buffer[SCSI_SENSE_BUFFERSIZE];
+
+ EXTRACHECKS_BUG_ON(dev->scsi_dev == NULL);
+
+ for (i = 0; i < 5; i++) {
+ /* Get control mode page */
+ memset(cmd, 0, sizeof(cmd));
+#if 0
+ cmd[0] = MODE_SENSE_10;
+ cmd[1] = 0;
+ cmd[2] = 0x0A;
+ cmd[8] = sizeof(buffer); /* it's < 256 */
+#else
+ cmd[0] = MODE_SENSE;
+ cmd[1] = 8; /* DBD */
+ cmd[2] = 0x0A;
+ cmd[4] = sizeof(buffer);
+#endif
+
+ memset(buffer, 0, sizeof(buffer));
+ memset(sense_buffer, 0, sizeof(sense_buffer));
+
+ TRACE(TRACE_SCSI, "%s", "Doing internal MODE_SENSE");
+ rc = scsi_execute(dev->scsi_dev, cmd, SCST_DATA_READ, buffer,
+ sizeof(buffer), sense_buffer, 15, 0, 0
+ , NULL
+ );
+
+ TRACE_DBG("MODE_SENSE done: %x", rc);
+
+ if (scsi_status_is_good(rc)) {
+ int q;
+
+ PRINT_BUFF_FLAG(TRACE_SCSI,
+ "Returned control mode page data",
+ buffer, sizeof(buffer));
+
+ dev->tst = buffer[4+2] >> 5;
+ q = buffer[4+3] >> 4;
+ if (q > SCST_CONTR_MODE_QUEUE_ALG_UNRESTRICTED_REORDER) {
+ PRINT_ERROR("Too big QUEUE ALG %x, dev %s",
+ dev->queue_alg, dev->virt_name);
+ }
+ dev->queue_alg = q;
+ dev->swp = (buffer[4+4] & 0x8) >> 3;
+ dev->tas = (buffer[4+5] & 0x40) >> 6;
+ dev->d_sense = (buffer[4+2] & 0x4) >> 2;
+
+ /*
+ * Unfortunately, SCSI ML doesn't provide a way to
+ * specify commands task attribute, so we can rely on
+ * device's restricted reordering only. Linux I/O
+ * subsystem doesn't reorder pass-through (PC) requests.
+ */
+ dev->has_own_order_mgmt = !dev->queue_alg;
+
+ PRINT_INFO("Device %s: TST %x, QUEUE ALG %x, SWP %x, "
+ "TAS %x, D_SENSE %d, has_own_order_mgmt %d",
+ dev->virt_name, dev->tst, dev->queue_alg,
+ dev->swp, dev->tas, dev->d_sense,
+ dev->has_own_order_mgmt);
+
+ goto out;
+ } else {
+ scst_check_internal_sense(dev, rc, sense_buffer,
+ sizeof(sense_buffer));
+#if 0
+ if ((status_byte(rc) == CHECK_CONDITION) &&
+ SCST_SENSE_VALID(sense_buffer)) {
+#else
+ /*
+ * 3ware controller is buggy and returns CONDITION_GOOD
+ * instead of CHECK_CONDITION
+ */
+ if (SCST_SENSE_VALID(sense_buffer)) {
+#endif
+ PRINT_BUFF_FLAG(TRACE_SCSI,
+ "Returned sense data",
+ sense_buffer, sizeof(sense_buffer));
+ if (scst_analyze_sense(sense_buffer,
+ sizeof(sense_buffer),
+ SCST_SENSE_KEY_VALID,
+ ILLEGAL_REQUEST, 0, 0)) {
+ PRINT_INFO("Device %s doesn't support "
+ "MODE SENSE", dev->virt_name);
+ break;
+ } else if (scst_analyze_sense(sense_buffer,
+ sizeof(sense_buffer),
+ SCST_SENSE_KEY_VALID,
+ NOT_READY, 0, 0)) {
+ PRINT_ERROR("Device %s not ready",
+ dev->virt_name);
+ break;
+ }
+ } else {
+ PRINT_INFO("Internal MODE SENSE to "
+ "device %s failed: %x",
+ dev->virt_name, rc);
+ PRINT_BUFF_FLAG(TRACE_SCSI, "MODE SENSE sense",
+ sense_buffer, sizeof(sense_buffer));
+ switch (host_byte(rc)) {
+ case DID_RESET:
+ case DID_ABORT:
+ case DID_SOFT_ERROR:
+ break;
+ default:
+ goto brk;
+ }
+ switch (driver_byte(rc)) {
+ case DRIVER_BUSY:
+ case DRIVER_SOFT:
+ break;
+ default:
+ goto brk;
+ }
+ }
+ }
+ }
+brk:
+ PRINT_WARNING("Unable to get device's %s control mode page, using "
+ "existing values/defaults: TST %x, QUEUE ALG %x, SWP %x, "
+ "TAS %x, D_SENSE %d, has_own_order_mgmt %d", dev->virt_name,
+ dev->tst, dev->queue_alg, dev->swp, dev->tas, dev->d_sense,
+ dev->has_own_order_mgmt);
+
+out:
+ return 0;
+}
+EXPORT_SYMBOL_GPL(scst_obtain_device_parameters);
+
+/* Called under dev_lock and BH off */
+void scst_process_reset(struct scst_device *dev,
+ struct scst_session *originator, struct scst_cmd *exclude_cmd,
+ struct scst_mgmt_cmd *mcmd, bool setUA)
+{
+ struct scst_tgt_dev *tgt_dev;
+ struct scst_cmd *cmd, *tcmd;
+
+ /* Clear RESERVE'ation, if necessary */
+ if (dev->dev_reserved) {
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ TRACE_MGMT_DBG("Clearing RESERVE'ation for "
+ "tgt_dev LUN %lld",
+ (long long unsigned int)tgt_dev->lun);
+ clear_bit(SCST_TGT_DEV_RESERVED,
+ &tgt_dev->tgt_dev_flags);
+ }
+ dev->dev_reserved = 0;
+ /*
+ * There is no need to send RELEASE, since the device is going
+ * to be resetted. Actually, since we can be in RESET TM
+ * function, it might be dangerous.
+ */
+ }
+
+ dev->dev_double_ua_possible = 1;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ struct scst_session *sess = tgt_dev->sess;
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+
+ scst_free_all_UA(tgt_dev);
+
+ memset(tgt_dev->tgt_dev_sense, 0,
+ sizeof(tgt_dev->tgt_dev_sense));
+
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+
+ spin_lock_irq(&sess->sess_list_lock);
+
+ TRACE_DBG("Searching in sess cmd list (sess=%p)", sess);
+ list_for_each_entry(cmd, &sess->sess_cmd_list,
+ sess_cmd_list_entry) {
+ if (cmd == exclude_cmd)
+ continue;
+ if ((cmd->tgt_dev == tgt_dev) ||
+ ((cmd->tgt_dev == NULL) &&
+ (cmd->lun == tgt_dev->lun))) {
+ scst_abort_cmd(cmd, mcmd,
+ (tgt_dev->sess != originator), 0);
+ }
+ }
+ spin_unlock_irq(&sess->sess_list_lock);
+ }
+
+ list_for_each_entry_safe(cmd, tcmd, &dev->blocked_cmd_list,
+ blocked_cmd_list_entry) {
+ if (test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)) {
+ list_del(&cmd->blocked_cmd_list_entry);
+ TRACE_MGMT_DBG("Adding aborted blocked cmd %p "
+ "to active cmd list", cmd);
+ spin_lock_irq(&cmd->cmd_threads->cmd_list_lock);
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&cmd->cmd_threads->cmd_list_lock);
+ }
+ }
+
+ if (setUA) {
+ uint8_t sense_buffer[SCST_STANDARD_SENSE_LEN];
+ int sl = scst_set_sense(sense_buffer, sizeof(sense_buffer),
+ dev->d_sense, SCST_LOAD_SENSE(scst_sense_reset_UA));
+ scst_dev_check_set_local_UA(dev, exclude_cmd, sense_buffer, sl);
+ }
+ return;
+}
+
+/* No locks, no IRQ or IRQ-disabled context allowed */
+int scst_set_pending_UA(struct scst_cmd *cmd)
+{
+ int res = 0, i;
+ struct scst_tgt_dev_UA *UA_entry;
+ bool first = true, global_unlock = false;
+ struct scst_session *sess = cmd->sess;
+
+ TRACE_MGMT_DBG("Setting pending UA cmd %p", cmd);
+
+ spin_lock_bh(&cmd->tgt_dev->tgt_dev_lock);
+
+again:
+ /* UA list could be cleared behind us, so retest */
+ if (list_empty(&cmd->tgt_dev->UA_list)) {
+ TRACE_DBG("%s",
+ "SCST_TGT_DEV_UA_PENDING set, but UA_list empty");
+ res = -1;
+ goto out_unlock_tgt_dev_lock;
+ }
+
+ UA_entry = list_entry(cmd->tgt_dev->UA_list.next, typeof(*UA_entry),
+ UA_list_entry);
+
+ TRACE_DBG("next %p UA_entry %p",
+ cmd->tgt_dev->UA_list.next, UA_entry);
+
+ if (UA_entry->global_UA && first) {
+ TRACE_MGMT_DBG("Global UA %p detected", UA_entry);
+
+ spin_unlock_bh(&cmd->tgt_dev->tgt_dev_lock);
+
+ /*
+ * cmd won't allow to suspend activities, so we can access
+ * sess->sess_tgt_dev_list_hash without any additional
+ * protection.
+ */
+
+ local_bh_disable();
+
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ /* Lockdep triggers here a false positive.. */
+ spin_lock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ first = false;
+ global_unlock = true;
+ goto again;
+ }
+
+ if (scst_set_cmd_error_sense(cmd, UA_entry->UA_sense_buffer,
+ UA_entry->UA_valid_sense_len) != 0)
+ goto out_unlock;
+
+ cmd->ua_ignore = 1;
+
+ list_del(&UA_entry->UA_list_entry);
+
+ if (UA_entry->global_UA) {
+ for (i = 0; i < TGT_DEV_HASH_SIZE; i++) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ struct scst_tgt_dev_UA *ua;
+ list_for_each_entry(ua, &tgt_dev->UA_list,
+ UA_list_entry) {
+ if (ua->global_UA &&
+ memcmp(ua->UA_sense_buffer,
+ UA_entry->UA_sense_buffer,
+ sizeof(ua->UA_sense_buffer)) == 0) {
+ TRACE_MGMT_DBG("Freeing not "
+ "needed global UA %p",
+ ua);
+ list_del(&ua->UA_list_entry);
+ mempool_free(ua, scst_ua_mempool);
+ break;
+ }
+ }
+ }
+ }
+ }
+
+ mempool_free(UA_entry, scst_ua_mempool);
+
+ if (list_empty(&cmd->tgt_dev->UA_list)) {
+ clear_bit(SCST_TGT_DEV_UA_PENDING,
+ &cmd->tgt_dev->tgt_dev_flags);
+ }
+
+out_unlock:
+ if (global_unlock) {
+ for (i = TGT_DEV_HASH_SIZE-1; i >= 0; i--) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[i];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry_reverse(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ spin_unlock(&tgt_dev->tgt_dev_lock);
+ }
+ }
+
+ local_bh_enable();
+ spin_lock_bh(&cmd->tgt_dev->tgt_dev_lock);
+ }
+
+out_unlock_tgt_dev_lock:
+ spin_unlock_bh(&cmd->tgt_dev->tgt_dev_lock);
+ return res;
+}
+
+/* Called under tgt_dev_lock and BH off */
+static void scst_alloc_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags)
+{
+ struct scst_tgt_dev_UA *UA_entry = NULL;
+
+ UA_entry = mempool_alloc(scst_ua_mempool, GFP_ATOMIC);
+ if (UA_entry == NULL) {
+ PRINT_CRIT_ERROR("%s", "UNIT ATTENTION memory "
+ "allocation failed. The UNIT ATTENTION "
+ "on some sessions will be missed");
+ PRINT_BUFFER("Lost UA", sense, sense_len);
+ goto out;
+ }
+ memset(UA_entry, 0, sizeof(*UA_entry));
+
+ UA_entry->global_UA = (flags & SCST_SET_UA_FLAG_GLOBAL) != 0;
+ if (UA_entry->global_UA)
+ TRACE_MGMT_DBG("Queuing global UA %p", UA_entry);
+
+ if (sense_len > (int)sizeof(UA_entry->UA_sense_buffer)) {
+ PRINT_WARNING("Sense truncated (needed %d), shall you increase "
+ "SCST_SENSE_BUFFERSIZE?", sense_len);
+ sense_len = sizeof(UA_entry->UA_sense_buffer);
+ }
+ memcpy(UA_entry->UA_sense_buffer, sense, sense_len);
+ UA_entry->UA_valid_sense_len = sense_len;
+
+ set_bit(SCST_TGT_DEV_UA_PENDING, &tgt_dev->tgt_dev_flags);
+
+ TRACE_MGMT_DBG("Adding new UA to tgt_dev %p", tgt_dev);
+
+ if (flags & SCST_SET_UA_FLAG_AT_HEAD)
+ list_add(&UA_entry->UA_list_entry, &tgt_dev->UA_list);
+ else
+ list_add_tail(&UA_entry->UA_list_entry, &tgt_dev->UA_list);
+
+out:
+ return;
+}
+
+/* tgt_dev_lock supposed to be held and BH off */
+static void __scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags)
+{
+ int skip_UA = 0;
+ struct scst_tgt_dev_UA *UA_entry_tmp;
+ int len = min((int)sizeof(UA_entry_tmp->UA_sense_buffer), sense_len);
+
+ list_for_each_entry(UA_entry_tmp, &tgt_dev->UA_list,
+ UA_list_entry) {
+ if (memcmp(sense, UA_entry_tmp->UA_sense_buffer, len) == 0) {
+ TRACE_MGMT_DBG("%s", "UA already exists");
+ skip_UA = 1;
+ break;
+ }
+ }
+
+ if (skip_UA == 0)
+ scst_alloc_set_UA(tgt_dev, sense, len, flags);
+ return;
+}
+
+void scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags)
+{
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+ __scst_check_set_UA(tgt_dev, sense, sense_len, flags);
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ return;
+}
+
+/* Called under dev_lock and BH off */
+void scst_dev_check_set_local_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len)
+{
+ struct scst_tgt_dev *tgt_dev, *exclude_tgt_dev = NULL;
+
+ if (exclude != NULL)
+ exclude_tgt_dev = exclude->tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if (tgt_dev != exclude_tgt_dev)
+ scst_check_set_UA(tgt_dev, sense, sense_len, 0);
+ }
+ return;
+}
+
+/* Called under dev_lock and BH off */
+void __scst_dev_check_set_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len)
+{
+
+ TRACE_MGMT_DBG("Processing UA dev %p", dev);
+
+ /* Check for reset UA */
+ if (scst_analyze_sense(sense, sense_len, SCST_SENSE_ASC_VALID,
+ 0, SCST_SENSE_ASC_UA_RESET, 0))
+ scst_process_reset(dev,
+ (exclude != NULL) ? exclude->sess : NULL,
+ exclude, NULL, false);
+
+ scst_dev_check_set_local_UA(dev, exclude, sense, sense_len);
+ return;
+}
+
+/* Called under tgt_dev_lock or when tgt_dev is unused */
+static void scst_free_all_UA(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_tgt_dev_UA *UA_entry, *t;
+
+ list_for_each_entry_safe(UA_entry, t,
+ &tgt_dev->UA_list, UA_list_entry) {
+ TRACE_MGMT_DBG("Clearing UA for tgt_dev LUN %lld",
+ (long long unsigned int)tgt_dev->lun);
+ list_del(&UA_entry->UA_list_entry);
+ mempool_free(UA_entry, scst_ua_mempool);
+ }
+ INIT_LIST_HEAD(&tgt_dev->UA_list);
+ clear_bit(SCST_TGT_DEV_UA_PENDING, &tgt_dev->tgt_dev_flags);
+ return;
+}
+
+/* No locks */
+struct scst_cmd *__scst_check_deferred_commands(struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_cmd *res = NULL, *cmd, *t;
+ typeof(tgt_dev->expected_sn) expected_sn = tgt_dev->expected_sn;
+
+ spin_lock_irq(&tgt_dev->sn_lock);
+
+ if (unlikely(tgt_dev->hq_cmd_count != 0))
+ goto out_unlock;
+
+restart:
+ list_for_each_entry_safe(cmd, t, &tgt_dev->deferred_cmd_list,
+ sn_cmd_list_entry) {
+ EXTRACHECKS_BUG_ON(cmd->queue_type ==
+ SCST_CMD_QUEUE_HEAD_OF_QUEUE);
+ if (cmd->sn == expected_sn) {
+ TRACE_SN("Deferred command %p (sn %d, set %d) found",
+ cmd, cmd->sn, cmd->sn_set);
+ tgt_dev->def_cmd_count--;
+ list_del(&cmd->sn_cmd_list_entry);
+ if (res == NULL)
+ res = cmd;
+ else {
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ TRACE_SN("Adding cmd %p to active cmd list",
+ cmd);
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ }
+ }
+ }
+ if (res != NULL)
+ goto out_unlock;
+
+ list_for_each_entry(cmd, &tgt_dev->skipped_sn_list,
+ sn_cmd_list_entry) {
+ EXTRACHECKS_BUG_ON(cmd->queue_type ==
+ SCST_CMD_QUEUE_HEAD_OF_QUEUE);
+ if (cmd->sn == expected_sn) {
+ atomic_t *slot = cmd->sn_slot;
+ /*
+ * !! At this point any pointer in cmd, except !!
+ * !! sn_slot and sn_cmd_list_entry, could be !!
+ * !! already destroyed !!
+ */
+ TRACE_SN("cmd %p (tag %llu) with skipped sn %d found",
+ cmd,
+ (long long unsigned int)cmd->tag,
+ cmd->sn);
+ tgt_dev->def_cmd_count--;
+ list_del(&cmd->sn_cmd_list_entry);
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ if (test_and_set_bit(SCST_CMD_CAN_BE_DESTROYED,
+ &cmd->cmd_flags))
+ scst_destroy_put_cmd(cmd);
+ scst_inc_expected_sn(tgt_dev, slot);
+ expected_sn = tgt_dev->expected_sn;
+ spin_lock_irq(&tgt_dev->sn_lock);
+ goto restart;
+ }
+ }
+
+out_unlock:
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ return res;
+}
+
+/*****************************************************************
+ ** The following thr_data functions are necessary, because the
+ ** kernel doesn't provide a better way to have threads local
+ ** storage
+ *****************************************************************/
+
+/**
+ * scst_add_thr_data() - add the current thread's local data
+ *
+ * Adds local to the current thread data to tgt_dev
+ * (they will be local for the tgt_dev and current thread).
+ */
+void scst_add_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct scst_thr_data_hdr *data,
+ void (*free_fn) (struct scst_thr_data_hdr *data))
+{
+ data->owner_thr = current;
+ atomic_set(&data->ref, 1);
+ EXTRACHECKS_BUG_ON(free_fn == NULL);
+ data->free_fn = free_fn;
+ spin_lock(&tgt_dev->thr_data_lock);
+ list_add_tail(&data->thr_data_list_entry, &tgt_dev->thr_data_list);
+ spin_unlock(&tgt_dev->thr_data_lock);
+}
+EXPORT_SYMBOL_GPL(scst_add_thr_data);
+
+/**
+ * scst_del_all_thr_data() - delete all thread's local data
+ *
+ * Deletes all local to threads data from tgt_dev
+ */
+void scst_del_all_thr_data(struct scst_tgt_dev *tgt_dev)
+{
+ spin_lock(&tgt_dev->thr_data_lock);
+ while (!list_empty(&tgt_dev->thr_data_list)) {
+ struct scst_thr_data_hdr *d = list_entry(
+ tgt_dev->thr_data_list.next, typeof(*d),
+ thr_data_list_entry);
+ list_del(&d->thr_data_list_entry);
+ spin_unlock(&tgt_dev->thr_data_lock);
+ scst_thr_data_put(d);
+ spin_lock(&tgt_dev->thr_data_lock);
+ }
+ spin_unlock(&tgt_dev->thr_data_lock);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_del_all_thr_data);
+
+/**
+ * scst_dev_del_all_thr_data() - delete all thread's local data from device
+ *
+ * Deletes all local to threads data from all tgt_dev's of the device
+ */
+void scst_dev_del_all_thr_data(struct scst_device *dev)
+{
+ struct scst_tgt_dev *tgt_dev;
+
+ mutex_lock(&scst_mutex);
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ scst_del_all_thr_data(tgt_dev);
+ }
+
+ mutex_unlock(&scst_mutex);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_dev_del_all_thr_data);
+
+/* thr_data_lock supposed to be held */
+static struct scst_thr_data_hdr *__scst_find_thr_data_locked(
+ struct scst_tgt_dev *tgt_dev, struct task_struct *tsk)
+{
+ struct scst_thr_data_hdr *res = NULL, *d;
+
+ list_for_each_entry(d, &tgt_dev->thr_data_list, thr_data_list_entry) {
+ if (d->owner_thr == tsk) {
+ res = d;
+ scst_thr_data_get(res);
+ break;
+ }
+ }
+ return res;
+}
+
+/**
+ * __scst_find_thr_data() - find local to the thread data
+ *
+ * Finds local to the thread data. Returns NULL, if they not found.
+ */
+struct scst_thr_data_hdr *__scst_find_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct task_struct *tsk)
+{
+ struct scst_thr_data_hdr *res;
+
+ spin_lock(&tgt_dev->thr_data_lock);
+ res = __scst_find_thr_data_locked(tgt_dev, tsk);
+ spin_unlock(&tgt_dev->thr_data_lock);
+
+ return res;
+}
+EXPORT_SYMBOL_GPL(__scst_find_thr_data);
+
+bool scst_del_thr_data(struct scst_tgt_dev *tgt_dev, struct task_struct *tsk)
+{
+ bool res;
+ struct scst_thr_data_hdr *td;
+
+ spin_lock(&tgt_dev->thr_data_lock);
+
+ td = __scst_find_thr_data_locked(tgt_dev, tsk);
+ if (td != NULL) {
+ list_del(&td->thr_data_list_entry);
+ res = true;
+ } else
+ res = false;
+
+ spin_unlock(&tgt_dev->thr_data_lock);
+
+ if (td != NULL) {
+ /* the find() fn also gets it */
+ scst_thr_data_put(td);
+ scst_thr_data_put(td);
+ }
+
+ return res;
+}
+
+/* dev_lock supposed to be held and BH disabled */
+void __scst_block_dev(struct scst_device *dev)
+{
+ dev->block_count++;
+ TRACE_MGMT_DBG("Device BLOCK(new %d), dev %p", dev->block_count, dev);
+}
+
+/* No locks */
+static void scst_block_dev(struct scst_device *dev, int outstanding)
+{
+ spin_lock_bh(&dev->dev_lock);
+ __scst_block_dev(dev);
+ spin_unlock_bh(&dev->dev_lock);
+
+ /*
+ * Memory barrier is necessary here, because we need to read
+ * on_dev_count in wait_event() below after we increased block_count.
+ * Otherwise, we can miss wake up in scst_dec_on_dev_cmd().
+ * We use the explicit barrier, because spin_unlock_bh() doesn't
+ * provide the necessary memory barrier functionality.
+ */
+ smp_mb();
+
+ TRACE_MGMT_DBG("Waiting during blocking outstanding %d (on_dev_count "
+ "%d)", outstanding, atomic_read(&dev->on_dev_count));
+ wait_event(dev->on_dev_waitQ,
+ atomic_read(&dev->on_dev_count) <= outstanding);
+ TRACE_MGMT_DBG("%s", "wait_event() returned");
+}
+
+/* No locks */
+void scst_block_dev_cmd(struct scst_cmd *cmd, int outstanding)
+{
+ BUG_ON(cmd->needs_unblocking);
+
+ cmd->needs_unblocking = 1;
+ TRACE_MGMT_DBG("Needs unblocking cmd %p (tag %llu)",
+ cmd, (long long unsigned int)cmd->tag);
+
+ scst_block_dev(cmd->dev, outstanding);
+}
+
+/* No locks */
+void scst_unblock_dev(struct scst_device *dev)
+{
+ spin_lock_bh(&dev->dev_lock);
+ TRACE_MGMT_DBG("Device UNBLOCK(new %d), dev %p",
+ dev->block_count-1, dev);
+ if (--dev->block_count == 0)
+ scst_unblock_cmds(dev);
+ spin_unlock_bh(&dev->dev_lock);
+ BUG_ON(dev->block_count < 0);
+}
+
+/* No locks */
+void scst_unblock_dev_cmd(struct scst_cmd *cmd)
+{
+ scst_unblock_dev(cmd->dev);
+ cmd->needs_unblocking = 0;
+}
+
+/* No locks */
+int scst_inc_on_dev_cmd(struct scst_cmd *cmd)
+{
+ int res = 0;
+ struct scst_device *dev = cmd->dev;
+
+ BUG_ON(cmd->inc_blocking || cmd->dec_on_dev_needed);
+
+ atomic_inc(&dev->on_dev_count);
+ cmd->dec_on_dev_needed = 1;
+ TRACE_DBG("New on_dev_count %d", atomic_read(&dev->on_dev_count));
+
+ if (unlikely(cmd->internal) && (cmd->cdb[0] == REQUEST_SENSE)) {
+ /*
+ * The original command can already block the device, so
+ * REQUEST SENSE command should always pass.
+ */
+ goto out;
+ }
+
+repeat:
+ if (unlikely(dev->block_count > 0)) {
+ spin_lock_bh(&dev->dev_lock);
+ if (unlikely(test_bit(SCST_CMD_ABORTED, &cmd->cmd_flags)))
+ goto out_unlock;
+ if (dev->block_count > 0) {
+ scst_dec_on_dev_cmd(cmd);
+ TRACE_MGMT_DBG("Delaying cmd %p due to blocking "
+ "(tag %llu, dev %p)", cmd,
+ (long long unsigned int)cmd->tag, dev);
+ list_add_tail(&cmd->blocked_cmd_list_entry,
+ &dev->blocked_cmd_list);
+ res = 1;
+ spin_unlock_bh(&dev->dev_lock);
+ goto out;
+ } else {
+ TRACE_MGMT_DBG("%s", "Somebody unblocked the device, "
+ "continuing");
+ }
+ spin_unlock_bh(&dev->dev_lock);
+ }
+ if (unlikely(dev->dev_double_ua_possible)) {
+ spin_lock_bh(&dev->dev_lock);
+ if (dev->block_count == 0) {
+ TRACE_MGMT_DBG("cmd %p (tag %llu), blocking further "
+ "cmds due to possible double reset UA (dev %p)",
+ cmd, (long long unsigned int)cmd->tag, dev);
+ __scst_block_dev(dev);
+ cmd->inc_blocking = 1;
+ } else {
+ spin_unlock_bh(&dev->dev_lock);
+ TRACE_MGMT_DBG("Somebody blocked the device, "
+ "repeating (count %d)", dev->block_count);
+ goto repeat;
+ }
+ spin_unlock_bh(&dev->dev_lock);
+ }
+
+out:
+ return res;
+
+out_unlock:
+ spin_unlock_bh(&dev->dev_lock);
+ goto out;
+}
+
+/* Called under dev_lock */
+static void scst_unblock_cmds(struct scst_device *dev)
+{
+ struct scst_cmd *cmd, *tcmd;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ list_for_each_entry_safe(cmd, tcmd, &dev->blocked_cmd_list,
+ blocked_cmd_list_entry) {
+ list_del(&cmd->blocked_cmd_list_entry);
+ TRACE_MGMT_DBG("Adding blocked cmd %p to active cmd list", cmd);
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ if (unlikely(cmd->queue_type == SCST_CMD_QUEUE_HEAD_OF_QUEUE))
+ list_add(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ else
+ list_add_tail(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ wake_up(&cmd->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ }
+ local_irq_restore(flags);
+ return;
+}
+
+static void __scst_unblock_deferred(struct scst_tgt_dev *tgt_dev,
+ struct scst_cmd *out_of_sn_cmd)
+{
+ EXTRACHECKS_BUG_ON(!out_of_sn_cmd->sn_set);
+
+ if (out_of_sn_cmd->sn == tgt_dev->expected_sn) {
+ scst_inc_expected_sn(tgt_dev, out_of_sn_cmd->sn_slot);
+ scst_make_deferred_commands_active(tgt_dev);
+ } else {
+ out_of_sn_cmd->out_of_sn = 1;
+ spin_lock_irq(&tgt_dev->sn_lock);
+ tgt_dev->def_cmd_count++;
+ list_add_tail(&out_of_sn_cmd->sn_cmd_list_entry,
+ &tgt_dev->skipped_sn_list);
+ TRACE_SN("out_of_sn_cmd %p with sn %d added to skipped_sn_list"
+ " (expected_sn %d)", out_of_sn_cmd, out_of_sn_cmd->sn,
+ tgt_dev->expected_sn);
+ spin_unlock_irq(&tgt_dev->sn_lock);
+ }
+
+ return;
+}
+
+void scst_unblock_deferred(struct scst_tgt_dev *tgt_dev,
+ struct scst_cmd *out_of_sn_cmd)
+{
+
+ if (!out_of_sn_cmd->sn_set) {
+ TRACE_SN("cmd %p without sn", out_of_sn_cmd);
+ goto out;
+ }
+
+ __scst_unblock_deferred(tgt_dev, out_of_sn_cmd);
+
+out:
+ return;
+}
+
+void scst_on_hq_cmd_response(struct scst_cmd *cmd)
+{
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ if (!cmd->hq_cmd_inced)
+ goto out;
+
+ spin_lock_irq(&tgt_dev->sn_lock);
+ tgt_dev->hq_cmd_count--;
+ spin_unlock_irq(&tgt_dev->sn_lock);
+
+ EXTRACHECKS_BUG_ON(tgt_dev->hq_cmd_count < 0);
+
+ /*
+ * There is no problem in checking hq_cmd_count in the
+ * non-locked state. In the worst case we will only have
+ * unneeded run of the deferred commands.
+ */
+ if (tgt_dev->hq_cmd_count == 0)
+ scst_make_deferred_commands_active(tgt_dev);
+
+out:
+ return;
+}
+
+void scst_store_sense(struct scst_cmd *cmd)
+{
+
+ if (SCST_SENSE_VALID(cmd->sense) &&
+ !test_bit(SCST_CMD_NO_RESP, &cmd->cmd_flags) &&
+ (cmd->tgt_dev != NULL)) {
+ struct scst_tgt_dev *tgt_dev = cmd->tgt_dev;
+
+ TRACE_DBG("Storing sense (cmd %p)", cmd);
+
+ spin_lock_bh(&tgt_dev->tgt_dev_lock);
+
+ if (cmd->sense_valid_len <= sizeof(tgt_dev->tgt_dev_sense))
+ tgt_dev->tgt_dev_valid_sense_len = cmd->sense_valid_len;
+ else {
+ tgt_dev->tgt_dev_valid_sense_len = sizeof(tgt_dev->tgt_dev_sense);
+ PRINT_ERROR("Stored sense truncated to size %d "
+ "(needed %d)", tgt_dev->tgt_dev_valid_sense_len,
+ cmd->sense_valid_len);
+ }
+ memcpy(tgt_dev->tgt_dev_sense, cmd->sense,
+ tgt_dev->tgt_dev_valid_sense_len);
+
+ spin_unlock_bh(&tgt_dev->tgt_dev_lock);
+ }
+ return;
+}
+
+void scst_xmit_process_aborted_cmd(struct scst_cmd *cmd)
+{
+
+ TRACE_MGMT_DBG("Aborted cmd %p done (cmd_ref %d, "
+ "scst_cmd_count %d)", cmd, atomic_read(&cmd->cmd_ref),
+ atomic_read(&scst_cmd_count));
+
+ scst_done_cmd_mgmt(cmd);
+
+ if (test_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags)) {
+ if (cmd->completed) {
+ /* It's completed and it's OK to return its result */
+ goto out;
+ }
+
+ /* For not yet inited commands cmd->dev can be NULL here */
+ if (test_bit(SCST_CMD_DEVICE_TAS, &cmd->cmd_flags)) {
+ TRACE_MGMT_DBG("Flag ABORTED OTHER set for cmd %p "
+ "(tag %llu), returning TASK ABORTED ", cmd,
+ (long long unsigned int)cmd->tag);
+ scst_set_cmd_error_status(cmd, SAM_STAT_TASK_ABORTED);
+ } else {
+ TRACE_MGMT_DBG("Flag ABORTED OTHER set for cmd %p "
+ "(tag %llu), aborting without delivery or "
+ "notification",
+ cmd, (long long unsigned int)cmd->tag);
+ /*
+ * There is no need to check/requeue possible UA,
+ * because, if it exists, it will be delivered
+ * by the "completed" branch above.
+ */
+ clear_bit(SCST_CMD_ABORTED_OTHER, &cmd->cmd_flags);
+ }
+ }
+
+out:
+ return;
+}
+
+/**
+ * scst_get_max_lun_commands() - return maximum supported commands count
+ *
+ * Returns maximum commands count which can be queued to this LUN in this
+ * session.
+ *
+ * If lun is NO_SUCH_LUN, returns minimum of maximum commands count which
+ * can be queued to any LUN in this session.
+ *
+ * If sess is NULL, returns minimum of maximum commands count which can be
+ * queued to any SCST device.
+ */
+int scst_get_max_lun_commands(struct scst_session *sess, uint64_t lun)
+{
+ return SCST_MAX_TGT_DEV_COMMANDS;
+}
+EXPORT_SYMBOL_GPL(scst_get_max_lun_commands);
+
+/**
+ * scst_get_next_lexem() - parse and return next lexem in the string
+ *
+ * Returns pointer to the next lexem from token_str skipping
+ * spaces and '=' character and using them then as a delimeter. Content
+ * of token_str is modified by setting '\0' at the delimeter's position.
+ */
+char *scst_get_next_lexem(char **token_str)
+{
+ char *p = *token_str;
+ char *q;
+ static const char blank = '\0';
+
+ if ((token_str == NULL) || (*token_str == NULL))
+ return (char *)␣
+
+ for (p = *token_str; (*p != '\0') && (isspace(*p) || (*p == '=')); p++)
+ ;
+
+ for (q = p; (*q != '\0') && !isspace(*q) && (*q != '='); q++)
+ ;
+
+ if (*q != '\0')
+ *q++ = '\0';
+
+ *token_str = q;
+ return p;
+}
+EXPORT_SYMBOL_GPL(scst_get_next_lexem);
+
+/**
+ * scst_restore_token_str() - restore string modified by scst_get_next_lexem()
+ *
+ * Restores token_str modified by scst_get_next_lexem() to the
+ * previous value before scst_get_next_lexem() was called. Prev_lexem is
+ * a pointer to lexem returned by scst_get_next_lexem().
+ */
+void scst_restore_token_str(char *prev_lexem, char *token_str)
+{
+ if (&prev_lexem[strlen(prev_lexem)] != token_str)
+ prev_lexem[strlen(prev_lexem)] = ' ';
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_restore_token_str);
+
+/**
+ * scst_get_next_token_str() - parse and return next token
+ *
+ * This function returns pointer to the next token strings from input_str
+ * using '\n', ';' and '\0' as a delimeter. Content of input_str is
+ * modified by setting '\0' at the delimeter's position.
+ */
+char *scst_get_next_token_str(char **input_str)
+{
+ char *p = *input_str;
+ int i = 0;
+
+ while ((p[i] != '\n') && (p[i] != ';') && (p[i] != '\0'))
+ i++;
+
+ if (i == 0)
+ return NULL;
+
+ if (p[i] == '\0')
+ *input_str = &p[i];
+ else
+ *input_str = &p[i+1];
+
+ p[i] = '\0';
+
+ return p;
+}
+EXPORT_SYMBOL_GPL(scst_get_next_token_str);
+
+static void __init scst_scsi_op_list_init(void)
+{
+ int i;
+ uint8_t op = 0xff;
+
+ for (i = 0; i < 256; i++)
+ scst_scsi_op_list[i] = SCST_CDB_TBL_SIZE;
+
+ for (i = 0; i < SCST_CDB_TBL_SIZE; i++) {
+ if (scst_scsi_op_table[i].ops != op) {
+ op = scst_scsi_op_table[i].ops;
+ scst_scsi_op_list[op] = i;
+ }
+ }
+ return;
+}
+
+int __init scst_lib_init(void)
+{
+ int res = 0;
+
+ scst_scsi_op_list_init();
+
+ scsi_io_context_cache = kmem_cache_create("scst_scsi_io_context",
+ sizeof(struct scsi_io_context),
+ 0, 0, NULL);
+ if (!scsi_io_context_cache) {
+ PRINT_ERROR("%s", "Can't init scsi io context cache");
+ res = -ENOMEM;
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+void scst_lib_exit(void)
+{
+ BUILD_BUG_ON(SCST_MAX_CDB_SIZE != BLK_MAX_CDB);
+ BUILD_BUG_ON(SCST_SENSE_BUFFERSIZE < SCSI_SENSE_BUFFERSIZE);
+
+ kmem_cache_destroy(scsi_io_context_cache);
+}
+
+#ifdef CONFIG_SCST_DEBUG
+
+/**
+ * scst_random() - return a pseudo-random number for debugging purposes.
+ *
+ * Returns a pseudo-random number for debugging purposes. Available only in
+ * the DEBUG build.
+ *
+ * Original taken from the XFS code
+ */
+unsigned long scst_random(void)
+{
+ static int Inited;
+ static unsigned long RandomValue;
+ static DEFINE_SPINLOCK(lock);
+ /* cycles pseudo-randomly through all values between 1 and 2^31 - 2 */
+ register long rv;
+ register long lo;
+ register long hi;
+ unsigned long flags;
+
+ spin_lock_irqsave(&lock, flags);
+ if (!Inited) {
+ RandomValue = jiffies;
+ Inited = 1;
+ }
+ rv = RandomValue;
+ hi = rv / 127773;
+ lo = rv % 127773;
+ rv = 16807 * lo - 2836 * hi;
+ if (rv <= 0)
+ rv += 2147483647;
+ RandomValue = rv;
+ spin_unlock_irqrestore(&lock, flags);
+ return rv;
+}
+EXPORT_SYMBOL_GPL(scst_random);
+#endif /* CONFIG_SCST_DEBUG */
+
+#ifdef CONFIG_SCST_DEBUG_TM
+
+#define TM_DBG_STATE_ABORT 0
+#define TM_DBG_STATE_RESET 1
+#define TM_DBG_STATE_OFFLINE 2
+
+#define INIT_TM_DBG_STATE TM_DBG_STATE_ABORT
+
+static void tm_dbg_timer_fn(unsigned long arg);
+
+static DEFINE_SPINLOCK(scst_tm_dbg_lock);
+/* All serialized by scst_tm_dbg_lock */
+static struct {
+ unsigned int tm_dbg_release:1;
+ unsigned int tm_dbg_blocked:1;
+} tm_dbg_flags;
+static LIST_HEAD(tm_dbg_delayed_cmd_list);
+static int tm_dbg_delayed_cmds_count;
+static int tm_dbg_passed_cmds_count;
+static int tm_dbg_state;
+static int tm_dbg_on_state_passes;
+static DEFINE_TIMER(tm_dbg_timer, tm_dbg_timer_fn, 0, 0);
+static struct scst_tgt_dev *tm_dbg_tgt_dev;
+
+static const int tm_dbg_on_state_num_passes[] = { 5, 1, 0x7ffffff };
+
+static void tm_dbg_init_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+ if (tgt_dev->lun == 6) {
+ unsigned long flags;
+
+ if (tm_dbg_tgt_dev != NULL)
+ tm_dbg_deinit_tgt_dev(tm_dbg_tgt_dev);
+
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ tm_dbg_state = INIT_TM_DBG_STATE;
+ tm_dbg_on_state_passes =
+ tm_dbg_on_state_num_passes[tm_dbg_state];
+ tm_dbg_tgt_dev = tgt_dev;
+ PRINT_INFO("LUN %lld connected from initiator %s is under "
+ "TM debugging (tgt_dev %p)",
+ (unsigned long long)tgt_dev->lun,
+ tgt_dev->sess->initiator_name, tgt_dev);
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ }
+ return;
+}
+
+static void tm_dbg_deinit_tgt_dev(struct scst_tgt_dev *tgt_dev)
+{
+ if (tm_dbg_tgt_dev == tgt_dev) {
+ unsigned long flags;
+ TRACE_MGMT_DBG("Deinit TM debugging tgt_dev %p", tgt_dev);
+ del_timer_sync(&tm_dbg_timer);
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ tm_dbg_tgt_dev = NULL;
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ }
+ return;
+}
+
+static void tm_dbg_timer_fn(unsigned long arg)
+{
+ TRACE_MGMT_DBG("%s", "delayed cmd timer expired");
+ tm_dbg_flags.tm_dbg_release = 1;
+ /* Used to make sure that all woken up threads see the new value */
+ smp_wmb();
+ wake_up_all(&tm_dbg_tgt_dev->active_cmd_threads->cmd_list_waitQ);
+ return;
+}
+
+/* Called under scst_tm_dbg_lock and IRQs off */
+static void tm_dbg_delay_cmd(struct scst_cmd *cmd)
+{
+ switch (tm_dbg_state) {
+ case TM_DBG_STATE_ABORT:
+ if (tm_dbg_delayed_cmds_count == 0) {
+ unsigned long d = 58*HZ + (scst_random() % (4*HZ));
+ TRACE_MGMT_DBG("STATE ABORT: delaying cmd %p (tag %llu)"
+ " for %ld.%ld seconds (%ld HZ), "
+ "tm_dbg_on_state_passes=%d", cmd, cmd->tag,
+ d/HZ, (d%HZ)*100/HZ, d, tm_dbg_on_state_passes);
+ mod_timer(&tm_dbg_timer, jiffies + d);
+#if 0
+ tm_dbg_flags.tm_dbg_blocked = 1;
+#endif
+ } else {
+ TRACE_MGMT_DBG("Delaying another timed cmd %p "
+ "(tag %llu), delayed_cmds_count=%d, "
+ "tm_dbg_on_state_passes=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count,
+ tm_dbg_on_state_passes);
+ if (tm_dbg_delayed_cmds_count == 2)
+ tm_dbg_flags.tm_dbg_blocked = 0;
+ }
+ break;
+
+ case TM_DBG_STATE_RESET:
+ case TM_DBG_STATE_OFFLINE:
+ TRACE_MGMT_DBG("STATE RESET/OFFLINE: delaying cmd %p "
+ "(tag %llu), delayed_cmds_count=%d, "
+ "tm_dbg_on_state_passes=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count, tm_dbg_on_state_passes);
+ tm_dbg_flags.tm_dbg_blocked = 1;
+ break;
+
+ default:
+ BUG();
+ }
+ /* IRQs already off */
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_add_tail(&cmd->cmd_list_entry, &tm_dbg_delayed_cmd_list);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ cmd->tm_dbg_delayed = 1;
+ tm_dbg_delayed_cmds_count++;
+ return;
+}
+
+/* No locks */
+void tm_dbg_check_released_cmds(void)
+{
+ if (tm_dbg_flags.tm_dbg_release) {
+ struct scst_cmd *cmd, *tc;
+ spin_lock_irq(&scst_tm_dbg_lock);
+ list_for_each_entry_safe_reverse(cmd, tc,
+ &tm_dbg_delayed_cmd_list, cmd_list_entry) {
+ TRACE_MGMT_DBG("Releasing timed cmd %p (tag %llu), "
+ "delayed_cmds_count=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count);
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_move(&cmd->cmd_list_entry,
+ &cmd->cmd_threads->active_cmd_list);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ }
+ tm_dbg_flags.tm_dbg_release = 0;
+ spin_unlock_irq(&scst_tm_dbg_lock);
+ }
+}
+
+/* Called under scst_tm_dbg_lock */
+static void tm_dbg_change_state(void)
+{
+ tm_dbg_flags.tm_dbg_blocked = 0;
+ if (--tm_dbg_on_state_passes == 0) {
+ switch (tm_dbg_state) {
+ case TM_DBG_STATE_ABORT:
+ TRACE_MGMT_DBG("%s", "Changing "
+ "tm_dbg_state to RESET");
+ tm_dbg_state = TM_DBG_STATE_RESET;
+ tm_dbg_flags.tm_dbg_blocked = 0;
+ break;
+ case TM_DBG_STATE_RESET:
+ case TM_DBG_STATE_OFFLINE:
+#ifdef CONFIG_SCST_TM_DBG_GO_OFFLINE
+ TRACE_MGMT_DBG("%s", "Changing "
+ "tm_dbg_state to OFFLINE");
+ tm_dbg_state = TM_DBG_STATE_OFFLINE;
+#else
+ TRACE_MGMT_DBG("%s", "Changing "
+ "tm_dbg_state to ABORT");
+ tm_dbg_state = TM_DBG_STATE_ABORT;
+#endif
+ break;
+ default:
+ BUG();
+ }
+ tm_dbg_on_state_passes =
+ tm_dbg_on_state_num_passes[tm_dbg_state];
+ }
+
+ TRACE_MGMT_DBG("%s", "Deleting timer");
+ del_timer_sync(&tm_dbg_timer);
+ return;
+}
+
+/* No locks */
+int tm_dbg_check_cmd(struct scst_cmd *cmd)
+{
+ int res = 0;
+ unsigned long flags;
+
+ if (cmd->tm_dbg_immut)
+ goto out;
+
+ if (cmd->tm_dbg_delayed) {
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ TRACE_MGMT_DBG("Processing delayed cmd %p (tag %llu), "
+ "delayed_cmds_count=%d", cmd, cmd->tag,
+ tm_dbg_delayed_cmds_count);
+
+ cmd->tm_dbg_immut = 1;
+ tm_dbg_delayed_cmds_count--;
+ if ((tm_dbg_delayed_cmds_count == 0) &&
+ (tm_dbg_state == TM_DBG_STATE_ABORT))
+ tm_dbg_change_state();
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ } else if (cmd->tgt_dev && (tm_dbg_tgt_dev == cmd->tgt_dev)) {
+ /* Delay 50th command */
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ if (tm_dbg_flags.tm_dbg_blocked ||
+ (++tm_dbg_passed_cmds_count % 50) == 0) {
+ tm_dbg_delay_cmd(cmd);
+ res = 1;
+ } else
+ cmd->tm_dbg_immut = 1;
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ }
+
+out:
+ return res;
+}
+
+/* No locks */
+void tm_dbg_release_cmd(struct scst_cmd *cmd)
+{
+ struct scst_cmd *c;
+ unsigned long flags;
+
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ list_for_each_entry(c, &tm_dbg_delayed_cmd_list,
+ cmd_list_entry) {
+ if (c == cmd) {
+ TRACE_MGMT_DBG("Abort request for "
+ "delayed cmd %p (tag=%llu), moving it to "
+ "active cmd list (delayed_cmds_count=%d)",
+ c, c->tag, tm_dbg_delayed_cmds_count);
+
+ if (!test_bit(SCST_CMD_ABORTED_OTHER,
+ &cmd->cmd_flags)) {
+ /* Test how completed commands handled */
+ if (((scst_random() % 10) == 5)) {
+ scst_set_cmd_error(cmd,
+ SCST_LOAD_SENSE(
+ scst_sense_hardw_error));
+ /* It's completed now */
+ }
+ }
+
+ spin_lock(&cmd->cmd_threads->cmd_list_lock);
+ list_move(&c->cmd_list_entry,
+ &c->cmd_threads->active_cmd_list);
+ wake_up(&c->cmd_threads->cmd_list_waitQ);
+ spin_unlock(&cmd->cmd_threads->cmd_list_lock);
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+ return;
+}
+
+/* Might be called under scst_mutex */
+void tm_dbg_task_mgmt(struct scst_device *dev, const char *fn, int force)
+{
+ unsigned long flags;
+
+ if (dev != NULL) {
+ if (tm_dbg_tgt_dev == NULL)
+ goto out;
+
+ if (tm_dbg_tgt_dev->dev != dev)
+ goto out;
+ }
+
+ spin_lock_irqsave(&scst_tm_dbg_lock, flags);
+ if ((tm_dbg_state != TM_DBG_STATE_OFFLINE) || force) {
+ TRACE_MGMT_DBG("%s: freeing %d delayed cmds", fn,
+ tm_dbg_delayed_cmds_count);
+ tm_dbg_change_state();
+ tm_dbg_flags.tm_dbg_release = 1;
+ /*
+ * Used to make sure that all woken up threads see the new
+ * value.
+ */
+ smp_wmb();
+ if (tm_dbg_tgt_dev != NULL)
+ wake_up_all(&tm_dbg_tgt_dev->active_cmd_threads->cmd_list_waitQ);
+ } else {
+ TRACE_MGMT_DBG("%s: while OFFLINE state, doing nothing", fn);
+ }
+ spin_unlock_irqrestore(&scst_tm_dbg_lock, flags);
+
+out:
+ return;
+}
+
+int tm_dbg_is_release(void)
+{
+ return tm_dbg_flags.tm_dbg_release;
+}
+#endif /* CONFIG_SCST_DEBUG_TM */
+
+#ifdef CONFIG_SCST_DEBUG_SN
+void scst_check_debug_sn(struct scst_cmd *cmd)
+{
+ static DEFINE_SPINLOCK(lock);
+ static int type;
+ static int cnt;
+ unsigned long flags;
+ int old = cmd->queue_type;
+
+ spin_lock_irqsave(&lock, flags);
+
+ if (cnt == 0) {
+ if ((scst_random() % 1000) == 500) {
+ if ((scst_random() % 3) == 1)
+ type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ else
+ type = SCST_CMD_QUEUE_ORDERED;
+ do {
+ cnt = scst_random() % 10;
+ } while (cnt == 0);
+ } else
+ goto out_unlock;
+ }
+
+ cmd->queue_type = type;
+ cnt--;
+
+ if (((scst_random() % 1000) == 750))
+ cmd->queue_type = SCST_CMD_QUEUE_ORDERED;
+ else if (((scst_random() % 1000) == 751))
+ cmd->queue_type = SCST_CMD_QUEUE_HEAD_OF_QUEUE;
+ else if (((scst_random() % 1000) == 752))
+ cmd->queue_type = SCST_CMD_QUEUE_SIMPLE;
+
+ TRACE_SN("DbgSN changed cmd %p: %d/%d (cnt %d)", cmd, old,
+ cmd->queue_type, cnt);
+
+out_unlock:
+ spin_unlock_irqrestore(&lock, flags);
+ return;
+}
+#endif /* CONFIG_SCST_DEBUG_SN */
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+static uint64_t scst_get_nsec(void)
+{
+ struct timespec ts;
+ ktime_get_ts(&ts);
+ return (uint64_t)ts.tv_sec * 1000000000 + ts.tv_nsec;
+}
+
+void scst_set_start_time(struct scst_cmd *cmd)
+{
+ cmd->start = scst_get_nsec();
+ TRACE_DBG("cmd %p: start %lld", cmd, cmd->start);
+}
+
+void scst_set_cur_start(struct scst_cmd *cmd)
+{
+ cmd->curr_start = scst_get_nsec();
+ TRACE_DBG("cmd %p: cur_start %lld", cmd, cmd->curr_start);
+}
+
+void scst_set_parse_time(struct scst_cmd *cmd)
+{
+ cmd->parse_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: parse_time %lld", cmd, cmd->parse_time);
+}
+
+void scst_set_alloc_buf_time(struct scst_cmd *cmd)
+{
+ cmd->alloc_buf_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: alloc_buf_time %lld", cmd, cmd->alloc_buf_time);
+}
+
+void scst_set_restart_waiting_time(struct scst_cmd *cmd)
+{
+ cmd->restart_waiting_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: restart_waiting_time %lld", cmd,
+ cmd->restart_waiting_time);
+}
+
+void scst_set_rdy_to_xfer_time(struct scst_cmd *cmd)
+{
+ cmd->rdy_to_xfer_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: rdy_to_xfer_time %lld", cmd, cmd->rdy_to_xfer_time);
+}
+
+void scst_set_pre_exec_time(struct scst_cmd *cmd)
+{
+ cmd->pre_exec_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: pre_exec_time %lld", cmd, cmd->pre_exec_time);
+}
+
+void scst_set_exec_time(struct scst_cmd *cmd)
+{
+ cmd->exec_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: exec_time %lld", cmd, cmd->exec_time);
+}
+
+void scst_set_dev_done_time(struct scst_cmd *cmd)
+{
+ cmd->dev_done_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: dev_done_time %lld", cmd, cmd->dev_done_time);
+}
+
+void scst_set_xmit_time(struct scst_cmd *cmd)
+{
+ cmd->xmit_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: xmit_time %lld", cmd, cmd->xmit_time);
+}
+
+void scst_set_tgt_on_free_time(struct scst_cmd *cmd)
+{
+ cmd->tgt_on_free_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: tgt_on_free_time %lld", cmd, cmd->tgt_on_free_time);
+}
+
+void scst_set_dev_on_free_time(struct scst_cmd *cmd)
+{
+ cmd->dev_on_free_time += scst_get_nsec() - cmd->curr_start;
+ TRACE_DBG("cmd %p: dev_on_free_time %lld", cmd, cmd->dev_on_free_time);
+}
+
+void scst_update_lat_stats(struct scst_cmd *cmd)
+{
+ uint64_t finish, scst_time, tgt_time, dev_time;
+ struct scst_session *sess = cmd->sess;
+ int data_len;
+ int i;
+ struct scst_ext_latency_stat *latency_stat, *dev_latency_stat;
+
+ finish = scst_get_nsec();
+
+ /* Determine the IO size for extended latency statistics */
+ data_len = cmd->bufflen;
+ i = SCST_LATENCY_STAT_INDEX_OTHER;
+ if (data_len <= SCST_IO_SIZE_THRESHOLD_SMALL)
+ i = SCST_LATENCY_STAT_INDEX_SMALL;
+ else if (data_len <= SCST_IO_SIZE_THRESHOLD_MEDIUM)
+ i = SCST_LATENCY_STAT_INDEX_MEDIUM;
+ else if (data_len <= SCST_IO_SIZE_THRESHOLD_LARGE)
+ i = SCST_LATENCY_STAT_INDEX_LARGE;
+ else if (data_len <= SCST_IO_SIZE_THRESHOLD_VERY_LARGE)
+ i = SCST_LATENCY_STAT_INDEX_VERY_LARGE;
+ latency_stat = &sess->sess_latency_stat[i];
+ dev_latency_stat = &cmd->tgt_dev->dev_latency_stat[i];
+
+ spin_lock_bh(&sess->lat_lock);
+
+ /* Calculate the latencies */
+ scst_time = finish - cmd->start - (cmd->parse_time +
+ cmd->alloc_buf_time + cmd->restart_waiting_time +
+ cmd->rdy_to_xfer_time + cmd->pre_exec_time +
+ cmd->exec_time + cmd->dev_done_time + cmd->xmit_time +
+ cmd->tgt_on_free_time + cmd->dev_on_free_time);
+ tgt_time = cmd->alloc_buf_time + cmd->restart_waiting_time +
+ cmd->rdy_to_xfer_time + cmd->pre_exec_time +
+ cmd->xmit_time + cmd->tgt_on_free_time;
+ dev_time = cmd->parse_time + cmd->exec_time + cmd->dev_done_time +
+ cmd->dev_on_free_time;
+
+ /* Save the basic latency information */
+ sess->scst_time += scst_time;
+ sess->tgt_time += tgt_time;
+ sess->dev_time += dev_time;
+ sess->processed_cmds++;
+
+ if ((sess->min_scst_time == 0) ||
+ (sess->min_scst_time > scst_time))
+ sess->min_scst_time = scst_time;
+ if ((sess->min_tgt_time == 0) ||
+ (sess->min_tgt_time > tgt_time))
+ sess->min_tgt_time = tgt_time;
+ if ((sess->min_dev_time == 0) ||
+ (sess->min_dev_time > dev_time))
+ sess->min_dev_time = dev_time;
+
+ if (sess->max_scst_time < scst_time)
+ sess->max_scst_time = scst_time;
+ if (sess->max_tgt_time < tgt_time)
+ sess->max_tgt_time = tgt_time;
+ if (sess->max_dev_time < dev_time)
+ sess->max_dev_time = dev_time;
+
+ /* Save the extended latency information */
+ if (cmd->data_direction & SCST_DATA_READ) {
+ latency_stat->scst_time_rd += scst_time;
+ latency_stat->tgt_time_rd += tgt_time;
+ latency_stat->dev_time_rd += dev_time;
+ latency_stat->processed_cmds_rd++;
+
+ if ((latency_stat->min_scst_time_rd == 0) ||
+ (latency_stat->min_scst_time_rd > scst_time))
+ latency_stat->min_scst_time_rd = scst_time;
+ if ((latency_stat->min_tgt_time_rd == 0) ||
+ (latency_stat->min_tgt_time_rd > tgt_time))
+ latency_stat->min_tgt_time_rd = tgt_time;
+ if ((latency_stat->min_dev_time_rd == 0) ||
+ (latency_stat->min_dev_time_rd > dev_time))
+ latency_stat->min_dev_time_rd = dev_time;
+
+ if (latency_stat->max_scst_time_rd < scst_time)
+ latency_stat->max_scst_time_rd = scst_time;
+ if (latency_stat->max_tgt_time_rd < tgt_time)
+ latency_stat->max_tgt_time_rd = tgt_time;
+ if (latency_stat->max_dev_time_rd < dev_time)
+ latency_stat->max_dev_time_rd = dev_time;
+
+ dev_latency_stat->scst_time_rd += scst_time;
+ dev_latency_stat->tgt_time_rd += tgt_time;
+ dev_latency_stat->dev_time_rd += dev_time;
+ dev_latency_stat->processed_cmds_rd++;
+
+ if ((dev_latency_stat->min_scst_time_rd == 0) ||
+ (dev_latency_stat->min_scst_time_rd > scst_time))
+ dev_latency_stat->min_scst_time_rd = scst_time;
+ if ((dev_latency_stat->min_tgt_time_rd == 0) ||
+ (dev_latency_stat->min_tgt_time_rd > tgt_time))
+ dev_latency_stat->min_tgt_time_rd = tgt_time;
+ if ((dev_latency_stat->min_dev_time_rd == 0) ||
+ (dev_latency_stat->min_dev_time_rd > dev_time))
+ dev_latency_stat->min_dev_time_rd = dev_time;
+
+ if (dev_latency_stat->max_scst_time_rd < scst_time)
+ dev_latency_stat->max_scst_time_rd = scst_time;
+ if (dev_latency_stat->max_tgt_time_rd < tgt_time)
+ dev_latency_stat->max_tgt_time_rd = tgt_time;
+ if (dev_latency_stat->max_dev_time_rd < dev_time)
+ dev_latency_stat->max_dev_time_rd = dev_time;
+ } else if (cmd->data_direction & SCST_DATA_WRITE) {
+ latency_stat->scst_time_wr += scst_time;
+ latency_stat->tgt_time_wr += tgt_time;
+ latency_stat->dev_time_wr += dev_time;
+ latency_stat->processed_cmds_wr++;
+
+ if ((latency_stat->min_scst_time_wr == 0) ||
+ (latency_stat->min_scst_time_wr > scst_time))
+ latency_stat->min_scst_time_wr = scst_time;
+ if ((latency_stat->min_tgt_time_wr == 0) ||
+ (latency_stat->min_tgt_time_wr > tgt_time))
+ latency_stat->min_tgt_time_wr = tgt_time;
+ if ((latency_stat->min_dev_time_wr == 0) ||
+ (latency_stat->min_dev_time_wr > dev_time))
+ latency_stat->min_dev_time_wr = dev_time;
+
+ if (latency_stat->max_scst_time_wr < scst_time)
+ latency_stat->max_scst_time_wr = scst_time;
+ if (latency_stat->max_tgt_time_wr < tgt_time)
+ latency_stat->max_tgt_time_wr = tgt_time;
+ if (latency_stat->max_dev_time_wr < dev_time)
+ latency_stat->max_dev_time_wr = dev_time;
+
+ dev_latency_stat->scst_time_wr += scst_time;
+ dev_latency_stat->tgt_time_wr += tgt_time;
+ dev_latency_stat->dev_time_wr += dev_time;
+ dev_latency_stat->processed_cmds_wr++;
+
+ if ((dev_latency_stat->min_scst_time_wr == 0) ||
+ (dev_latency_stat->min_scst_time_wr > scst_time))
+ dev_latency_stat->min_scst_time_wr = scst_time;
+ if ((dev_latency_stat->min_tgt_time_wr == 0) ||
+ (dev_latency_stat->min_tgt_time_wr > tgt_time))
+ dev_latency_stat->min_tgt_time_wr = tgt_time;
+ if ((dev_latency_stat->min_dev_time_wr == 0) ||
+ (dev_latency_stat->min_dev_time_wr > dev_time))
+ dev_latency_stat->min_dev_time_wr = dev_time;
+
+ if (dev_latency_stat->max_scst_time_wr < scst_time)
+ dev_latency_stat->max_scst_time_wr = scst_time;
+ if (dev_latency_stat->max_tgt_time_wr < tgt_time)
+ dev_latency_stat->max_tgt_time_wr = tgt_time;
+ if (dev_latency_stat->max_dev_time_wr < dev_time)
+ dev_latency_stat->max_dev_time_wr = dev_time;
+ }
+
+ spin_unlock_bh(&sess->lat_lock);
+
+ TRACE_DBG("cmd %p: finish %lld, scst_time %lld, "
+ "tgt_time %lld, dev_time %lld", cmd, finish, scst_time,
+ tgt_time, dev_time);
+ return;
+}
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 6/12/1/5] SCST core's private header
[not found] ` <4BC44D08.4060907@vlnb.net>
` (4 preceding siblings ...)
2010-04-13 13:05 ` [PATCH][RFC 5/12/1/5] SCST core's scst_lib.c Vladislav Bolkhovitin
@ 2010-04-13 13:06 ` Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 7/12/1/5] SCST SGV cache Vladislav Bolkhovitin
` (3 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:06 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains file scst_priv.h, which contains internal SCST types, constants
and declarations.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_priv.h | 609 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 609 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/scst_priv.h linux-2.6.33/drivers/scst/scst_priv.h
--- orig/linux-2.6.33/drivers/scst/scst_priv.h
+++ linux-2.6.33/drivers/scst/scst_priv.h
@@ -0,0 +1,609 @@
+/*
+ * scst_priv.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_PRIV_H
+#define __SCST_PRIV_H
+
+#include <linux/types.h>
+
+#include <scsi/scsi.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_driver.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_host.h>
+
+#define LOG_PREFIX "scst"
+
+#include "scst_debug.h"
+
+#define TRACE_RTRY 0x80000000
+#define TRACE_SCSI_SERIALIZING 0x40000000
+/** top being the edge away from the interupt */
+#define TRACE_SND_TOP 0x20000000
+#define TRACE_RCV_TOP 0x01000000
+/** bottom being the edge toward the interupt */
+#define TRACE_SND_BOT 0x08000000
+#define TRACE_RCV_BOT 0x04000000
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+#define trace_flag scst_trace_flag
+extern unsigned long scst_trace_flag;
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+
+#define SCST_DEFAULT_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_MINOR | TRACE_PID | \
+ TRACE_LINE | TRACE_FUNCTION | TRACE_SPECIAL | TRACE_MGMT | \
+ TRACE_MGMT_DEBUG | TRACE_RTRY)
+
+#define TRACE_RETRY(args...) TRACE_DBG_FLAG(TRACE_RTRY, args)
+#define TRACE_SN(args...) TRACE_DBG_FLAG(TRACE_SCSI_SERIALIZING, args)
+#define TRACE_SEND_TOP(args...) TRACE_DBG_FLAG(TRACE_SND_TOP, args)
+#define TRACE_RECV_TOP(args...) TRACE_DBG_FLAG(TRACE_RCV_TOP, args)
+#define TRACE_SEND_BOT(args...) TRACE_DBG_FLAG(TRACE_SND_BOT, args)
+#define TRACE_RECV_BOT(args...) TRACE_DBG_FLAG(TRACE_RCV_BOT, args)
+
+#else /* CONFIG_SCST_DEBUG */
+
+# ifdef CONFIG_SCST_TRACING
+#define SCST_DEFAULT_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_MGMT | \
+ TRACE_SPECIAL)
+# else
+#define SCST_DEFAULT_LOG_FLAGS 0
+# endif
+
+#define TRACE_RETRY(args...)
+#define TRACE_SN(args...)
+#define TRACE_SEND_TOP(args...)
+#define TRACE_RECV_TOP(args...)
+#define TRACE_SEND_BOT(args...)
+#define TRACE_RECV_BOT(args...)
+
+#endif
+
+/**
+ ** Bits for scst_flags
+ **/
+
+/*
+ * Set if new commands initialization is being suspended for a while.
+ * Used to let TM commands execute while preparing the suspend, since
+ * RESET or ABORT could be necessary to free SCSI commands.
+ */
+#define SCST_FLAG_SUSPENDING 0
+
+/* Set if new commands initialization is suspended for a while */
+#define SCST_FLAG_SUSPENDED 1
+
+/**
+ ** Return codes for cmd state process functions. Codes are the same as
+ ** for SCST_EXEC_* to avoid translation to them and, hence, have better code.
+ **/
+#define SCST_CMD_STATE_RES_CONT_NEXT SCST_EXEC_COMPLETED
+#define SCST_CMD_STATE_RES_CONT_SAME SCST_EXEC_NOT_COMPLETED
+#define SCST_CMD_STATE_RES_NEED_THREAD SCST_EXEC_NEED_THREAD
+
+/**
+ ** Maximum count of uncompleted commands that an initiator could
+ ** queue on any device. Then it will start getting TASK QUEUE FULL status.
+ **/
+#define SCST_MAX_TGT_DEV_COMMANDS 48
+
+/**
+ ** Maximum count of uncompleted commands that could be queued on any device.
+ ** Then initiators sending commands to this device will start getting
+ ** TASK QUEUE FULL status.
+ **/
+#define SCST_MAX_DEV_COMMANDS 256
+
+#define SCST_TGT_RETRY_TIMEOUT (3/2*HZ)
+
+/* Definitions of symbolic constants for LUN addressing method */
+#define SCST_LUN_ADDR_METHOD_PERIPHERAL 0
+#define SCST_LUN_ADDR_METHOD_FLAT 1
+
+extern int scst_threads;
+
+extern unsigned int scst_max_dev_cmd_mem;
+
+extern mempool_t *scst_mgmt_mempool;
+extern mempool_t *scst_mgmt_stub_mempool;
+extern mempool_t *scst_ua_mempool;
+extern mempool_t *scst_sense_mempool;
+extern mempool_t *scst_aen_mempool;
+
+extern struct kmem_cache *scst_cmd_cachep;
+extern struct kmem_cache *scst_sess_cachep;
+extern struct kmem_cache *scst_tgtd_cachep;
+extern struct kmem_cache *scst_acgd_cachep;
+
+extern spinlock_t scst_main_lock;
+
+extern struct scst_sgv_pools scst_sgv;
+
+extern unsigned long scst_flags;
+extern atomic_t scst_cmd_count;
+extern struct list_head scst_template_list;
+extern struct list_head scst_dev_list;
+extern struct list_head scst_dev_type_list;
+extern wait_queue_head_t scst_dev_cmd_waitQ;
+
+extern unsigned int scst_setup_id;
+
+extern spinlock_t scst_init_lock;
+extern struct list_head scst_init_cmd_list;
+extern wait_queue_head_t scst_init_cmd_list_waitQ;
+extern unsigned int scst_init_poll_cnt;
+
+extern struct scst_cmd_threads scst_main_cmd_threads;
+
+extern spinlock_t scst_mcmd_lock;
+/* The following lists protected by scst_mcmd_lock */
+extern struct list_head scst_active_mgmt_cmd_list;
+extern struct list_head scst_delayed_mgmt_cmd_list;
+extern wait_queue_head_t scst_mgmt_cmd_list_waitQ;
+
+struct scst_tasklet {
+ spinlock_t tasklet_lock;
+ struct list_head tasklet_cmd_list;
+ struct tasklet_struct tasklet;
+};
+extern struct scst_tasklet scst_tasklets[NR_CPUS];
+
+extern wait_queue_head_t scst_mgmt_waitQ;
+extern spinlock_t scst_mgmt_lock;
+extern struct list_head scst_sess_init_list;
+extern struct list_head scst_sess_shut_list;
+
+struct scst_cmd_thread_t {
+ struct task_struct *cmd_thread;
+ struct list_head thread_list_entry;
+};
+
+static inline bool scst_set_io_context(struct scst_cmd *cmd,
+ struct io_context **old)
+{
+ bool res;
+
+ if (cmd->cmd_threads == &scst_main_cmd_threads) {
+ EXTRACHECKS_BUG_ON(in_interrupt());
+ /*
+ * No need for any ref counting action, because io_context
+ * supposed to be cleared in the end of the caller function.
+ */
+ current->io_context = cmd->tgt_dev->async_io_context;
+ res = true;
+ TRACE_DBG("io_context %p (tgt_dev %p)", current->io_context,
+ cmd->tgt_dev);
+ EXTRACHECKS_BUG_ON(current->io_context == NULL);
+ } else
+ res = false;
+
+ return res;
+}
+
+static inline void scst_reset_io_context(struct scst_tgt_dev *tgt_dev,
+ struct io_context *old)
+{
+ current->io_context = old;
+ TRACE_DBG("io_context %p reset", current->io_context);
+ return;
+}
+
+/*
+ * Converts string presentation of threads pool type to enum.
+ * Returns SCST_THREADS_POOL_TYPE_INVALID if the string is invalid.
+ */
+extern enum scst_dev_type_threads_pool_type scst_parse_threads_pool_type(
+ const char *p, int len);
+
+extern int scst_add_threads(struct scst_cmd_threads *cmd_threads,
+ struct scst_device *dev, struct scst_tgt_dev *tgt_dev, int num);
+extern void scst_del_threads(struct scst_cmd_threads *cmd_threads, int num);
+
+extern int scst_create_dev_threads(struct scst_device *dev);
+extern void scst_stop_dev_threads(struct scst_device *dev);
+
+extern int scst_tgt_dev_setup_threads(struct scst_tgt_dev *tgt_dev);
+extern void scst_tgt_dev_stop_threads(struct scst_tgt_dev *tgt_dev);
+
+extern bool scst_del_thr_data(struct scst_tgt_dev *tgt_dev,
+ struct task_struct *tsk);
+
+extern struct scst_dev_type scst_null_devtype;
+
+extern struct scst_cmd *__scst_check_deferred_commands(
+ struct scst_tgt_dev *tgt_dev);
+
+/* Used to save the function call on the fast path */
+static inline struct scst_cmd *scst_check_deferred_commands(
+ struct scst_tgt_dev *tgt_dev)
+{
+ if (tgt_dev->def_cmd_count == 0)
+ return NULL;
+ else
+ return __scst_check_deferred_commands(tgt_dev);
+}
+
+static inline void scst_make_deferred_commands_active(
+ struct scst_tgt_dev *tgt_dev)
+{
+ struct scst_cmd *c;
+
+ c = __scst_check_deferred_commands(tgt_dev);
+ if (c != NULL) {
+ TRACE_SN("Adding cmd %p to active cmd list", c);
+ spin_lock_irq(&c->cmd_threads->cmd_list_lock);
+ list_add_tail(&c->cmd_list_entry,
+ &c->cmd_threads->active_cmd_list);
+ wake_up(&c->cmd_threads->cmd_list_waitQ);
+ spin_unlock_irq(&c->cmd_threads->cmd_list_lock);
+ }
+
+ return;
+}
+
+void scst_inc_expected_sn(struct scst_tgt_dev *tgt_dev, atomic_t *slot);
+int scst_check_hq_cmd(struct scst_cmd *cmd);
+
+void scst_unblock_deferred(struct scst_tgt_dev *tgt_dev,
+ struct scst_cmd *cmd_sn);
+
+void scst_on_hq_cmd_response(struct scst_cmd *cmd);
+void scst_xmit_process_aborted_cmd(struct scst_cmd *cmd);
+
+int scst_cmd_thread(void *arg);
+void scst_cmd_tasklet(long p);
+int scst_init_thread(void *arg);
+int scst_tm_thread(void *arg);
+int scst_global_mgmt_thread(void *arg);
+
+int scst_queue_retry_cmd(struct scst_cmd *cmd, int finished_cmds);
+
+static inline void scst_tgtt_cleanup(struct scst_tgt_template *tgtt) { }
+static inline void scst_devt_cleanup(struct scst_dev_type *devt) { }
+
+int scst_alloc_tgt(struct scst_tgt_template *tgtt, struct scst_tgt **tgt);
+void scst_free_tgt(struct scst_tgt *tgt);
+
+int scst_alloc_device(gfp_t gfp_mask, struct scst_device **out_dev);
+void scst_free_device(struct scst_device *dev);
+
+struct scst_acg *scst_alloc_add_acg(struct scst_tgt *tgt,
+ const char *acg_name);
+void scst_clear_acg(struct scst_acg *acg);
+void scst_destroy_acg(struct scst_acg *acg);
+void scst_free_acg(struct scst_acg *acg);
+
+struct scst_acg *scst_tgt_find_acg(struct scst_tgt *tgt, const char *name);
+
+struct scst_acg *scst_find_acg(const struct scst_session *sess);
+
+void scst_check_reassign_sessions(void);
+
+int scst_sess_alloc_tgt_devs(struct scst_session *sess);
+void scst_sess_free_tgt_devs(struct scst_session *sess);
+void scst_nexus_loss(struct scst_tgt_dev *tgt_dev, bool queue_UA);
+
+int scst_acg_add_dev(struct scst_acg *acg, struct scst_device *dev,
+ uint64_t lun, int read_only, bool gen_scst_report_luns_changed);
+int scst_acg_remove_dev(struct scst_acg *acg, struct scst_device *dev,
+ bool gen_scst_report_luns_changed);
+
+void scst_acg_dev_destroy(struct scst_acg_dev *acg_dev);
+
+int scst_acg_add_name(struct scst_acg *acg, const char *name);
+void scst_acg_remove_acn(struct scst_acn *acn);
+struct scst_acn *scst_acg_find_name(struct scst_acg *acg, const char *name);
+
+/* The activity supposed to be suspended and scst_mutex held */
+static inline bool scst_acg_sess_is_empty(struct scst_acg *acg)
+{
+ return list_empty(&acg->acg_sess_list);
+}
+
+int scst_prepare_request_sense(struct scst_cmd *orig_cmd);
+int scst_finish_internal_cmd(struct scst_cmd *cmd);
+
+void scst_store_sense(struct scst_cmd *cmd);
+
+int scst_assign_dev_handler(struct scst_device *dev,
+ struct scst_dev_type *handler);
+
+struct scst_session *scst_alloc_session(struct scst_tgt *tgt, gfp_t gfp_mask,
+ const char *initiator_name);
+void scst_free_session(struct scst_session *sess);
+void scst_release_session(struct scst_session *sess);
+void scst_free_session_callback(struct scst_session *sess);
+
+struct scst_cmd *scst_alloc_cmd(gfp_t gfp_mask);
+void scst_free_cmd(struct scst_cmd *cmd);
+static inline void scst_destroy_cmd(struct scst_cmd *cmd)
+{
+ kmem_cache_free(scst_cmd_cachep, cmd);
+ return;
+}
+
+void scst_check_retries(struct scst_tgt *tgt);
+
+int scst_scsi_exec_async(struct scst_cmd *cmd,
+ void (*done)(void *, char *, int, int));
+
+int scst_alloc_space(struct scst_cmd *cmd);
+
+int scst_lib_init(void);
+void scst_lib_exit(void);
+
+uint64_t scst_pack_lun(const uint64_t lun, unsigned int addr_method);
+uint64_t scst_unpack_lun(const uint8_t *lun, int len);
+
+struct scst_mgmt_cmd *scst_alloc_mgmt_cmd(gfp_t gfp_mask);
+void scst_free_mgmt_cmd(struct scst_mgmt_cmd *mcmd);
+void scst_done_cmd_mgmt(struct scst_cmd *cmd);
+
+int scst_sysfs_init(void);
+void scst_sysfs_cleanup(void);
+int scst_create_tgtt_sysfs(struct scst_tgt_template *tgtt);
+void scst_tgtt_sysfs_put(struct scst_tgt_template *tgtt);
+int scst_create_tgt_sysfs(struct scst_tgt *tgt);
+void scst_tgt_sysfs_prepare_put(struct scst_tgt *tgt);
+void scst_tgt_sysfs_put(struct scst_tgt *tgt);
+int scst_create_sess_sysfs(struct scst_session *sess);
+void scst_sess_sysfs_put(struct scst_session *sess);
+int scst_create_sgv_sysfs(struct sgv_pool *pool);
+void scst_sgv_sysfs_put(struct sgv_pool *pool);
+int scst_create_devt_sysfs(struct scst_dev_type *devt);
+void scst_devt_sysfs_put(struct scst_dev_type *devt);
+int scst_create_device_sysfs(struct scst_device *dev);
+void scst_device_sysfs_put(struct scst_device *dev);
+int scst_create_devt_dev_sysfs(struct scst_device *dev);
+void scst_devt_dev_sysfs_put(struct scst_device *dev);
+int scst_create_acn_sysfs(struct scst_acg *acg, struct scst_acn *acn);
+void scst_acn_sysfs_del(struct scst_acg *acg, struct scst_acn *acn,
+ bool reassign);
+
+void scst_acg_sysfs_put(struct scst_acg *acg);
+
+int scst_get_cdb_len(const uint8_t *cdb);
+
+void __scst_dev_check_set_UA(struct scst_device *dev, struct scst_cmd *exclude,
+ const uint8_t *sense, int sense_len);
+static inline void scst_dev_check_set_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len)
+{
+ spin_lock_bh(&dev->dev_lock);
+ __scst_dev_check_set_UA(dev, exclude, sense, sense_len);
+ spin_unlock_bh(&dev->dev_lock);
+ return;
+}
+void scst_dev_check_set_local_UA(struct scst_device *dev,
+ struct scst_cmd *exclude, const uint8_t *sense, int sense_len);
+
+#define SCST_SET_UA_FLAG_AT_HEAD 1
+#define SCST_SET_UA_FLAG_GLOBAL 2
+
+void scst_check_set_UA(struct scst_tgt_dev *tgt_dev,
+ const uint8_t *sense, int sense_len, int flags);
+int scst_set_pending_UA(struct scst_cmd *cmd);
+
+void scst_report_luns_changed(struct scst_acg *acg);
+
+void scst_abort_cmd(struct scst_cmd *cmd, struct scst_mgmt_cmd *mcmd,
+ int other_ini, int call_dev_task_mgmt_fn);
+void scst_process_reset(struct scst_device *dev,
+ struct scst_session *originator, struct scst_cmd *exclude_cmd,
+ struct scst_mgmt_cmd *mcmd, bool setUA);
+
+bool scst_is_ua_global(const uint8_t *sense, int len);
+void scst_requeue_ua(struct scst_cmd *cmd);
+
+void scst_gen_aen_or_ua(struct scst_tgt_dev *tgt_dev,
+ int key, int asc, int ascq);
+
+static inline bool scst_is_implicit_hq(struct scst_cmd *cmd)
+{
+ return (cmd->op_flags & SCST_IMPLICIT_HQ) != 0;
+}
+
+/*
+ * Some notes on devices "blocking". Blocking means that no
+ * commands will go from SCST to underlying SCSI device until it
+ * is unblocked. But we don't care about all commands that
+ * already on the device.
+ */
+
+extern int scst_inc_on_dev_cmd(struct scst_cmd *cmd);
+
+extern void __scst_block_dev(struct scst_device *dev);
+extern void scst_block_dev_cmd(struct scst_cmd *cmd, int outstanding);
+extern void scst_unblock_dev(struct scst_device *dev);
+extern void scst_unblock_dev_cmd(struct scst_cmd *cmd);
+
+/* No locks */
+static inline void scst_dec_on_dev_cmd(struct scst_cmd *cmd)
+{
+ struct scst_device *dev = cmd->dev;
+ bool unblock_dev = cmd->inc_blocking;
+
+ if (cmd->inc_blocking) {
+ TRACE_MGMT_DBG("cmd %p (tag %llu): unblocking dev %p", cmd,
+ (long long unsigned int)cmd->tag, cmd->dev);
+ cmd->inc_blocking = 0;
+ }
+ cmd->dec_on_dev_needed = 0;
+
+ if (unblock_dev)
+ scst_unblock_dev(dev);
+
+ atomic_dec(&dev->on_dev_count);
+ /* See comment in scst_block_dev() */
+ smp_mb__after_atomic_dec();
+
+ TRACE_DBG("New on_dev_count %d", atomic_read(&dev->on_dev_count));
+
+ BUG_ON(atomic_read(&dev->on_dev_count) < 0);
+
+ if (unlikely(dev->block_count != 0))
+ wake_up_all(&dev->on_dev_waitQ);
+
+ return;
+}
+
+static inline void __scst_get(int barrier)
+{
+ atomic_inc(&scst_cmd_count);
+ TRACE_DBG("Incrementing scst_cmd_count(new value %d)",
+ atomic_read(&scst_cmd_count));
+
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ if (barrier)
+ smp_mb__after_atomic_inc();
+}
+
+static inline void __scst_put(void)
+{
+ int f;
+ f = atomic_dec_and_test(&scst_cmd_count);
+ /* See comment about smp_mb() in scst_suspend_activity() */
+ if (f && unlikely(test_bit(SCST_FLAG_SUSPENDED, &scst_flags))) {
+ TRACE_MGMT_DBG("%s", "Waking up scst_dev_cmd_waitQ");
+ wake_up_all(&scst_dev_cmd_waitQ);
+ }
+ TRACE_DBG("Decrementing scst_cmd_count(new value %d)",
+ atomic_read(&scst_cmd_count));
+}
+
+void scst_sched_session_free(struct scst_session *sess);
+
+static inline void scst_sess_get(struct scst_session *sess)
+{
+ atomic_inc(&sess->refcnt);
+ TRACE_DBG("Incrementing sess %p refcnt (new value %d)",
+ sess, atomic_read(&sess->refcnt));
+}
+
+static inline void scst_sess_put(struct scst_session *sess)
+{
+ TRACE_DBG("Decrementing sess %p refcnt (new value %d)",
+ sess, atomic_read(&sess->refcnt)-1);
+ if (atomic_dec_and_test(&sess->refcnt))
+ scst_sched_session_free(sess);
+}
+
+static inline void __scst_cmd_get(struct scst_cmd *cmd)
+{
+ atomic_inc(&cmd->cmd_ref);
+ TRACE_DBG("Incrementing cmd %p ref (new value %d)",
+ cmd, atomic_read(&cmd->cmd_ref));
+}
+
+static inline void __scst_cmd_put(struct scst_cmd *cmd)
+{
+ TRACE_DBG("Decrementing cmd %p ref (new value %d)",
+ cmd, atomic_read(&cmd->cmd_ref)-1);
+ if (atomic_dec_and_test(&cmd->cmd_ref))
+ scst_free_cmd(cmd);
+}
+
+extern void scst_throttle_cmd(struct scst_cmd *cmd);
+extern void scst_unthrottle_cmd(struct scst_cmd *cmd);
+
+static inline void scst_check_restore_sg_buff(struct scst_cmd *cmd)
+{
+ if (cmd->sg_buff_modified) {
+ TRACE_MEM("cmd %p, sg %p, orig_sg_entry %d, "
+ "orig_entry_len %d, orig_sg_cnt %d", cmd, cmd->sg,
+ cmd->orig_sg_entry, cmd->orig_entry_len,
+ cmd->orig_sg_cnt);
+ cmd->sg[cmd->orig_sg_entry].length = cmd->orig_entry_len;
+ cmd->sg_cnt = cmd->orig_sg_cnt;
+ cmd->sg_buff_modified = 0;
+ }
+}
+
+#ifdef CONFIG_SCST_DEBUG_TM
+extern void tm_dbg_check_released_cmds(void);
+extern int tm_dbg_check_cmd(struct scst_cmd *cmd);
+extern void tm_dbg_release_cmd(struct scst_cmd *cmd);
+extern void tm_dbg_task_mgmt(struct scst_device *dev, const char *fn,
+ int force);
+extern int tm_dbg_is_release(void);
+#else
+static inline void tm_dbg_check_released_cmds(void) {}
+static inline int tm_dbg_check_cmd(struct scst_cmd *cmd)
+{
+ return 0;
+}
+static inline void tm_dbg_release_cmd(struct scst_cmd *cmd) {}
+static inline void tm_dbg_task_mgmt(struct scst_device *dev, const char *fn,
+ int force) {}
+static inline int tm_dbg_is_release(void)
+{
+ return 0;
+}
+#endif /* CONFIG_SCST_DEBUG_TM */
+
+#ifdef CONFIG_SCST_DEBUG_SN
+void scst_check_debug_sn(struct scst_cmd *cmd);
+#else
+static inline void scst_check_debug_sn(struct scst_cmd *cmd) {}
+#endif
+
+static inline int scst_sn_before(uint32_t seq1, uint32_t seq2)
+{
+ return (int32_t)(seq1-seq2) < 0;
+}
+
+bool scst_is_relative_target_port_id_unique(uint16_t id, struct scst_tgt *t);
+int gen_relative_target_port_id(uint16_t *id);
+
+#ifdef CONFIG_SCST_MEASURE_LATENCY
+
+void scst_set_start_time(struct scst_cmd *cmd);
+void scst_set_cur_start(struct scst_cmd *cmd);
+void scst_set_parse_time(struct scst_cmd *cmd);
+void scst_set_alloc_buf_time(struct scst_cmd *cmd);
+void scst_set_restart_waiting_time(struct scst_cmd *cmd);
+void scst_set_rdy_to_xfer_time(struct scst_cmd *cmd);
+void scst_set_pre_exec_time(struct scst_cmd *cmd);
+void scst_set_exec_time(struct scst_cmd *cmd);
+void scst_set_dev_done_time(struct scst_cmd *cmd);
+void scst_set_xmit_time(struct scst_cmd *cmd);
+void scst_set_tgt_on_free_time(struct scst_cmd *cmd);
+void scst_set_dev_on_free_time(struct scst_cmd *cmd);
+void scst_update_lat_stats(struct scst_cmd *cmd);
+
+#else
+
+static inline void scst_set_start_time(struct scst_cmd *cmd) {}
+static inline void scst_set_cur_start(struct scst_cmd *cmd) {}
+static inline void scst_set_parse_time(struct scst_cmd *cmd) {}
+static inline void scst_set_alloc_buf_time(struct scst_cmd *cmd) {}
+static inline void scst_set_restart_waiting_time(struct scst_cmd *cmd) {}
+static inline void scst_set_rdy_to_xfer_time(struct scst_cmd *cmd) {}
+static inline void scst_set_pre_exec_time(struct scst_cmd *cmd) {}
+static inline void scst_set_exec_time(struct scst_cmd *cmd) {}
+static inline void scst_set_dev_done_time(struct scst_cmd *cmd) {}
+static inline void scst_set_xmit_time(struct scst_cmd *cmd) {}
+static inline void scst_set_tgt_on_free_time(struct scst_cmd *cmd) {}
+static inline void scst_set_dev_on_free_time(struct scst_cmd *cmd) {}
+static inline void scst_update_lat_stats(struct scst_cmd *cmd) {}
+
+#endif /* CONFIG_SCST_MEASURE_LATENCY */
+
+#endif /* __SCST_PRIV_H */
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 7/12/1/5] SCST SGV cache
[not found] ` <4BC44D08.4060907@vlnb.net>
` (5 preceding siblings ...)
2010-04-13 13:06 ` [PATCH][RFC 6/12/1/5] SCST core's private header Vladislav Bolkhovitin
@ 2010-04-13 13:06 ` Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 8/12/1/5] SCST sysfs interface Vladislav Bolkhovitin
` (2 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:06 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains SCST SGV cache. SGV cache is a memory management
subsystem in SCST. More info about it you can find in the documentation
in this patch.
P.S. Solaris COMSTAR also has similar facility.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Documentation/scst/sgv_cache.txt | 224 ++++
drivers/scst/scst_mem.c | 1788 +++++++++++++++++++++++++++++++++++++++
drivers/scst/scst_mem.h | 150 +++
include/scst/scst_sgv.h | 97 ++
4 files changed, 2259 insertions(+)
diff -uprN orig/linux-2.6.33/include/scst/scst_sgv.h linux-2.6.33/include/scst/scst_sgv.h
--- orig/linux-2.6.33/include/scst/scst_sgv.h
+++ linux-2.6.33/include/scst/scst_sgv.h
@@ -0,0 +1,97 @@
+/*
+ * include/scst_sgv.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Include file for SCST SGV cache.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef __SCST_SGV_H
+#define __SCST_SGV_H
+
+/** SGV pool routines and flag bits **/
+
+/* Set if the allocated object must be not from the cache */
+#define SGV_POOL_ALLOC_NO_CACHED 1
+
+/* Set if there should not be any memory allocations on a cache miss */
+#define SGV_POOL_NO_ALLOC_ON_CACHE_MISS 2
+
+/* Set an object should be returned even if it doesn't have SG vector built */
+#define SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL 4
+
+/*
+ * Set if the allocated object must be a new one, i.e. from the cache,
+ * but not cached
+ */
+#define SGV_POOL_ALLOC_GET_NEW 8
+
+struct sgv_pool_obj;
+struct sgv_pool;
+
+/*
+ * Structure to keep a memory limit for an SCST object
+ */
+struct scst_mem_lim {
+ /* How much memory allocated under this object */
+ atomic_t alloced_pages;
+
+ /*
+ * How much memory allowed to allocated under this object. Put here
+ * mostly to save a possible cache miss accessing scst_max_dev_cmd_mem.
+ */
+ int max_allowed_pages;
+};
+
+/* Types of clustering */
+enum sgv_clustering_types {
+ /* No clustering performed */
+ sgv_no_clustering = 0,
+
+ /*
+ * A page will only be merged with the latest previously allocated
+ * page, so the order of pages in the SG will be preserved.
+ */
+ sgv_tail_clustering,
+
+ /*
+ * Free merging of pages at any place in the SG is allowed. This mode
+ * usually provides the best merging rate.
+ */
+ sgv_full_clustering,
+};
+
+struct sgv_pool *sgv_pool_create(const char *name,
+ enum sgv_clustering_types clustered, int single_alloc_pages,
+ bool shared, int purge_interval);
+void sgv_pool_del(struct sgv_pool *pool);
+
+void sgv_pool_get(struct sgv_pool *pool);
+void sgv_pool_put(struct sgv_pool *pool);
+
+void sgv_pool_flush(struct sgv_pool *pool);
+
+void sgv_pool_set_allocator(struct sgv_pool *pool,
+ struct page *(*alloc_pages_fn)(struct scatterlist *, gfp_t, void *),
+ void (*free_pages_fn)(struct scatterlist *, int, void *));
+
+struct scatterlist *sgv_pool_alloc(struct sgv_pool *pool, unsigned int size,
+ gfp_t gfp_mask, int flags, int *count,
+ struct sgv_pool_obj **sgv, struct scst_mem_lim *mem_lim, void *priv);
+void sgv_pool_free(struct sgv_pool_obj *sgv, struct scst_mem_lim *mem_lim);
+
+void *sgv_get_priv(struct sgv_pool_obj *sgv);
+
+void scst_init_mem_lim(struct scst_mem_lim *mem_lim);
+
+#endif /* __SCST_SGV_H */
diff -uprN orig/linux-2.6.33/drivers/scst/scst_mem.h linux-2.6.33/drivers/scst/scst_mem.h
--- orig/linux-2.6.33/drivers/scst/scst_mem.h
+++ linux-2.6.33/drivers/scst/scst_mem.h
@@ -0,0 +1,150 @@
+/*
+ * scst_mem.h
+ *
+ * Copyright (C) 2006 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/scatterlist.h>
+#include <linux/workqueue.h>
+
+#define SGV_POOL_ELEMENTS 11
+
+/*
+ * sg_num is indexed by the page number, pg_count is indexed by the sg number.
+ * Made in one entry to simplify the code (eg all sizeof(*) parts) and save
+ * some CPU cache for non-clustered case.
+ */
+struct trans_tbl_ent {
+ unsigned short sg_num;
+ unsigned short pg_count;
+};
+
+/*
+ * SGV pool object
+ */
+struct sgv_pool_obj {
+ int cache_num;
+ int pages;
+
+ /* jiffies, protected by sgv_pool_lock */
+ unsigned long time_stamp;
+
+ struct list_head recycling_list_entry;
+ struct list_head sorted_recycling_list_entry;
+
+ struct sgv_pool *owner_pool;
+ int orig_sg;
+ int orig_length;
+ int sg_count;
+ void *allocator_priv;
+ struct trans_tbl_ent *trans_tbl;
+ struct scatterlist *sg_entries;
+ struct scatterlist sg_entries_data[0];
+};
+
+/*
+ * SGV pool statistics accounting structure
+ */
+struct sgv_pool_cache_acc {
+ atomic_t total_alloc, hit_alloc;
+ atomic_t merged;
+};
+
+/*
+ * SGV pool allocation functions
+ */
+struct sgv_pool_alloc_fns {
+ struct page *(*alloc_pages_fn)(struct scatterlist *sg, gfp_t gfp_mask,
+ void *priv);
+ void (*free_pages_fn)(struct scatterlist *sg, int sg_count,
+ void *priv);
+};
+
+/*
+ * SGV pool
+ */
+struct sgv_pool {
+ enum sgv_clustering_types clustering_type;
+ int single_alloc_pages;
+ int max_cached_pages;
+
+ struct sgv_pool_alloc_fns alloc_fns;
+
+ /* <=4K, <=8, <=16, <=32, <=64, <=128, <=256, <=512, <=1024, <=2048 */
+ struct kmem_cache *caches[SGV_POOL_ELEMENTS];
+
+ spinlock_t sgv_pool_lock; /* outer lock for sgv_pools_lock! */
+
+ int purge_interval;
+
+ /* Protected by sgv_pool_lock, if necessary */
+ unsigned int purge_work_scheduled:1;
+ unsigned int sgv_kobj_initialized:1;
+
+ /* Protected by sgv_pool_lock */
+ struct list_head sorted_recycling_list;
+
+ int inactive_cached_pages; /* protected by sgv_pool_lock */
+
+ /* Protected by sgv_pool_lock */
+ struct list_head recycling_lists[SGV_POOL_ELEMENTS];
+
+ int cached_pages, cached_entries; /* protected by sgv_pool_lock */
+
+ struct sgv_pool_cache_acc cache_acc[SGV_POOL_ELEMENTS];
+
+ struct delayed_work sgv_purge_work;
+
+ struct list_head sgv_active_pools_list_entry;
+
+ atomic_t big_alloc, big_pages, big_merged;
+ atomic_t other_alloc, other_pages, other_merged;
+
+ atomic_t sgv_pool_ref;
+
+ int max_caches;
+
+ /* SCST_MAX_NAME + few more bytes to match scst_user expectations */
+ char cache_names[SGV_POOL_ELEMENTS][SCST_MAX_NAME + 10];
+ char name[SCST_MAX_NAME + 10];
+
+ struct mm_struct *owner_mm;
+
+ struct list_head sgv_pools_list_entry;
+
+ struct kobject sgv_kobj;
+};
+
+static inline struct scatterlist *sgv_pool_sg(struct sgv_pool_obj *obj)
+{
+ return obj->sg_entries;
+}
+
+int scst_sgv_pools_init(unsigned long mem_hwmark, unsigned long mem_lwmark);
+void scst_sgv_pools_deinit(void);
+
+void sgv_pool_destroy(struct sgv_pool *pool);
+
+ssize_t sgv_sysfs_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+ssize_t sgv_sysfs_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count);
+ssize_t sgv_sysfs_global_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+ssize_t sgv_sysfs_global_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count);
+
+void scst_sgv_pool_use_norm(struct scst_tgt_dev *tgt_dev);
+void scst_sgv_pool_use_norm_clust(struct scst_tgt_dev *tgt_dev);
+void scst_sgv_pool_use_dma(struct scst_tgt_dev *tgt_dev);
diff -uprN orig/linux-2.6.33/drivers/scst/scst_mem.c linux-2.6.33/drivers/scst/scst_mem.c
--- orig/linux-2.6.33/drivers/scst/scst_mem.c
+++ linux-2.6.33/drivers/scst/scst_mem.c
@@ -0,0 +1,1788 @@
+/*
+ * scst_mem.c
+ *
+ * Copyright (C) 2006 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/unistd.h>
+#include <linux/string.h>
+
+#include "scst.h"
+#include "scst_priv.h"
+#include "scst_mem.h"
+
+#define SGV_DEFAULT_PURGE_INTERVAL (60 * HZ)
+#define SGV_MIN_SHRINK_INTERVAL (1 * HZ)
+
+/* Max pages freed from a pool per shrinking iteration */
+#define MAX_PAGES_PER_POOL 50
+
+static struct sgv_pool *sgv_norm_clust_pool, *sgv_norm_pool, *sgv_dma_pool;
+
+static atomic_t sgv_pages_total = ATOMIC_INIT(0);
+
+/* Both read-only */
+static int sgv_hi_wmk;
+static int sgv_lo_wmk;
+
+static int sgv_max_local_pages, sgv_max_trans_pages;
+
+static DEFINE_SPINLOCK(sgv_pools_lock); /* inner lock for sgv_pool_lock! */
+static DEFINE_MUTEX(sgv_pools_mutex);
+
+/* Both protected by sgv_pools_lock */
+static struct sgv_pool *sgv_cur_purge_pool;
+static LIST_HEAD(sgv_active_pools_list);
+
+static atomic_t sgv_releases_on_hiwmk = ATOMIC_INIT(0);
+static atomic_t sgv_releases_on_hiwmk_failed = ATOMIC_INIT(0);
+
+static atomic_t sgv_other_total_alloc = ATOMIC_INIT(0);
+
+static struct shrinker sgv_shrinker;
+
+/*
+ * Protected by sgv_pools_mutex AND sgv_pools_lock for writes,
+ * either one for reads.
+ */
+static LIST_HEAD(sgv_pools_list);
+
+static inline bool sgv_pool_clustered(const struct sgv_pool *pool)
+{
+ return pool->clustering_type != sgv_no_clustering;
+}
+
+void scst_sgv_pool_use_norm(struct scst_tgt_dev *tgt_dev)
+{
+ tgt_dev->gfp_mask = __GFP_NOWARN;
+ tgt_dev->pool = sgv_norm_pool;
+ clear_bit(SCST_TGT_DEV_CLUST_POOL, &tgt_dev->tgt_dev_flags);
+}
+
+void scst_sgv_pool_use_norm_clust(struct scst_tgt_dev *tgt_dev)
+{
+ TRACE_MEM("%s", "Use clustering");
+ tgt_dev->gfp_mask = __GFP_NOWARN;
+ tgt_dev->pool = sgv_norm_clust_pool;
+ set_bit(SCST_TGT_DEV_CLUST_POOL, &tgt_dev->tgt_dev_flags);
+}
+
+void scst_sgv_pool_use_dma(struct scst_tgt_dev *tgt_dev)
+{
+ TRACE_MEM("%s", "Use ISA DMA memory");
+ tgt_dev->gfp_mask = __GFP_NOWARN | GFP_DMA;
+ tgt_dev->pool = sgv_dma_pool;
+ clear_bit(SCST_TGT_DEV_CLUST_POOL, &tgt_dev->tgt_dev_flags);
+}
+
+/* Must be no locks */
+static void sgv_dtor_and_free(struct sgv_pool_obj *obj)
+{
+ struct sgv_pool *pool = obj->owner_pool;
+
+ TRACE_MEM("Destroying sgv obj %p", obj);
+
+ if (obj->sg_count != 0) {
+ pool->alloc_fns.free_pages_fn(obj->sg_entries,
+ obj->sg_count, obj->allocator_priv);
+ }
+ if (obj->sg_entries != obj->sg_entries_data) {
+ if (obj->trans_tbl !=
+ (struct trans_tbl_ent *)obj->sg_entries_data) {
+ /* kfree() handles NULL parameter */
+ kfree(obj->trans_tbl);
+ obj->trans_tbl = NULL;
+ }
+ kfree(obj->sg_entries);
+ }
+
+ kmem_cache_free(pool->caches[obj->cache_num], obj);
+ return;
+}
+
+/* Might be called under sgv_pool_lock */
+static inline void sgv_del_from_active(struct sgv_pool *pool)
+{
+ struct list_head *next;
+
+ TRACE_MEM("Deleting sgv pool %p from the active list", pool);
+
+ spin_lock_bh(&sgv_pools_lock);
+
+ next = pool->sgv_active_pools_list_entry.next;
+ list_del(&pool->sgv_active_pools_list_entry);
+
+ if (sgv_cur_purge_pool == pool) {
+ TRACE_MEM("Sgv pool %p is sgv cur purge pool", pool);
+
+ if (next == &sgv_active_pools_list)
+ next = next->next;
+
+ if (next == &sgv_active_pools_list) {
+ sgv_cur_purge_pool = NULL;
+ TRACE_MEM("%s", "Sgv active list now empty");
+ } else {
+ sgv_cur_purge_pool = list_entry(next, typeof(*pool),
+ sgv_active_pools_list_entry);
+ TRACE_MEM("New sgv cur purge pool %p",
+ sgv_cur_purge_pool);
+ }
+ }
+
+ spin_unlock_bh(&sgv_pools_lock);
+ return;
+}
+
+/* Must be called under sgv_pool_lock held */
+static void sgv_dec_cached_entries(struct sgv_pool *pool, int pages)
+{
+ pool->cached_entries--;
+ pool->cached_pages -= pages;
+
+ if (pool->cached_entries == 0)
+ sgv_del_from_active(pool);
+
+ return;
+}
+
+/* Must be called under sgv_pool_lock held */
+static void __sgv_purge_from_cache(struct sgv_pool_obj *obj)
+{
+ int pages = obj->pages;
+ struct sgv_pool *pool = obj->owner_pool;
+
+ TRACE_MEM("Purging sgv obj %p from pool %p (new cached_entries %d)",
+ obj, pool, pool->cached_entries-1);
+
+ list_del(&obj->sorted_recycling_list_entry);
+ list_del(&obj->recycling_list_entry);
+
+ pool->inactive_cached_pages -= pages;
+ sgv_dec_cached_entries(pool, pages);
+
+ atomic_sub(pages, &sgv_pages_total);
+
+ return;
+}
+
+/* Must be called under sgv_pool_lock held */
+static bool sgv_purge_from_cache(struct sgv_pool_obj *obj, int min_interval,
+ unsigned long cur_time)
+{
+ EXTRACHECKS_BUG_ON(min_interval < 0);
+
+ TRACE_MEM("Checking if sgv obj %p should be purged (cur time %ld, "
+ "obj time %ld, time to purge %ld)", obj, cur_time,
+ obj->time_stamp, obj->time_stamp + min_interval);
+
+ if (time_after_eq(cur_time, (obj->time_stamp + min_interval))) {
+ __sgv_purge_from_cache(obj);
+ return true;
+ }
+ return false;
+}
+
+/* No locks */
+static int sgv_shrink_pool(struct sgv_pool *pool, int nr, int min_interval,
+ unsigned long cur_time)
+{
+ int freed = 0;
+
+ TRACE_MEM("Trying to shrink pool %p (nr %d, min_interval %d)",
+ pool, nr, min_interval);
+
+ if (pool->purge_interval < 0) {
+ TRACE_MEM("Not shrinkable pool %p, skipping", pool);
+ goto out;
+ }
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ while (!list_empty(&pool->sorted_recycling_list) &&
+ (atomic_read(&sgv_pages_total) > sgv_lo_wmk)) {
+ struct sgv_pool_obj *obj = list_entry(
+ pool->sorted_recycling_list.next,
+ struct sgv_pool_obj, sorted_recycling_list_entry);
+
+ if (sgv_purge_from_cache(obj, min_interval, cur_time)) {
+ int pages = obj->pages;
+
+ freed += pages;
+ nr -= pages;
+
+ TRACE_MEM("%d pages purged from pool %p (nr left %d, "
+ "total freed %d)", pages, pool, nr, freed);
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ sgv_dtor_and_free(obj);
+ spin_lock_bh(&pool->sgv_pool_lock);
+ } else
+ break;
+
+ if ((nr <= 0) || (freed >= MAX_PAGES_PER_POOL)) {
+ if (freed >= MAX_PAGES_PER_POOL)
+ TRACE_MEM("%d pages purged from pool %p, "
+ "leaving", freed, pool);
+ break;
+ }
+ }
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+out:
+ return nr;
+}
+
+/* No locks */
+static int __sgv_shrink(int nr, int min_interval)
+{
+ struct sgv_pool *pool;
+ unsigned long cur_time = jiffies;
+ int prev_nr = nr;
+ bool circle = false;
+
+ TRACE_MEM("Trying to shrink %d pages from all sgv pools "
+ "(min_interval %d)", nr, min_interval);
+
+ while (nr > 0) {
+ struct list_head *next;
+
+ spin_lock_bh(&sgv_pools_lock);
+
+ pool = sgv_cur_purge_pool;
+ if (pool == NULL) {
+ if (list_empty(&sgv_active_pools_list)) {
+ TRACE_MEM("%s", "Active pools list is empty");
+ goto out_unlock;
+ }
+
+ pool = list_entry(sgv_active_pools_list.next,
+ typeof(*pool),
+ sgv_active_pools_list_entry);
+ }
+ sgv_pool_get(pool);
+
+ next = pool->sgv_active_pools_list_entry.next;
+ if (next == &sgv_active_pools_list) {
+ if (circle && (prev_nr == nr)) {
+ TRACE_MEM("Full circle done, but no progress, "
+ "leaving (nr %d)", nr);
+ goto out_unlock_put;
+ }
+ circle = true;
+ prev_nr = nr;
+
+ next = next->next;
+ }
+
+ sgv_cur_purge_pool = list_entry(next, typeof(*pool),
+ sgv_active_pools_list_entry);
+ TRACE_MEM("New cur purge pool %p", sgv_cur_purge_pool);
+
+ spin_unlock_bh(&sgv_pools_lock);
+
+ nr = sgv_shrink_pool(pool, nr, min_interval, cur_time);
+
+ sgv_pool_put(pool);
+ }
+
+out:
+ return nr;
+
+out_unlock:
+ spin_unlock_bh(&sgv_pools_lock);
+ goto out;
+
+out_unlock_put:
+ spin_unlock_bh(&sgv_pools_lock);
+ sgv_pool_put(pool);
+ goto out;
+}
+
+static int sgv_shrink(int nr, gfp_t gfpm)
+{
+
+ if (nr > 0) {
+ nr = __sgv_shrink(nr, SGV_MIN_SHRINK_INTERVAL);
+ TRACE_MEM("Left %d", nr);
+ } else {
+ struct sgv_pool *pool;
+ int inactive_pages = 0;
+
+ spin_lock_bh(&sgv_pools_lock);
+ list_for_each_entry(pool, &sgv_active_pools_list,
+ sgv_active_pools_list_entry) {
+ if (pool->purge_interval > 0)
+ inactive_pages += pool->inactive_cached_pages;
+ }
+ spin_unlock_bh(&sgv_pools_lock);
+
+ nr = max((int)0, inactive_pages - sgv_lo_wmk);
+ TRACE_MEM("Can free %d (total %d)", nr,
+ atomic_read(&sgv_pages_total));
+ }
+ return nr;
+}
+
+static void sgv_purge_work_fn(struct delayed_work *work)
+{
+ unsigned long cur_time = jiffies;
+ struct sgv_pool *pool = container_of(work, struct sgv_pool,
+ sgv_purge_work);
+
+ TRACE_MEM("Purge work for pool %p", pool);
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ pool->purge_work_scheduled = false;
+
+ while (!list_empty(&pool->sorted_recycling_list)) {
+ struct sgv_pool_obj *obj = list_entry(
+ pool->sorted_recycling_list.next,
+ struct sgv_pool_obj, sorted_recycling_list_entry);
+
+ if (sgv_purge_from_cache(obj, pool->purge_interval, cur_time)) {
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ sgv_dtor_and_free(obj);
+ spin_lock_bh(&pool->sgv_pool_lock);
+ } else {
+ /*
+ * Let's reschedule it for full period to not get here
+ * too often. In the worst case we have shrinker
+ * to reclaim buffers quickier.
+ */
+ TRACE_MEM("Rescheduling purge work for pool %p (delay "
+ "%d HZ/%d sec)", pool, pool->purge_interval,
+ pool->purge_interval/HZ);
+ schedule_delayed_work(&pool->sgv_purge_work,
+ pool->purge_interval);
+ pool->purge_work_scheduled = true;
+ break;
+ }
+ }
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ TRACE_MEM("Leaving purge work for pool %p", pool);
+ return;
+}
+
+static int sgv_check_full_clustering(struct scatterlist *sg, int cur, int hint)
+{
+ int res = -1;
+ int i = hint;
+ unsigned long pfn_cur = page_to_pfn(sg_page(&sg[cur]));
+ int len_cur = sg[cur].length;
+ unsigned long pfn_cur_next = pfn_cur + (len_cur >> PAGE_SHIFT);
+ int full_page_cur = (len_cur & (PAGE_SIZE - 1)) == 0;
+ unsigned long pfn, pfn_next;
+ bool full_page;
+
+#if 0
+ TRACE_MEM("pfn_cur %ld, pfn_cur_next %ld, len_cur %d, full_page_cur %d",
+ pfn_cur, pfn_cur_next, len_cur, full_page_cur);
+#endif
+
+ /* check the hint first */
+ if (i >= 0) {
+ pfn = page_to_pfn(sg_page(&sg[i]));
+ pfn_next = pfn + (sg[i].length >> PAGE_SHIFT);
+ full_page = (sg[i].length & (PAGE_SIZE - 1)) == 0;
+
+ if ((pfn == pfn_cur_next) && full_page_cur)
+ goto out_head;
+
+ if ((pfn_next == pfn_cur) && full_page)
+ goto out_tail;
+ }
+
+ /* ToDo: implement more intelligent search */
+ for (i = cur - 1; i >= 0; i--) {
+ pfn = page_to_pfn(sg_page(&sg[i]));
+ pfn_next = pfn + (sg[i].length >> PAGE_SHIFT);
+ full_page = (sg[i].length & (PAGE_SIZE - 1)) == 0;
+
+ if ((pfn == pfn_cur_next) && full_page_cur)
+ goto out_head;
+
+ if ((pfn_next == pfn_cur) && full_page)
+ goto out_tail;
+ }
+
+out:
+ return res;
+
+out_tail:
+ TRACE_MEM("SG segment %d will be tail merged with segment %d", cur, i);
+ sg[i].length += len_cur;
+ sg_clear(&sg[cur]);
+ res = i;
+ goto out;
+
+out_head:
+ TRACE_MEM("SG segment %d will be head merged with segment %d", cur, i);
+ sg_assign_page(&sg[i], sg_page(&sg[cur]));
+ sg[i].length += len_cur;
+ sg_clear(&sg[cur]);
+ res = i;
+ goto out;
+}
+
+static int sgv_check_tail_clustering(struct scatterlist *sg, int cur, int hint)
+{
+ int res = -1;
+ unsigned long pfn_cur = page_to_pfn(sg_page(&sg[cur]));
+ int len_cur = sg[cur].length;
+ int prev;
+ unsigned long pfn_prev;
+ bool full_page;
+
+#ifdef SCST_HIGHMEM
+ if (page >= highmem_start_page) {
+ TRACE_MEM("%s", "HIGHMEM page allocated, no clustering")
+ goto out;
+ }
+#endif
+
+#if 0
+ TRACE_MEM("pfn_cur %ld, pfn_cur_next %ld, len_cur %d, full_page_cur %d",
+ pfn_cur, pfn_cur_next, len_cur, full_page_cur);
+#endif
+
+ if (cur == 0)
+ goto out;
+
+ prev = cur - 1;
+ pfn_prev = page_to_pfn(sg_page(&sg[prev])) +
+ (sg[prev].length >> PAGE_SHIFT);
+ full_page = (sg[prev].length & (PAGE_SIZE - 1)) == 0;
+
+ if ((pfn_prev == pfn_cur) && full_page) {
+ TRACE_MEM("SG segment %d will be tail merged with segment %d",
+ cur, prev);
+ sg[prev].length += len_cur;
+ sg_clear(&sg[cur]);
+ res = prev;
+ }
+
+out:
+ return res;
+}
+
+static void sgv_free_sys_sg_entries(struct scatterlist *sg, int sg_count,
+ void *priv)
+{
+ int i;
+
+ TRACE_MEM("sg=%p, sg_count=%d", sg, sg_count);
+
+ for (i = 0; i < sg_count; i++) {
+ struct page *p = sg_page(&sg[i]);
+ int len = sg[i].length;
+ int pages =
+ (len >> PAGE_SHIFT) + ((len & ~PAGE_MASK) != 0);
+
+ TRACE_MEM("page %lx, len %d, pages %d",
+ (unsigned long)p, len, pages);
+
+ while (pages > 0) {
+ int order = 0;
+
+/*
+ * __free_pages() doesn't like freeing pages with not that order with
+ * which they were allocated, so disable this small optimization.
+ */
+#if 0
+ if (len > 0) {
+ while (((1 << order) << PAGE_SHIFT) < len)
+ order++;
+ len = 0;
+ }
+#endif
+ TRACE_MEM("free_pages(): order %d, page %lx",
+ order, (unsigned long)p);
+
+ __free_pages(p, order);
+
+ pages -= 1 << order;
+ p += 1 << order;
+ }
+ }
+}
+
+static struct page *sgv_alloc_sys_pages(struct scatterlist *sg,
+ gfp_t gfp_mask, void *priv)
+{
+ struct page *page = alloc_pages(gfp_mask, 0);
+
+ sg_set_page(sg, page, PAGE_SIZE, 0);
+ TRACE_MEM("page=%p, sg=%p, priv=%p", page, sg, priv);
+ if (page == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of "
+ "sg page failed");
+ }
+ return page;
+}
+
+static int sgv_alloc_sg_entries(struct scatterlist *sg, int pages,
+ gfp_t gfp_mask, enum sgv_clustering_types clustering_type,
+ struct trans_tbl_ent *trans_tbl,
+ const struct sgv_pool_alloc_fns *alloc_fns, void *priv)
+{
+ int sg_count = 0;
+ int pg, i, j;
+ int merged = -1;
+
+ TRACE_MEM("pages=%d, clustering_type=%d", pages, clustering_type);
+
+#if 0
+ gfp_mask |= __GFP_COLD;
+#endif
+#ifdef CONFIG_SCST_STRICT_SECURITY
+ gfp_mask |= __GFP_ZERO;
+#endif
+
+ for (pg = 0; pg < pages; pg++) {
+ void *rc;
+#ifdef CONFIG_SCST_DEBUG_OOM
+ if (((gfp_mask & __GFP_NOFAIL) != __GFP_NOFAIL) &&
+ ((scst_random() % 10000) == 55))
+ rc = NULL;
+ else
+#endif
+ rc = alloc_fns->alloc_pages_fn(&sg[sg_count], gfp_mask,
+ priv);
+ if (rc == NULL)
+ goto out_no_mem;
+
+ /*
+ * This code allows compiler to see full body of the clustering
+ * functions and gives it a chance to generate better code.
+ * At least, the resulting code is smaller, comparing to
+ * calling them using a function pointer.
+ */
+ if (clustering_type == sgv_full_clustering)
+ merged = sgv_check_full_clustering(sg, sg_count, merged);
+ else if (clustering_type == sgv_tail_clustering)
+ merged = sgv_check_tail_clustering(sg, sg_count, merged);
+ else
+ merged = -1;
+
+ if (merged == -1)
+ sg_count++;
+
+ TRACE_MEM("pg=%d, merged=%d, sg_count=%d", pg, merged,
+ sg_count);
+ }
+
+ if ((clustering_type != sgv_no_clustering) && (trans_tbl != NULL)) {
+ pg = 0;
+ for (i = 0; i < pages; i++) {
+ int n = (sg[i].length >> PAGE_SHIFT) +
+ ((sg[i].length & ~PAGE_MASK) != 0);
+ trans_tbl[i].pg_count = pg;
+ for (j = 0; j < n; j++)
+ trans_tbl[pg++].sg_num = i+1;
+ TRACE_MEM("i=%d, n=%d, pg_count=%d", i, n,
+ trans_tbl[i].pg_count);
+ }
+ }
+
+out:
+ TRACE_MEM("sg_count=%d", sg_count);
+ return sg_count;
+
+out_no_mem:
+ alloc_fns->free_pages_fn(sg, sg_count, priv);
+ sg_count = 0;
+ goto out;
+}
+
+static int sgv_alloc_arrays(struct sgv_pool_obj *obj,
+ int pages_to_alloc, gfp_t gfp_mask)
+{
+ int sz, tsz = 0;
+ int res = 0;
+
+ sz = pages_to_alloc * sizeof(obj->sg_entries[0]);
+
+ obj->sg_entries = kmalloc(sz, gfp_mask);
+ if (unlikely(obj->sg_entries == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of sgv_pool_obj "
+ "SG vector failed (size %d)", sz);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ sg_init_table(obj->sg_entries, pages_to_alloc);
+
+ if (sgv_pool_clustered(obj->owner_pool)) {
+ if (pages_to_alloc <= sgv_max_trans_pages) {
+ obj->trans_tbl =
+ (struct trans_tbl_ent *)obj->sg_entries_data;
+ /*
+ * No need to clear trans_tbl, if needed, it will be
+ * fully rewritten in sgv_alloc_sg_entries()
+ */
+ } else {
+ tsz = pages_to_alloc * sizeof(obj->trans_tbl[0]);
+ obj->trans_tbl = kzalloc(tsz, gfp_mask);
+ if (unlikely(obj->trans_tbl == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of "
+ "trans_tbl failed (size %d)", tsz);
+ res = -ENOMEM;
+ goto out_free;
+ }
+ }
+ }
+
+ TRACE_MEM("pages_to_alloc %d, sz %d, tsz %d, obj %p, sg_entries %p, "
+ "trans_tbl %p", pages_to_alloc, sz, tsz, obj, obj->sg_entries,
+ obj->trans_tbl);
+
+out:
+ return res;
+
+out_free:
+ kfree(obj->sg_entries);
+ obj->sg_entries = NULL;
+ goto out;
+}
+
+static struct sgv_pool_obj *sgv_get_obj(struct sgv_pool *pool, int cache_num,
+ int pages, gfp_t gfp_mask, bool get_new)
+{
+ struct sgv_pool_obj *obj;
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ if (unlikely(get_new)) {
+ /* Used only for buffers preallocation */
+ goto get_new;
+ }
+
+ if (likely(!list_empty(&pool->recycling_lists[cache_num]))) {
+ obj = list_entry(pool->recycling_lists[cache_num].next,
+ struct sgv_pool_obj, recycling_list_entry);
+
+ list_del(&obj->sorted_recycling_list_entry);
+ list_del(&obj->recycling_list_entry);
+
+ pool->inactive_cached_pages -= pages;
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ goto out;
+ }
+
+get_new:
+ if (pool->cached_entries == 0) {
+ TRACE_MEM("Adding pool %p to the active list", pool);
+ spin_lock_bh(&sgv_pools_lock);
+ list_add_tail(&pool->sgv_active_pools_list_entry,
+ &sgv_active_pools_list);
+ spin_unlock_bh(&sgv_pools_lock);
+ }
+
+ pool->cached_entries++;
+ pool->cached_pages += pages;
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ TRACE_MEM("New cached entries %d (pool %p)", pool->cached_entries,
+ pool);
+
+ obj = kmem_cache_alloc(pool->caches[cache_num],
+ gfp_mask & ~(__GFP_HIGHMEM|GFP_DMA));
+ if (likely(obj)) {
+ memset(obj, 0, sizeof(*obj));
+ obj->cache_num = cache_num;
+ obj->pages = pages;
+ obj->owner_pool = pool;
+ } else {
+ spin_lock_bh(&pool->sgv_pool_lock);
+ sgv_dec_cached_entries(pool, pages);
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ }
+
+out:
+ return obj;
+}
+
+static void sgv_put_obj(struct sgv_pool_obj *obj)
+{
+ struct sgv_pool *pool = obj->owner_pool;
+ struct list_head *entry;
+ struct list_head *list = &pool->recycling_lists[obj->cache_num];
+ int pages = obj->pages;
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ TRACE_MEM("sgv %p, cache num %d, pages %d, sg_count %d", obj,
+ obj->cache_num, pages, obj->sg_count);
+
+ if (sgv_pool_clustered(pool)) {
+ /* Make objects with less entries more preferred */
+ __list_for_each(entry, list) {
+ struct sgv_pool_obj *tmp = list_entry(entry,
+ struct sgv_pool_obj, recycling_list_entry);
+
+ TRACE_MEM("tmp %p, cache num %d, pages %d, sg_count %d",
+ tmp, tmp->cache_num, tmp->pages, tmp->sg_count);
+
+ if (obj->sg_count <= tmp->sg_count)
+ break;
+ }
+ entry = entry->prev;
+ } else
+ entry = list;
+
+ TRACE_MEM("Adding in %p (list %p)", entry, list);
+ list_add(&obj->recycling_list_entry, entry);
+
+ list_add_tail(&obj->sorted_recycling_list_entry,
+ &pool->sorted_recycling_list);
+
+ obj->time_stamp = jiffies;
+
+ pool->inactive_cached_pages += pages;
+
+ if (!pool->purge_work_scheduled) {
+ TRACE_MEM("Scheduling purge work for pool %p", pool);
+ pool->purge_work_scheduled = true;
+ schedule_delayed_work(&pool->sgv_purge_work,
+ pool->purge_interval);
+ }
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ return;
+}
+
+/* No locks */
+static int sgv_hiwmk_check(int pages_to_alloc)
+{
+ int res = 0;
+ int pages = pages_to_alloc;
+
+ pages += atomic_read(&sgv_pages_total);
+
+ if (unlikely(pages > sgv_hi_wmk)) {
+ pages -= sgv_hi_wmk;
+ atomic_inc(&sgv_releases_on_hiwmk);
+
+ pages = __sgv_shrink(pages, 0);
+ if (pages > 0) {
+ TRACE(TRACE_OUT_OF_MEM, "Requested amount of "
+ "memory (%d pages) for being executed "
+ "commands together with the already "
+ "allocated memory exceeds the allowed "
+ "maximum %d. Should you increase "
+ "scst_max_cmd_mem?", pages_to_alloc,
+ sgv_hi_wmk);
+ atomic_inc(&sgv_releases_on_hiwmk_failed);
+ res = -ENOMEM;
+ goto out_unlock;
+ }
+ }
+
+ atomic_add(pages_to_alloc, &sgv_pages_total);
+
+out_unlock:
+ TRACE_MEM("pages_to_alloc %d, new total %d", pages_to_alloc,
+ atomic_read(&sgv_pages_total));
+
+ return res;
+}
+
+/* No locks */
+static void sgv_hiwmk_uncheck(int pages)
+{
+ atomic_sub(pages, &sgv_pages_total);
+ TRACE_MEM("pages %d, new total %d", pages,
+ atomic_read(&sgv_pages_total));
+ return;
+}
+
+/* No locks */
+static bool sgv_check_allowed_mem(struct scst_mem_lim *mem_lim, int pages)
+{
+ int alloced;
+ bool res = true;
+
+ alloced = atomic_add_return(pages, &mem_lim->alloced_pages);
+ if (unlikely(alloced > mem_lim->max_allowed_pages)) {
+ TRACE(TRACE_OUT_OF_MEM, "Requested amount of memory "
+ "(%d pages) for being executed commands on a device "
+ "together with the already allocated memory exceeds "
+ "the allowed maximum %d. Should you increase "
+ "scst_max_dev_cmd_mem?", pages,
+ mem_lim->max_allowed_pages);
+ atomic_sub(pages, &mem_lim->alloced_pages);
+ res = false;
+ }
+
+ TRACE_MEM("mem_lim %p, pages %d, res %d, new alloced %d", mem_lim,
+ pages, res, atomic_read(&mem_lim->alloced_pages));
+
+ return res;
+}
+
+/* No locks */
+static void sgv_uncheck_allowed_mem(struct scst_mem_lim *mem_lim, int pages)
+{
+ atomic_sub(pages, &mem_lim->alloced_pages);
+
+ TRACE_MEM("mem_lim %p, pages %d, new alloced %d", mem_lim,
+ pages, atomic_read(&mem_lim->alloced_pages));
+ return;
+}
+
+/**
+ * sgv_pool_alloc - allocate an SG vector from the SGV pool
+ * @pool: the cache to alloc from
+ * @size: size of the resulting SG vector in bytes
+ * @gfp_mask: the allocation mask
+ * @flags: the allocation flags
+ * @count: the resulting count of SG entries in the resulting SG vector
+ * @sgv: the resulting SGV object
+ * @mem_lim: memory limits
+ * @priv: pointer to private for this allocation data
+ *
+ * Description:
+ * Allocate an SG vector from the SGV pool and returns pointer to it or
+ * NULL in case of any error. See the SGV pool documentation for more details.
+ */
+struct scatterlist *sgv_pool_alloc(struct sgv_pool *pool, unsigned int size,
+ gfp_t gfp_mask, int flags, int *count,
+ struct sgv_pool_obj **sgv, struct scst_mem_lim *mem_lim, void *priv)
+{
+ struct sgv_pool_obj *obj;
+ int cache_num, pages, cnt;
+ struct scatterlist *res = NULL;
+ int pages_to_alloc;
+ int no_cached = flags & SGV_POOL_ALLOC_NO_CACHED;
+ bool allowed_mem_checked = false, hiwmk_checked = false;
+
+ if (unlikely(size == 0))
+ goto out;
+
+ EXTRACHECKS_BUG_ON((gfp_mask & __GFP_NOFAIL) == __GFP_NOFAIL);
+
+ pages = ((size + PAGE_SIZE - 1) >> PAGE_SHIFT);
+ if (pool->single_alloc_pages == 0) {
+ int pages_order = get_order(size);
+ cache_num = pages_order;
+ pages_to_alloc = (1 << pages_order);
+ } else {
+ cache_num = 0;
+ pages_to_alloc = max(pool->single_alloc_pages, pages);
+ }
+
+ TRACE_MEM("size=%d, pages=%d, pages_to_alloc=%d, cache num=%d, "
+ "flags=%x, no_cached=%d, *sgv=%p", size, pages,
+ pages_to_alloc, cache_num, flags, no_cached, *sgv);
+
+ if (*sgv != NULL) {
+ obj = *sgv;
+
+ TRACE_MEM("Supplied obj %p, cache num %d", obj, obj->cache_num);
+
+ EXTRACHECKS_BUG_ON(obj->sg_count != 0);
+
+ if (unlikely(!sgv_check_allowed_mem(mem_lim, pages_to_alloc)))
+ goto out_fail_free_sg_entries;
+ allowed_mem_checked = true;
+
+ if (unlikely(sgv_hiwmk_check(pages_to_alloc) != 0))
+ goto out_fail_free_sg_entries;
+ hiwmk_checked = true;
+ } else if ((pages_to_alloc <= pool->max_cached_pages) && !no_cached) {
+ if (unlikely(!sgv_check_allowed_mem(mem_lim, pages_to_alloc)))
+ goto out_fail;
+ allowed_mem_checked = true;
+
+ obj = sgv_get_obj(pool, cache_num, pages_to_alloc, gfp_mask,
+ flags & SGV_POOL_ALLOC_GET_NEW);
+ if (unlikely(obj == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of "
+ "sgv_pool_obj failed (size %d)", size);
+ goto out_fail;
+ }
+
+ if (obj->sg_count != 0) {
+ TRACE_MEM("Cached obj %p", obj);
+ atomic_inc(&pool->cache_acc[cache_num].hit_alloc);
+ goto success;
+ }
+
+ if (flags & SGV_POOL_NO_ALLOC_ON_CACHE_MISS) {
+ if (!(flags & SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL))
+ goto out_fail_free;
+ }
+
+ TRACE_MEM("Brand new obj %p", obj);
+
+ if (pages_to_alloc <= sgv_max_local_pages) {
+ obj->sg_entries = obj->sg_entries_data;
+ sg_init_table(obj->sg_entries, pages_to_alloc);
+ TRACE_MEM("sg_entries %p", obj->sg_entries);
+ if (sgv_pool_clustered(pool)) {
+ obj->trans_tbl = (struct trans_tbl_ent *)
+ (obj->sg_entries + pages_to_alloc);
+ TRACE_MEM("trans_tbl %p", obj->trans_tbl);
+ /*
+ * No need to clear trans_tbl, if needed, it
+ * will be fully rewritten in
+ * sgv_alloc_sg_entries().
+ */
+ }
+ } else {
+ if (unlikely(sgv_alloc_arrays(obj, pages_to_alloc,
+ gfp_mask) != 0))
+ goto out_fail_free;
+ }
+
+ if ((flags & SGV_POOL_NO_ALLOC_ON_CACHE_MISS) &&
+ (flags & SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL))
+ goto out_return;
+
+ obj->allocator_priv = priv;
+
+ if (unlikely(sgv_hiwmk_check(pages_to_alloc) != 0))
+ goto out_fail_free_sg_entries;
+ hiwmk_checked = true;
+ } else {
+ int sz;
+
+ pages_to_alloc = pages;
+
+ if (unlikely(!sgv_check_allowed_mem(mem_lim, pages_to_alloc)))
+ goto out_fail;
+ allowed_mem_checked = true;
+
+ if (flags & SGV_POOL_NO_ALLOC_ON_CACHE_MISS)
+ goto out_return2;
+
+ sz = sizeof(*obj) + pages * sizeof(obj->sg_entries[0]);
+
+ obj = kmalloc(sz, gfp_mask);
+ if (unlikely(obj == NULL)) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of "
+ "sgv_pool_obj failed (size %d)", size);
+ goto out_fail;
+ }
+ memset(obj, 0, sizeof(*obj));
+
+ obj->owner_pool = pool;
+ cache_num = -1;
+ obj->cache_num = cache_num;
+ obj->pages = pages_to_alloc;
+ obj->allocator_priv = priv;
+
+ obj->sg_entries = obj->sg_entries_data;
+ sg_init_table(obj->sg_entries, pages);
+
+ if (unlikely(sgv_hiwmk_check(pages_to_alloc) != 0))
+ goto out_fail_free_sg_entries;
+ hiwmk_checked = true;
+
+ TRACE_MEM("Big or no_cached obj %p (size %d)", obj, sz);
+ }
+
+ obj->sg_count = sgv_alloc_sg_entries(obj->sg_entries,
+ pages_to_alloc, gfp_mask, pool->clustering_type,
+ obj->trans_tbl, &pool->alloc_fns, priv);
+ if (unlikely(obj->sg_count <= 0)) {
+ obj->sg_count = 0;
+ if ((flags & SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL) &&
+ (cache_num >= 0))
+ goto out_return1;
+ else
+ goto out_fail_free_sg_entries;
+ }
+
+ if (cache_num >= 0) {
+ atomic_add(pages_to_alloc - obj->sg_count,
+ &pool->cache_acc[cache_num].merged);
+ } else {
+ if (no_cached) {
+ atomic_add(pages_to_alloc,
+ &pool->other_pages);
+ atomic_add(pages_to_alloc - obj->sg_count,
+ &pool->other_merged);
+ } else {
+ atomic_add(pages_to_alloc,
+ &pool->big_pages);
+ atomic_add(pages_to_alloc - obj->sg_count,
+ &pool->big_merged);
+ }
+ }
+
+success:
+ if (cache_num >= 0) {
+ int sg;
+ atomic_inc(&pool->cache_acc[cache_num].total_alloc);
+ if (sgv_pool_clustered(pool))
+ cnt = obj->trans_tbl[pages-1].sg_num;
+ else
+ cnt = pages;
+ sg = cnt-1;
+ obj->orig_sg = sg;
+ obj->orig_length = obj->sg_entries[sg].length;
+ if (sgv_pool_clustered(pool)) {
+ obj->sg_entries[sg].length =
+ (pages - obj->trans_tbl[sg].pg_count) << PAGE_SHIFT;
+ }
+ } else {
+ cnt = obj->sg_count;
+ if (no_cached)
+ atomic_inc(&pool->other_alloc);
+ else
+ atomic_inc(&pool->big_alloc);
+ }
+
+ *count = cnt;
+ res = obj->sg_entries;
+ *sgv = obj;
+
+ if (size & ~PAGE_MASK)
+ obj->sg_entries[cnt-1].length -=
+ PAGE_SIZE - (size & ~PAGE_MASK);
+
+ TRACE_MEM("obj=%p, sg_entries %p (size=%d, pages=%d, sg_count=%d, "
+ "count=%d, last_len=%d)", obj, obj->sg_entries, size, pages,
+ obj->sg_count, *count, obj->sg_entries[obj->orig_sg].length);
+
+out:
+ return res;
+
+out_return:
+ obj->allocator_priv = priv;
+ obj->owner_pool = pool;
+
+out_return1:
+ *sgv = obj;
+ TRACE_MEM("Returning failed obj %p (count %d)", obj, *count);
+
+out_return2:
+ *count = pages_to_alloc;
+ res = NULL;
+ goto out_uncheck;
+
+out_fail_free_sg_entries:
+ if (obj->sg_entries != obj->sg_entries_data) {
+ if (obj->trans_tbl !=
+ (struct trans_tbl_ent *)obj->sg_entries_data) {
+ /* kfree() handles NULL parameter */
+ kfree(obj->trans_tbl);
+ obj->trans_tbl = NULL;
+ }
+ kfree(obj->sg_entries);
+ obj->sg_entries = NULL;
+ }
+
+out_fail_free:
+ if (cache_num >= 0) {
+ spin_lock_bh(&pool->sgv_pool_lock);
+ sgv_dec_cached_entries(pool, pages_to_alloc);
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ kmem_cache_free(pool->caches[obj->cache_num], obj);
+ } else
+ kfree(obj);
+
+out_fail:
+ res = NULL;
+ *count = 0;
+ *sgv = NULL;
+ TRACE_MEM("%s", "Allocation failed");
+
+out_uncheck:
+ if (hiwmk_checked)
+ sgv_hiwmk_uncheck(pages_to_alloc);
+ if (allowed_mem_checked)
+ sgv_uncheck_allowed_mem(mem_lim, pages_to_alloc);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_alloc);
+
+/**
+ * sgv_get_priv - return the private allocation data
+ *
+ * Allows to get the allocation private data for this SGV
+ * cache object. The private data supposed to be set by sgv_pool_alloc().
+ */
+void *sgv_get_priv(struct sgv_pool_obj *obj)
+{
+ return obj->allocator_priv;
+}
+EXPORT_SYMBOL_GPL(sgv_get_priv);
+
+/**
+ * sgv_pool_free - free previously allocated SG vector
+ * @sgv: the SGV object to free
+ * @mem_lim: memory limits
+ *
+ * Description:
+ * Frees previously allocated SG vector and updates memory limits
+ */
+void sgv_pool_free(struct sgv_pool_obj *obj, struct scst_mem_lim *mem_lim)
+{
+ int pages = (obj->sg_count != 0) ? obj->pages : 0;
+
+ TRACE_MEM("Freeing obj %p, cache num %d, pages %d, sg_entries %p, "
+ "sg_count %d, allocator_priv %p", obj, obj->cache_num, pages,
+ obj->sg_entries, obj->sg_count, obj->allocator_priv);
+ if (obj->cache_num >= 0) {
+ obj->sg_entries[obj->orig_sg].length = obj->orig_length;
+ sgv_put_obj(obj);
+ } else {
+ obj->owner_pool->alloc_fns.free_pages_fn(obj->sg_entries,
+ obj->sg_count, obj->allocator_priv);
+ kfree(obj);
+ sgv_hiwmk_uncheck(pages);
+ }
+
+ sgv_uncheck_allowed_mem(mem_lim, pages);
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_free);
+
+/**
+ * scst_alloc() - allocates an SG vector
+ *
+ * Allocates and returns pointer to SG vector with data size "size".
+ * In *count returned the count of entries in the vector.
+ * Returns NULL for failure.
+ */
+struct scatterlist *scst_alloc(int size, gfp_t gfp_mask, int *count)
+{
+ struct scatterlist *res;
+ int pages = (size >> PAGE_SHIFT) + ((size & ~PAGE_MASK) != 0);
+ struct sgv_pool_alloc_fns sys_alloc_fns = {
+ sgv_alloc_sys_pages, sgv_free_sys_sg_entries };
+ int no_fail = ((gfp_mask & __GFP_NOFAIL) == __GFP_NOFAIL);
+
+ atomic_inc(&sgv_other_total_alloc);
+
+ if (unlikely(sgv_hiwmk_check(pages) != 0)) {
+ if (!no_fail) {
+ res = NULL;
+ goto out;
+ } else {
+ /*
+ * Update active_pages_total since alloc can't fail.
+ * If it wasn't updated then the counter would cross 0
+ * on free again.
+ */
+ sgv_hiwmk_uncheck(-pages);
+ }
+ }
+
+ res = kmalloc(pages*sizeof(*res), gfp_mask);
+ if (res == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Unable to allocate sg for %d pages",
+ pages);
+ goto out_uncheck;
+ }
+
+ sg_init_table(res, pages);
+
+ /*
+ * If we allow use clustering here, we will have troubles in
+ * scst_free() to figure out how many pages are in the SG vector.
+ * So, always don't use clustering.
+ */
+ *count = sgv_alloc_sg_entries(res, pages, gfp_mask, sgv_no_clustering,
+ NULL, &sys_alloc_fns, NULL);
+ if (*count <= 0)
+ goto out_free;
+
+out:
+ TRACE_MEM("Alloced sg %p (count %d) \"no fail\" %d", res, *count, no_fail);
+ return res;
+
+out_free:
+ kfree(res);
+ res = NULL;
+
+out_uncheck:
+ if (!no_fail)
+ sgv_hiwmk_uncheck(pages);
+ goto out;
+}
+EXPORT_SYMBOL_GPL(scst_alloc);
+
+/**
+ * scst_free() - frees SG vector
+ *
+ * Frees SG vector returned by scst_alloc().
+ */
+void scst_free(struct scatterlist *sg, int count)
+{
+ TRACE_MEM("Freeing sg=%p", sg);
+
+ sgv_hiwmk_uncheck(count);
+
+ sgv_free_sys_sg_entries(sg, count, NULL);
+ kfree(sg);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_free);
+
+/* Must be called under sgv_pools_mutex */
+static void sgv_pool_init_cache(struct sgv_pool *pool, int cache_num)
+{
+ int size;
+ int pages;
+ struct sgv_pool_obj *obj;
+
+ atomic_set(&pool->cache_acc[cache_num].total_alloc, 0);
+ atomic_set(&pool->cache_acc[cache_num].hit_alloc, 0);
+ atomic_set(&pool->cache_acc[cache_num].merged, 0);
+
+ if (pool->single_alloc_pages == 0)
+ pages = 1 << cache_num;
+ else
+ pages = pool->single_alloc_pages;
+
+ if (pages <= sgv_max_local_pages) {
+ size = sizeof(*obj) + pages *
+ (sizeof(obj->sg_entries[0]) +
+ ((pool->clustering_type != sgv_no_clustering) ?
+ sizeof(obj->trans_tbl[0]) : 0));
+ } else if (pages <= sgv_max_trans_pages) {
+ /*
+ * sg_entries is allocated outside object,
+ * but trans_tbl is still embedded.
+ */
+ size = sizeof(*obj) + pages *
+ (((pool->clustering_type != sgv_no_clustering) ?
+ sizeof(obj->trans_tbl[0]) : 0));
+ } else {
+ size = sizeof(*obj);
+ /* both sgv and trans_tbl are kmalloc'ed() */
+ }
+
+ TRACE_MEM("pages=%d, size=%d", pages, size);
+
+ scnprintf(pool->cache_names[cache_num],
+ sizeof(pool->cache_names[cache_num]),
+ "%s-%uK", pool->name, (pages << PAGE_SHIFT) >> 10);
+ pool->caches[cache_num] = kmem_cache_create(
+ pool->cache_names[cache_num], size, 0, SCST_SLAB_FLAGS, NULL
+ );
+ return;
+}
+
+/* Must be called under sgv_pools_mutex */
+static int sgv_pool_init(struct sgv_pool *pool, const char *name,
+ enum sgv_clustering_types clustering_type, int single_alloc_pages,
+ int purge_interval)
+{
+ int res = -ENOMEM;
+ int i;
+
+ if (single_alloc_pages < 0) {
+ PRINT_ERROR("Wrong single_alloc_pages value %d",
+ single_alloc_pages);
+ res = -EINVAL;
+ goto out;
+ }
+
+ memset(pool, 0, sizeof(*pool));
+
+ atomic_set(&pool->big_alloc, 0);
+ atomic_set(&pool->big_pages, 0);
+ atomic_set(&pool->big_merged, 0);
+ atomic_set(&pool->other_alloc, 0);
+ atomic_set(&pool->other_pages, 0);
+ atomic_set(&pool->other_merged, 0);
+
+ pool->clustering_type = clustering_type;
+ pool->single_alloc_pages = single_alloc_pages;
+ if (purge_interval != 0) {
+ pool->purge_interval = purge_interval;
+ if (purge_interval < 0) {
+ /* Let's pretend that it's always scheduled */
+ pool->purge_work_scheduled = 1;
+ }
+ } else
+ pool->purge_interval = SGV_DEFAULT_PURGE_INTERVAL;
+ if (single_alloc_pages == 0) {
+ pool->max_caches = SGV_POOL_ELEMENTS;
+ pool->max_cached_pages = 1 << (SGV_POOL_ELEMENTS - 1);
+ } else {
+ pool->max_caches = 1;
+ pool->max_cached_pages = single_alloc_pages;
+ }
+ pool->alloc_fns.alloc_pages_fn = sgv_alloc_sys_pages;
+ pool->alloc_fns.free_pages_fn = sgv_free_sys_sg_entries;
+
+ TRACE_MEM("name %s, sizeof(*obj)=%zd, clustering_type=%d, "
+ "single_alloc_pages=%d, max_caches=%d, max_cached_pages=%d",
+ name, sizeof(struct sgv_pool_obj), clustering_type,
+ single_alloc_pages, pool->max_caches, pool->max_cached_pages);
+
+ strlcpy(pool->name, name, sizeof(pool->name)-1);
+
+ pool->owner_mm = current->mm;
+
+ for (i = 0; i < pool->max_caches; i++) {
+ sgv_pool_init_cache(pool, i);
+ if (pool->caches[i] == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocation of sgv_pool "
+ "cache %s(%d) failed", name, i);
+ goto out_free;
+ }
+ }
+
+ atomic_set(&pool->sgv_pool_ref, 1);
+ spin_lock_init(&pool->sgv_pool_lock);
+ INIT_LIST_HEAD(&pool->sorted_recycling_list);
+ for (i = 0; i < pool->max_caches; i++)
+ INIT_LIST_HEAD(&pool->recycling_lists[i]);
+
+ INIT_DELAYED_WORK(&pool->sgv_purge_work,
+ (void (*)(struct work_struct *))sgv_purge_work_fn);
+
+ spin_lock_bh(&sgv_pools_lock);
+ list_add_tail(&pool->sgv_pools_list_entry, &sgv_pools_list);
+ spin_unlock_bh(&sgv_pools_lock);
+
+ res = 0;
+
+out:
+ return res;
+
+out_free:
+ for (i = 0; i < pool->max_caches; i++) {
+ if (pool->caches[i]) {
+ kmem_cache_destroy(pool->caches[i]);
+ pool->caches[i] = NULL;
+ } else
+ break;
+ }
+ goto out;
+}
+
+static void sgv_evaluate_local_max_pages(void)
+{
+ int space4sgv_ttbl = PAGE_SIZE - sizeof(struct sgv_pool_obj);
+
+ sgv_max_local_pages = space4sgv_ttbl /
+ (sizeof(struct trans_tbl_ent) + sizeof(struct scatterlist));
+
+ sgv_max_trans_pages = space4sgv_ttbl / sizeof(struct trans_tbl_ent);
+
+ TRACE_MEM("sgv_max_local_pages %d, sgv_max_trans_pages %d",
+ sgv_max_local_pages, sgv_max_trans_pages);
+ return;
+}
+
+/**
+ * sgv_pool_flush - flushe the SGV pool
+ *
+ * Flushes, i.e. frees, all the cached entries in the SGV pool.
+ */
+void sgv_pool_flush(struct sgv_pool *pool)
+{
+ int i;
+
+ for (i = 0; i < pool->max_caches; i++) {
+ struct sgv_pool_obj *obj;
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+
+ while (!list_empty(&pool->recycling_lists[i])) {
+ obj = list_entry(pool->recycling_lists[i].next,
+ struct sgv_pool_obj, recycling_list_entry);
+
+ __sgv_purge_from_cache(obj);
+
+ spin_unlock_bh(&pool->sgv_pool_lock);
+
+ EXTRACHECKS_BUG_ON(obj->owner_pool != pool);
+ sgv_dtor_and_free(obj);
+
+ spin_lock_bh(&pool->sgv_pool_lock);
+ }
+ spin_unlock_bh(&pool->sgv_pool_lock);
+ }
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_flush);
+
+static void sgv_pool_deinit_put(struct sgv_pool *pool)
+{
+
+ cancel_delayed_work_sync(&pool->sgv_purge_work);
+
+ sgv_pool_flush(pool);
+
+ mutex_lock(&sgv_pools_mutex);
+ spin_lock_bh(&sgv_pools_lock);
+ list_del(&pool->sgv_pools_list_entry);
+ spin_unlock_bh(&sgv_pools_lock);
+ mutex_unlock(&sgv_pools_mutex);
+
+ scst_sgv_sysfs_put(pool);
+
+ /* pool can be dead here */
+ return;
+}
+
+/**
+ * sgv_pool_set_allocator - set custom pages allocator
+ * @pool: the cache
+ * @alloc_pages_fn: pages allocation function
+ * @free_pages_fn: pages freeing function
+ *
+ * Description:
+ * Allows to set custom pages allocator for the SGV pool.
+ * See the SGV pool documentation for more details.
+ */
+void sgv_pool_set_allocator(struct sgv_pool *pool,
+ struct page *(*alloc_pages_fn)(struct scatterlist *, gfp_t, void *),
+ void (*free_pages_fn)(struct scatterlist *, int, void *))
+{
+ pool->alloc_fns.alloc_pages_fn = alloc_pages_fn;
+ pool->alloc_fns.free_pages_fn = free_pages_fn;
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_set_allocator);
+
+/**
+ * sgv_pool_create - creates and initializes an SGV pool
+ * @name: the name of the SGV pool
+ * @clustered: sets type of the pages clustering.
+ * @single_alloc_pages: if 0, then the SGV pool will work in the set of
+ * power 2 size buffers mode. If >0, then the SGV pool will
+ * work in the fixed size buffers mode. In this case
+ * single_alloc_pages sets the size of each buffer in pages.
+ * @shared: sets if the SGV pool can be shared between devices or not.
+ * The cache sharing allowed only between devices created inside
+ * the same address space. If an SGV pool is shared, each
+ * subsequent call of sgv_pool_create() with the same cache name
+ * will not create a new cache, but instead return a reference
+ * to it.
+ * @purge_interval: sets the cache purging interval. I.e., an SG buffer
+ * will be freed if it's unused for time t
+ * purge_interval <= t < 2*purge_interval. If purge_interval
+ * is 0, then the default interval will be used (60 seconds).
+ * If purge_interval <0, then the automatic purging will be
+ * disabled.
+ *
+ * Description:
+ * Returns the resulting SGV pool or NULL in case of any error.
+ */
+struct sgv_pool *sgv_pool_create(const char *name,
+ enum sgv_clustering_types clustering_type,
+ int single_alloc_pages, bool shared, int purge_interval)
+{
+ struct sgv_pool *pool;
+ int rc;
+
+ mutex_lock(&sgv_pools_mutex);
+ list_for_each_entry(pool, &sgv_pools_list, sgv_pools_list_entry) {
+ if (strcmp(pool->name, name) == 0) {
+ if (shared) {
+ if (pool->owner_mm != current->mm) {
+ PRINT_ERROR("Attempt of a shared use "
+ "of SGV pool %s with "
+ "different MM", name);
+ goto out_err_unlock;
+ }
+ sgv_pool_get(pool);
+ goto out_unlock;
+ } else {
+ PRINT_ERROR("SGV pool %s already exists", name);
+ goto out_err_unlock;
+ }
+ }
+ }
+
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (pool == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "%s", "Allocation of sgv_pool failed");
+ goto out_unlock;
+ }
+
+ rc = sgv_pool_init(pool, name, clustering_type, single_alloc_pages,
+ purge_interval);
+ if (rc != 0)
+ goto out_free_unlock;
+
+ rc = scst_create_sgv_sysfs(pool);
+ if (rc != 0)
+ goto out_err_unlock_put;
+
+out_unlock:
+ mutex_unlock(&sgv_pools_mutex);
+ return pool;
+
+out_free_unlock:
+ kfree(pool);
+
+out_err_unlock:
+ pool = NULL;
+ goto out_unlock;
+
+out_err_unlock_put:
+ mutex_unlock(&sgv_pools_mutex);
+ sgv_pool_deinit_put(pool);
+ goto out_err_unlock;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_create);
+
+void sgv_pool_destroy(struct sgv_pool *pool)
+{
+ int i;
+
+ for (i = 0; i < pool->max_caches; i++) {
+ if (pool->caches[i])
+ kmem_cache_destroy(pool->caches[i]);
+ pool->caches[i] = NULL;
+ }
+
+ kfree(pool);
+ return;
+}
+
+/**
+ * sgv_pool_get - increase ref counter for the corresponding SGV pool
+ *
+ * Increases ref counter for the corresponding SGV pool
+ */
+void sgv_pool_get(struct sgv_pool *pool)
+{
+ atomic_inc(&pool->sgv_pool_ref);
+ TRACE_MEM("Incrementing sgv pool %p ref (new value %d)",
+ pool, atomic_read(&pool->sgv_pool_ref));
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_get);
+
+/**
+ * sgv_pool_put - decrease ref counter for the corresponding SGV pool
+ *
+ * Decreases ref counter for the corresponding SGV pool. If the ref
+ * counter reaches 0, the cache will be destroyed.
+ */
+void sgv_pool_put(struct sgv_pool *pool)
+{
+ TRACE_MEM("Decrementing sgv pool %p ref (new value %d)",
+ pool, atomic_read(&pool->sgv_pool_ref)-1);
+ if (atomic_dec_and_test(&pool->sgv_pool_ref))
+ sgv_pool_deinit_put(pool);
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_put);
+
+/**
+ * sgv_pool_del - deletes the corresponding SGV pool
+ * @pool: the cache to delete.
+ *
+ * Description:
+ * If the cache is shared, it will decrease its reference counter.
+ * If the reference counter reaches 0, the cache will be destroyed.
+ */
+void sgv_pool_del(struct sgv_pool *pool)
+{
+
+ sgv_pool_put(pool);
+ return;
+}
+EXPORT_SYMBOL_GPL(sgv_pool_del);
+
+/* Both parameters in pages */
+int scst_sgv_pools_init(unsigned long mem_hwmark, unsigned long mem_lwmark)
+{
+ int res = 0;
+
+ sgv_hi_wmk = mem_hwmark;
+ sgv_lo_wmk = mem_lwmark;
+
+ sgv_evaluate_local_max_pages();
+
+ sgv_norm_pool = sgv_pool_create("sgv", sgv_no_clustering, 0, false, 0);
+ if (sgv_norm_pool == NULL)
+ goto out_err;
+
+ sgv_norm_clust_pool = sgv_pool_create("sgv-clust",
+ sgv_full_clustering, 0, false, 0);
+ if (sgv_norm_clust_pool == NULL)
+ goto out_free_norm;
+
+ sgv_dma_pool = sgv_pool_create("sgv-dma", sgv_no_clustering, 0,
+ false, 0);
+ if (sgv_dma_pool == NULL)
+ goto out_free_clust;
+
+ sgv_shrinker.shrink = sgv_shrink;
+ sgv_shrinker.seeks = DEFAULT_SEEKS;
+ register_shrinker(&sgv_shrinker);
+
+out:
+ return res;
+
+out_free_clust:
+ sgv_pool_deinit_put(sgv_norm_clust_pool);
+
+out_free_norm:
+ sgv_pool_deinit_put(sgv_norm_pool);
+
+out_err:
+ res = -ENOMEM;
+ goto out;
+}
+
+void scst_sgv_pools_deinit(void)
+{
+
+ unregister_shrinker(&sgv_shrinker);
+
+ sgv_pool_deinit_put(sgv_dma_pool);
+ sgv_pool_deinit_put(sgv_norm_pool);
+ sgv_pool_deinit_put(sgv_norm_clust_pool);
+
+ flush_scheduled_work();
+ return;
+}
+
+ssize_t sgv_sysfs_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct sgv_pool *pool;
+ int i, total = 0, hit = 0, merged = 0, allocated = 0;
+ int oa, om, res;
+
+ pool = container_of(kobj, struct sgv_pool, sgv_kobj);
+
+ for (i = 0; i < SGV_POOL_ELEMENTS; i++) {
+ int t;
+
+ hit += atomic_read(&pool->cache_acc[i].hit_alloc);
+ total += atomic_read(&pool->cache_acc[i].total_alloc);
+
+ t = atomic_read(&pool->cache_acc[i].total_alloc) -
+ atomic_read(&pool->cache_acc[i].hit_alloc);
+ allocated += t * (1 << i);
+ merged += atomic_read(&pool->cache_acc[i].merged);
+ }
+
+ res = sprintf(buf, "%-30s %-11s %-11s %-11s %-11s", "Name", "Hit", "Total",
+ "% merged", "Cached (P/I/O)");
+
+ res += sprintf(&buf[res], "\n%-30s %-11d %-11d %-11d %d/%d/%d\n",
+ pool->name, hit, total,
+ (allocated != 0) ? merged*100/allocated : 0,
+ pool->cached_pages, pool->inactive_cached_pages,
+ pool->cached_entries);
+
+ for (i = 0; i < SGV_POOL_ELEMENTS; i++) {
+ int t = atomic_read(&pool->cache_acc[i].total_alloc) -
+ atomic_read(&pool->cache_acc[i].hit_alloc);
+ allocated = t * (1 << i);
+ merged = atomic_read(&pool->cache_acc[i].merged);
+
+ res += sprintf(&buf[res], " %-28s %-11d %-11d %d\n",
+ pool->cache_names[i],
+ atomic_read(&pool->cache_acc[i].hit_alloc),
+ atomic_read(&pool->cache_acc[i].total_alloc),
+ (allocated != 0) ? merged*100/allocated : 0);
+ }
+
+ allocated = atomic_read(&pool->big_pages);
+ merged = atomic_read(&pool->big_merged);
+ oa = atomic_read(&pool->other_pages);
+ om = atomic_read(&pool->other_merged);
+
+ res += sprintf(&buf[res], " %-40s %d/%-9d %d/%d\n", "big/other",
+ atomic_read(&pool->big_alloc), atomic_read(&pool->other_alloc),
+ (allocated != 0) ? merged*100/allocated : 0,
+ (oa != 0) ? om/oa : 0);
+
+ return res;
+}
+
+ssize_t sgv_sysfs_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ struct sgv_pool *pool;
+ int i;
+
+ pool = container_of(kobj, struct sgv_pool, sgv_kobj);
+
+ for (i = 0; i < SGV_POOL_ELEMENTS; i++) {
+ atomic_set(&pool->cache_acc[i].hit_alloc, 0);
+ atomic_set(&pool->cache_acc[i].total_alloc, 0);
+ atomic_set(&pool->cache_acc[i].merged, 0);
+ }
+
+ atomic_set(&pool->big_pages, 0);
+ atomic_set(&pool->big_merged, 0);
+ atomic_set(&pool->big_alloc, 0);
+ atomic_set(&pool->other_pages, 0);
+ atomic_set(&pool->other_merged, 0);
+ atomic_set(&pool->other_alloc, 0);
+
+ PRINT_INFO("Statistics for SGV pool %s resetted", pool->name);
+ return count;
+}
+
+ssize_t sgv_sysfs_global_stat_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct sgv_pool *pool;
+ int inactive_pages = 0, res;
+
+ spin_lock_bh(&sgv_pools_lock);
+ list_for_each_entry(pool, &sgv_active_pools_list,
+ sgv_active_pools_list_entry) {
+ inactive_pages += pool->inactive_cached_pages;
+ }
+ spin_unlock_bh(&sgv_pools_lock);
+
+ res = sprintf(buf, "%-42s %d/%d\n%-42s %d/%d\n%-42s %d/%d\n"
+ "%-42s %-11d\n",
+ "Inactive/active pages", inactive_pages,
+ atomic_read(&sgv_pages_total) - inactive_pages,
+ "Hi/lo watermarks [pages]", sgv_hi_wmk, sgv_lo_wmk,
+ "Hi watermark releases/failures",
+ atomic_read(&sgv_releases_on_hiwmk),
+ atomic_read(&sgv_releases_on_hiwmk_failed),
+ "Other allocs", atomic_read(&sgv_other_total_alloc));
+ return res;
+}
+
+ssize_t sgv_sysfs_global_stat_reset(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+
+ atomic_set(&sgv_releases_on_hiwmk, 0);
+ atomic_set(&sgv_releases_on_hiwmk_failed, 0);
+ atomic_set(&sgv_other_total_alloc, 0);
+
+ PRINT_INFO("%s", "Global SGV pool statistics resetted");
+ return count;
+}
+
diff -uprN orig/linux-2.6.33/Documentation/scst/sgv_cache.txt linux-2.6.33/Documentation/scst/sgv_cache.txt
--- orig/linux-2.6.33/Documentation/scst/sgv_cache.txt
+++ linux-2.6.33/Documentation/scst/sgv_cache.txt
@@ -0,0 +1,224 @@
+ SCST SGV CACHE.
+
+ PROGRAMMING INTERFACE DESCRIPTION.
+
+ For SCST version 1.0.2
+
+SCST SGV cache is a memory management subsystem in SCST. One can call it
+a "memory pool", but Linux kernel already have a mempool interface,
+which serves different purposes. SGV cache provides to SCST core, target
+drivers and backend dev handlers facilities to allocate, build and cache
+SG vectors for data buffers. The main advantage of it is the caching
+facility, when it doesn't free to the system each vector, which is not
+used anymore, but keeps it for a while (possibly indefinitely) to let it
+be reused by the next consecutive command. This allows to:
+
+ - Reduce commands processing latencies and, hence, improve performance;
+
+ - Make commands processing latencies predictable, which is essential
+ for RT applications.
+
+The freed SG vectors are kept by the SGV cache either for some (possibly
+indefinite) time, or, optionally, until the system needs more memory and
+asks to free some using the set_shrinker() interface. Also the SGV cache
+allows to:
+
+ - Cluster pages together. "Cluster" means merging adjacent pages in a
+single SG entry. It allows to have less SG entries in the resulting SG
+vector, hence improve performance handling it as well as allow to
+work with bigger buffers on hardware with limited SG capabilities.
+
+ - Set custom page allocator functions. For instance, scst_user device
+handler uses this facility to eliminate unneeded mapping/unmapping of
+user space pages and avoid unneeded IOCTL calls for buffers allocations.
+In fileio_tgt application, which uses a regular malloc() function to
+allocate data buffers, this facility allows ~30% less CPU load and
+considerable performance increase.
+
+ - Prevent each initiator or all initiators altogether to allocate too
+much memory and DoS the target. Consider 10 initiators, which can have
+access to 10 devices each. Any of them can queue up to 64 commands, each
+can transfer up to 1MB of data. So, all of them in a peak can allocate
+up to 10*10*64 = ~6.5GB of memory for data buffers. This amount must be
+limited somehow and the SGV cache performs this function.
+
+From implementation POV the SGV cache is a simple extension of the kmem
+cache. It can work in 2 modes:
+
+1. With fixed size buffers.
+
+2. With a set of power 2 size buffers. In this mode each SGV cache
+(struct sgv_pool) has SGV_POOL_ELEMENTS (11 currently) of kmem caches.
+Each of those kmem caches keeps SGV cache objects (struct sgv_pool_obj)
+corresponding to SG vectors with size of order X pages. For instance,
+request to allocate 4 pages will be served from kmem cache[2], since the
+order of the of number of requested pages is 2. If later request to
+allocate 11KB comes, the same SG vector with 4 pages will be reused (see
+below). This mode is in average allows less memory overhead comparing
+with the fixed size buffers mode.
+
+Consider how the SGV cache works in the set of buffers mode. When a
+request to allocate new SG vector comes, sgv_pool_alloc() via
+sgv_get_obj() checks if there is already a cached vector with that
+order. If yes, then that vector will be reused and its length, if
+necessary, will be modified to match the requested size. In the above
+example request for 11KB buffer, 4 pages vector will be reused and
+modified using trans_tbl to contain 3 pages and the last entry will be
+modified to contain the requested length - 2*PAGE_SIZE. If there is no
+cached object, then a new sgv_pool_obj will be allocated from the
+corresponding kmem cache, chosen by the order of number of requested
+pages. Then that vector will be filled by pages and returned.
+
+In the fixed size buffers mode the SGV cache works similarly, except
+that it always allocate buffer with the predefined fixed size. I.e.
+even for 4K request the whole buffer with predefined size, say, 1MB,
+will be used.
+
+In both modes, if size of a request exceeds the maximum allowed for
+caching buffer size, the requested buffer will be allocated, but not
+cached.
+
+Freed cached sgv_pool_obj objects are actually freed to the system
+either by the purge work, which is scheduled once in 60 seconds, or in
+sgv_shrink() called by system, when it's asking for memory.
+
+ Interface.
+
+struct sgv_pool *sgv_pool_create(const char *name,
+ enum sgv_clustering_types clustered, int single_alloc_pages,
+ bool shared, int purge_interval)
+
+This function creates and initializes an SGV cache. It has the following
+arguments:
+
+ - name - the name of the SGV cache
+
+ - clustered - sets type of the pages clustering. The type can be:
+
+ * sgv_no_clustering - no clustering performed.
+
+ * sgv_tail_clustering - a page will only be merged with the latest
+ previously allocated page, so the order of pages in the SG will be
+ preserved
+
+ * sgv_full_clustering - free merging of pages at any place in
+ the SG is allowed. This mode usually provides the best merging
+ rate.
+
+ - single_alloc_pages - if 0, then the SGV cache will work in the set of
+ power 2 size buffers mode. If >0, then the SGV cache will work in the
+ fixed size buffers mode. In this case single_alloc_pages sets the
+ size of each buffer in pages.
+
+ - shared - sets if the SGV cache can be shared between devices or not.
+ The cache sharing allowed only between devices created inside the same
+ address space. If an SGV cache is shared, each subsequent call of
+ sgv_pool_create() with the same cache name will not create a new cache,
+ but instead return a reference to it.
+
+ - purge_interval - sets the cache purging interval. I.e. an SG buffer
+ will be freed if it's unused for time t purge_interval <= t <
+ 2*purge_interval. If purge_interval is 0, then the default interval
+ will be used (60 seconds). If purge_interval <0, then the automatic
+ purging will be disabled. Shrinking by the system's demand will also
+ be disabled.
+
+Returns the resulting SGV cache or NULL in case of any error.
+
+void sgv_pool_del(struct sgv_pool *pool)
+
+This function deletes the corresponding SGV cache. If the cache is
+shared, it will decrease its reference counter. If the reference counter
+reaches 0, the cache will be destroyed.
+
+void sgv_pool_flush(struct sgv_pool *pool)
+
+This function flushes, i.e. frees, all the cached entries in the SGV
+cache.
+
+void sgv_pool_set_allocator(struct sgv_pool *pool,
+ struct page *(*alloc_pages_fn)(struct scatterlist *sg, gfp_t gfp, void *priv),
+ void (*free_pages_fn)(struct scatterlist *sg, int sg_count, void *priv));
+
+This function allows to set for the SGV cache a custom pages allocator. For
+instance, scst_user uses such function to supply to the cache mapped from
+user space pages.
+
+alloc_pages_fn() has the following parameters:
+
+ - sg - SG entry, to which the allocated page should be added.
+
+ - gfp - the allocation GFP flags
+
+ - priv - pointer to a private data supplied to sgv_pool_alloc()
+
+This function should return the allocated page or NULL, if no page was
+allocated.
+
+free_pages_fn() has the following parameters:
+
+ - sg - SG vector to free
+
+ - sg_count - number of SG entries in the sg
+
+ - priv - pointer to a private data supplied to the corresponding sgv_pool_alloc()
+
+struct scatterlist *sgv_pool_alloc(struct sgv_pool *pool, unsigned int size,
+ gfp_t gfp_mask, int flags, int *count,
+ struct sgv_pool_obj **sgv, struct scst_mem_lim *mem_lim, void *priv)
+
+This function allocates an SG vector from the SGV cache. It has the
+following parameters:
+
+ - pool - the cache to alloc from
+
+ - size - size of the resulting SG vector in bytes
+
+ - gfp_mask - the allocation mask
+
+ - flags - the allocation flags. The following flags are possible and
+ can be set using OR operation:
+
+ * SGV_POOL_ALLOC_NO_CACHED - the SG vector must not be cached.
+
+ * SGV_POOL_NO_ALLOC_ON_CACHE_MISS - don't do an allocation on a
+ cache miss.
+
+ * SGV_POOL_RETURN_OBJ_ON_ALLOC_FAIL - return an empty SGV object,
+ i.e. without the SG vector, if the allocation can't be completed.
+ For instance, because SGV_POOL_NO_ALLOC_ON_CACHE_MISS flag set.
+
+ - count - the resulting count of SG entries in the resulting SG vector.
+
+ - sgv - the resulting SGV object. It should be used to free the
+ resulting SG vector.
+
+ - mem_lim - memory limits, see below.
+
+ - priv - pointer to private for this allocation data. This pointer will
+ be supplied to alloc_pages_fn() and free_pages_fn() and can be
+ retrieved by sgv_get_priv().
+
+This function returns pointer to the resulting SG vector or NULL in case
+of any error.
+
+void sgv_pool_free(struct sgv_pool_obj *sgv, struct scst_mem_lim *mem_lim)
+
+This function frees previously allocated SG vector, referenced by SGV
+cache object sgv.
+
+void *sgv_get_priv(struct sgv_pool_obj *sgv)
+
+This function allows to get the allocation private data for this SGV
+cache object sgv. The private data are set by sgv_pool_alloc().
+
+void scst_init_mem_lim(struct scst_mem_lim *mem_lim)
+
+This function initializes memory limits structure mem_lim according to
+the current system configuration. This structure should be latter used
+to track and limit allocated by one or more SGV caches memory.
+
+ Runtime information and statistics.
+
+Runtime information and statistics is available in /sys/kernel/scst_tgt/sgv.
+
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 8/12/1/5] SCST sysfs interface
[not found] ` <4BC44D08.4060907@vlnb.net>
` (6 preceding siblings ...)
2010-04-13 13:06 ` [PATCH][RFC 7/12/1/5] SCST SGV cache Vladislav Bolkhovitin
@ 2010-04-13 13:06 ` Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 9/12/1/5] SCST debugging support Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 10/12/1/5] SCST external modules support Vladislav Bolkhovitin
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:06 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver, Daniel Henrique Debonzi
This patch contains file scst_sysfs.c.
Signed-off-by: Daniel Henrique Debonzi <debonzi@linux.vnet.ibm.com>
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_sysfs.c | 3884 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 3884 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/scst_sysfs.c linux-2.6.33/drivers/scst/scst_sysfs.c
--- orig/linux-2.6.33/drivers/scst/scst_sysfs.c
+++ linux-2.6.33/drivers/scst/scst_sysfs.c
@@ -0,0 +1,3884 @@
+/*
+ * scst_sysfs.c
+ *
+ * Copyright (C) 2009 Daniel Henrique Debonzi <debonzi@linux.vnet.ibm.com>
+ * Copyright (C) 2009 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2009 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kobject.h>
+#include <linux/string.h>
+#include <linux/sysfs.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/ctype.h>
+
+#include "scst.h"
+#include "scst_priv.h"
+#include "scst_mem.h"
+
+static DECLARE_COMPLETION(scst_sysfs_root_release_completion);
+
+static struct kobject scst_sysfs_root_kobj;
+static struct kobject *scst_targets_kobj;
+static struct kobject *scst_devices_kobj;
+static struct kobject *scst_sgv_kobj;
+static struct kobject *scst_handlers_kobj;
+
+/* Regular SCST sysfs operations */
+struct sysfs_ops scst_sysfs_ops;
+EXPORT_SYMBOL_GPL(scst_sysfs_ops);
+
+static const char *scst_dev_handler_types[] = {
+ "Direct-access device (e.g., magnetic disk)",
+ "Sequential-access device (e.g., magnetic tape)",
+ "Printer device",
+ "Processor device",
+ "Write-once device (e.g., some optical disks)",
+ "CD-ROM device",
+ "Scanner device (obsolete)",
+ "Optical memory device (e.g., some optical disks)",
+ "Medium changer device (e.g., jukeboxes)",
+ "Communications device (obsolete)",
+ "Defined by ASC IT8 (Graphic arts pre-press devices)",
+ "Defined by ASC IT8 (Graphic arts pre-press devices)",
+ "Storage array controller device (e.g., RAID)",
+ "Enclosure services device",
+ "Simplified direct-access device (e.g., magnetic disk)",
+ "Optical card reader/writer device"
+};
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static DEFINE_MUTEX(scst_log_mutex);
+
+static struct scst_trace_log scst_trace_tbl[] = {
+ { TRACE_OUT_OF_MEM, "out_of_mem" },
+ { TRACE_MINOR, "minor" },
+ { TRACE_SG_OP, "sg" },
+ { TRACE_MEMORY, "mem" },
+ { TRACE_BUFF, "buff" },
+ { TRACE_PID, "pid" },
+ { TRACE_LINE, "line" },
+ { TRACE_FUNCTION, "function" },
+ { TRACE_DEBUG, "debug" },
+ { TRACE_SPECIAL, "special" },
+ { TRACE_SCSI, "scsi" },
+ { TRACE_MGMT, "mgmt" },
+ { TRACE_MGMT_DEBUG, "mgmt_dbg" },
+ { TRACE_FLOW_CONTROL, "flow_control" },
+ { 0, NULL }
+};
+
+static struct scst_trace_log scst_local_trace_tbl[] = {
+ { TRACE_RTRY, "retry" },
+ { TRACE_SCSI_SERIALIZING, "scsi_serializing" },
+ { TRACE_RCV_BOT, "recv_bot" },
+ { TRACE_SND_BOT, "send_bot" },
+ { TRACE_RCV_TOP, "recv_top" },
+ { TRACE_SND_TOP, "send_top" },
+ { 0, NULL }
+};
+
+static ssize_t scst_trace_level_show(const struct scst_trace_log *local_tbl,
+ unsigned long log_level, char *buf, const char *help);
+static int scst_write_trace(const char *buf, size_t length,
+ unsigned long *log_level, unsigned long default_level,
+ const char *name, const struct scst_trace_log *tbl);
+
+#endif /* defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_luns_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_tgt_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_tgt_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_tgt_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_tgt_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_ini_group_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_ini_group_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_rel_tgt_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_rel_tgt_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_ini_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_acg_ini_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_acg_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acg_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf);
+static ssize_t scst_acg_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count);
+static ssize_t scst_acn_file_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf);
+
+static void scst_sysfs_release(struct kobject *kobj)
+{
+ kfree(kobj);
+}
+
+/*
+ * Target Template
+ */
+
+static void scst_tgtt_release(struct kobject *kobj)
+{
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ complete_all(&tgtt->tgtt_kobj_release_cmpl);
+
+ scst_tgtt_cleanup(tgtt);
+ return;
+}
+
+static struct kobj_type tgtt_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_tgtt_release,
+};
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static ssize_t scst_tgtt_trace_level_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ return scst_trace_level_show(tgtt->trace_tbl,
+ tgtt->trace_flags ? *tgtt->trace_flags : 0, buf,
+ tgtt->trace_tbl_help);
+}
+
+static ssize_t scst_tgtt_trace_level_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ if (mutex_lock_interruptible(&scst_log_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = scst_write_trace(buf, count, tgtt->trace_flags,
+ tgtt->default_trace_flags, tgtt->name, tgtt->trace_tbl);
+
+ mutex_unlock(&scst_log_mutex);
+
+out:
+ return res;
+}
+
+static struct kobj_attribute tgtt_trace_attr =
+ __ATTR(trace_level, S_IRUGO | S_IWUSR,
+ scst_tgtt_trace_level_show, scst_tgtt_trace_level_store);
+
+#endif /* #if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_tgtt_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ char *help = "Usage: echo \"add_target target_name [parameters]\" "
+ ">mgmt\n"
+ " echo \"del_target target_name\" >mgmt\n"
+ "%s"
+ "\n"
+ "where parameters are one or more "
+ "param_name=value pairs separated by ';'\n"
+ "%s%s";
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ if (tgtt->add_target_parameters_help != NULL)
+ return sprintf(buf, help,
+ (tgtt->mgmt_cmd_help) ? tgtt->mgmt_cmd_help : "",
+ "\nThe following parameters available: ",
+ tgtt->add_target_parameters_help);
+ else
+ return sprintf(buf, help,
+ (tgtt->mgmt_cmd_help) ? tgtt->mgmt_cmd_help : "",
+ "", "");
+}
+
+static ssize_t scst_tgtt_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ char *buffer, *p, *pp, *target_name;
+ struct scst_tgt_template *tgtt;
+
+ tgtt = container_of(kobj, struct scst_tgt_template, tgtt_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ pp = buffer;
+ if (pp[strlen(pp) - 1] == '\n')
+ pp[strlen(pp) - 1] = '\0';
+
+ p = scst_get_next_lexem(&pp);
+
+ if (strcasecmp("add_target", p) == 0) {
+ target_name = scst_get_next_lexem(&pp);
+ if (*target_name == '\0') {
+ PRINT_ERROR("%s", "Target name required");
+ res = -EINVAL;
+ goto out_free;
+ }
+ res = tgtt->add_target(target_name, pp);
+ } else if (strcasecmp("del_target", p) == 0) {
+ target_name = scst_get_next_lexem(&pp);
+ if (*target_name == '\0') {
+ PRINT_ERROR("%s", "Target name required");
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ p = scst_get_next_lexem(&pp);
+ if (*p != '\0')
+ goto out_syntax_err;
+
+ res = tgtt->del_target(target_name);
+ } else if (tgtt->mgmt_cmd != NULL) {
+ scst_restore_token_str(p, pp);
+ res = tgtt->mgmt_cmd(buffer);
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ if (res == 0)
+ res = count;
+
+out_free:
+ kfree(buffer);
+
+out:
+ return res;
+
+out_syntax_err:
+ PRINT_ERROR("Syntax error on \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+}
+
+static struct kobj_attribute scst_tgtt_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_tgtt_mgmt_show,
+ scst_tgtt_mgmt_store);
+
+int scst_create_tgtt_sysfs(struct scst_tgt_template *tgtt)
+{
+ int retval = 0;
+ const struct attribute **pattr;
+
+ init_completion(&tgtt->tgtt_kobj_release_cmpl);
+
+ tgtt->tgtt_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&tgtt->tgtt_kobj, &tgtt_ktype,
+ scst_targets_kobj, tgtt->name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgtt %s to sysfs", tgtt->name);
+ goto out;
+ }
+
+ /*
+ * In case of errors there's no need for additional cleanup, because
+ * it will be done by the _put function() called by the caller.
+ */
+
+ if (tgtt->add_target != NULL) {
+ retval = sysfs_create_file(&tgtt->tgtt_kobj,
+ &scst_tgtt_mgmt.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add mgmt attr for target driver %s",
+ tgtt->name);
+ goto out;
+ }
+ }
+
+ pattr = tgtt->tgtt_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ TRACE_DBG("Creating attr %s for target driver %s",
+ (*pattr)->name, tgtt->name);
+ retval = sysfs_create_file(&tgtt->tgtt_kobj, *pattr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add attr %s for target "
+ "driver %s", (*pattr)->name,
+ tgtt->name);
+ goto out;
+ }
+ pattr++;
+ }
+ }
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (tgtt->trace_flags != NULL) {
+ retval = sysfs_create_file(&tgtt->tgtt_kobj,
+ &tgtt_trace_attr.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add trace_flag for target "
+ "driver %s", tgtt->name);
+ goto out;
+ }
+ }
+#endif
+
+out:
+ return retval;
+}
+
+void scst_tgtt_sysfs_put(struct scst_tgt_template *tgtt)
+{
+
+ if (tgtt->tgtt_kobj_initialized) {
+ int rc;
+
+ kobject_del(&tgtt->tgtt_kobj);
+ kobject_put(&tgtt->tgtt_kobj);
+
+ rc = wait_for_completion_timeout(&tgtt->tgtt_kobj_release_cmpl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for target template %s...", tgtt->name);
+ wait_for_completion(&tgtt->tgtt_kobj_release_cmpl);
+ PRINT_INFO("Done waiting for releasing sysfs "
+ "entry for target template %s", tgtt->name);
+ }
+ } else
+ scst_tgtt_cleanup(tgtt);
+ return;
+}
+
+/*
+ * Target directory implementation
+ */
+
+static void scst_tgt_release(struct kobject *kobj)
+{
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ /* Let's make lockdep happy */
+ up_write(&tgt->tgt_attr_rwsem);
+
+ scst_free_tgt(tgt);
+ return;
+}
+
+static ssize_t scst_tgt_attr_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ int res;
+ struct kobj_attribute *kobj_attr;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ if (down_read_trylock(&tgt->tgt_attr_rwsem) == 0) {
+ res = -ENOENT;
+ goto out;
+ }
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ res = kobj_attr->show(kobj, kobj_attr, buf);
+
+ up_read(&tgt->tgt_attr_rwsem);
+
+out:
+ return res;
+}
+
+static ssize_t scst_tgt_attr_store(struct kobject *kobj,
+ struct attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct kobj_attribute *kobj_attr;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ if (down_read_trylock(&tgt->tgt_attr_rwsem) == 0) {
+ res = -ENOENT;
+ goto out;
+ }
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ res = kobj_attr->store(kobj, kobj_attr, buf, count);
+
+ up_read(&tgt->tgt_attr_rwsem);
+
+out:
+ return res;
+}
+
+static struct sysfs_ops scst_tgt_sysfs_ops = {
+ .show = scst_tgt_attr_show,
+ .store = scst_tgt_attr_store,
+};
+
+static struct kobj_type tgt_ktype = {
+ .sysfs_ops = &scst_tgt_sysfs_ops,
+ .release = scst_tgt_release,
+};
+
+static void scst_acg_release(struct kobject *kobj)
+{
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ scst_destroy_acg(acg);
+ return;
+}
+
+static struct kobj_type acg_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_acg_release,
+};
+
+static struct kobj_attribute scst_luns_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_luns_mgmt_show,
+ scst_luns_mgmt_store);
+
+static struct kobj_attribute scst_acg_luns_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_luns_mgmt_show,
+ scst_acg_luns_mgmt_store);
+
+static struct kobj_attribute scst_acg_ini_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_acg_ini_mgmt_show,
+ scst_acg_ini_mgmt_store);
+
+static struct kobj_attribute scst_ini_group_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_ini_group_mgmt_show,
+ scst_ini_group_mgmt_store);
+
+static struct kobj_attribute scst_tgt_addr_method =
+ __ATTR(addr_method, S_IRUGO | S_IWUSR, scst_tgt_addr_method_show,
+ scst_tgt_addr_method_store);
+
+static struct kobj_attribute scst_tgt_io_grouping_type =
+ __ATTR(io_grouping_type, S_IRUGO | S_IWUSR,
+ scst_tgt_io_grouping_type_show,
+ scst_tgt_io_grouping_type_store);
+
+static struct kobj_attribute scst_rel_tgt_id =
+ __ATTR(rel_tgt_id, S_IRUGO | S_IWUSR, scst_rel_tgt_id_show,
+ scst_rel_tgt_id_store);
+
+static struct kobj_attribute scst_acg_addr_method =
+ __ATTR(addr_method, S_IRUGO | S_IWUSR, scst_acg_addr_method_show,
+ scst_acg_addr_method_store);
+
+static struct kobj_attribute scst_acg_io_grouping_type =
+ __ATTR(io_grouping_type, S_IRUGO | S_IWUSR,
+ scst_acg_io_grouping_type_show,
+ scst_acg_io_grouping_type_store);
+
+static ssize_t scst_tgt_enable_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt *tgt;
+ int res;
+ bool enabled;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ enabled = tgt->tgtt->is_target_enabled(tgt);
+
+ res = sprintf(buf, "%d\n", enabled ? 1 : 0);
+ return res;
+}
+
+static ssize_t scst_tgt_enable_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_tgt *tgt;
+ bool enable;
+
+ if (buf == NULL) {
+ PRINT_ERROR("%s: NULL buffer?", __func__);
+ res = -EINVAL;
+ goto out;
+ }
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ switch (buf[0]) {
+ case '0':
+ enable = false;
+ break;
+ case '1':
+ if (tgt->rel_tgt_id == 0) {
+ res = gen_relative_target_port_id(&tgt->rel_tgt_id);
+ if (res)
+ goto out;
+ PRINT_INFO("Using autogenerated rel ID %d for target "
+ "%s", tgt->rel_tgt_id, tgt->tgt_name);
+ } else {
+ if (!scst_is_relative_target_port_id_unique(
+ tgt->rel_tgt_id, tgt)) {
+ PRINT_ERROR("Relative port id %d is not unique",
+ tgt->rel_tgt_id);
+ res = -EBADSLT;
+ goto out;
+ }
+ }
+ enable = true;
+ break;
+ default:
+ PRINT_ERROR("%s: Requested action not understood: %s",
+ __func__, buf);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = tgt->tgtt->enable_target(tgt, enable);
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute tgt_enable_attr =
+ __ATTR(enabled, S_IRUGO | S_IWUSR,
+ scst_tgt_enable_show, scst_tgt_enable_store);
+
+int scst_create_tgt_sysfs(struct scst_tgt *tgt)
+{
+ int retval;
+ const struct attribute **pattr;
+
+ init_rwsem(&tgt->tgt_attr_rwsem);
+
+ tgt->tgt_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&tgt->tgt_kobj, &tgt_ktype,
+ &tgt->tgtt->tgtt_kobj, tgt->tgt_name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt %s to sysfs", tgt->tgt_name);
+ goto out;
+ }
+
+ /*
+ * In case of errors there's no need for additional cleanup, because
+ * it will be done by the _put function() called by the caller.
+ */
+
+ if ((tgt->tgtt->enable_target != NULL) &&
+ (tgt->tgtt->is_target_enabled != NULL)) {
+ retval = sysfs_create_file(&tgt->tgt_kobj,
+ &tgt_enable_attr.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add attr %s to sysfs",
+ tgt_enable_attr.attr.name);
+ goto out;
+ }
+ }
+
+ tgt->tgt_sess_kobj = kobject_create_and_add("sessions", &tgt->tgt_kobj);
+ if (tgt->tgt_sess_kobj == NULL) {
+ PRINT_ERROR("Can't create sess kobj for tgt %s", tgt->tgt_name);
+ goto out_nomem;
+ }
+
+ tgt->tgt_luns_kobj = kobject_create_and_add("luns", &tgt->tgt_kobj);
+ if (tgt->tgt_luns_kobj == NULL) {
+ PRINT_ERROR("Can't create luns kobj for tgt %s", tgt->tgt_name);
+ goto out_nomem;
+ }
+
+ retval = sysfs_create_file(tgt->tgt_luns_kobj, &scst_luns_mgmt.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_luns_mgmt.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ tgt->tgt_ini_grp_kobj = kobject_create_and_add("ini_groups",
+ &tgt->tgt_kobj);
+ if (tgt->tgt_ini_grp_kobj == NULL) {
+ PRINT_ERROR("Can't create ini_grp kobj for tgt %s",
+ tgt->tgt_name);
+ goto out_nomem;
+ }
+
+ retval = sysfs_create_file(tgt->tgt_ini_grp_kobj,
+ &scst_ini_group_mgmt.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_ini_group_mgmt.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ retval = sysfs_create_file(&tgt->tgt_kobj,
+ &scst_rel_tgt_id.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_rel_tgt_id.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ retval = sysfs_create_file(&tgt->tgt_kobj,
+ &scst_tgt_addr_method.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_tgt_addr_method.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ retval = sysfs_create_file(&tgt->tgt_kobj,
+ &scst_tgt_io_grouping_type.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add attribute %s for tgt %s",
+ scst_tgt_io_grouping_type.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ pattr = tgt->tgtt->tgt_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ TRACE_DBG("Creating attr %s for tgt %s", (*pattr)->name,
+ tgt->tgt_name);
+ retval = sysfs_create_file(&tgt->tgt_kobj, *pattr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ (*pattr)->name, tgt->tgt_name);
+ goto out;
+ }
+ pattr++;
+ }
+ }
+
+out:
+ return retval;
+
+out_nomem:
+ retval = -ENOMEM;
+ goto out;
+}
+
+/*
+ * Must not be called under scst_mutex or there can be a deadlock with
+ * tgt_attr_rwsem
+ */
+void scst_tgt_sysfs_prepare_put(struct scst_tgt *tgt)
+{
+ if (tgt->tgt_kobj_initialized) {
+ down_write(&tgt->tgt_attr_rwsem);
+ tgt->tgt_kobj_put_prepared = 1;
+ }
+
+ return;
+}
+
+/*
+ * Must not be called under scst_mutex or there can be a deadlock with
+ * tgt_attr_rwsem
+ */
+void scst_tgt_sysfs_put(struct scst_tgt *tgt)
+{
+ if (tgt->tgt_kobj_initialized) {
+ kobject_del(tgt->tgt_sess_kobj);
+ kobject_put(tgt->tgt_sess_kobj);
+
+ kobject_del(tgt->tgt_luns_kobj);
+ kobject_put(tgt->tgt_luns_kobj);
+
+ kobject_del(tgt->tgt_ini_grp_kobj);
+ kobject_put(tgt->tgt_ini_grp_kobj);
+
+ kobject_del(&tgt->tgt_kobj);
+
+ if (!tgt->tgt_kobj_put_prepared)
+ down_write(&tgt->tgt_attr_rwsem);
+ kobject_put(&tgt->tgt_kobj);
+ } else
+ scst_free_tgt(tgt);
+ return;
+}
+
+/*
+ * Devices directory implementation
+ */
+
+ssize_t scst_device_sysfs_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ pos = sprintf(buf, "%d - %s\n", dev->type,
+ (unsigned)dev->type > ARRAY_SIZE(scst_dev_handler_types) ?
+ "unknown" : scst_dev_handler_types[dev->type]);
+
+ return pos;
+}
+
+static struct kobj_attribute device_type_attr =
+ __ATTR(type, S_IRUGO, scst_device_sysfs_type_show, NULL);
+
+static ssize_t scst_device_sysfs_threads_num_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ pos = sprintf(buf, "%d\n%s", dev->threads_num,
+ (dev->threads_num != dev->handler->threads_num) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ return pos;
+}
+
+static ssize_t scst_device_sysfs_threads_data_store(struct scst_device *dev,
+ int threads_num, enum scst_dev_type_threads_pool_type threads_pool_type)
+{
+ int res = 0;
+
+ if (dev->threads_num < 0) {
+ PRINT_ERROR("Threads pool disabled for device %s",
+ dev->virt_name);
+ res = -EPERM;
+ goto out;
+ }
+
+ if ((threads_num == dev->threads_num) &&
+ (threads_pool_type == dev->threads_pool_type))
+ goto out;
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ scst_stop_dev_threads(dev);
+
+ dev->threads_num = threads_num;
+ dev->threads_pool_type = threads_pool_type;
+
+ res = scst_create_dev_threads(dev);
+ if (res != 0)
+ goto out_up;
+
+out_up:
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+}
+
+static ssize_t scst_device_sysfs_threads_num_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_device *dev;
+ long newtn;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ res = strict_strtoul(buf, 0, &newtn);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %d ", buf, res);
+ goto out;
+ }
+
+ if (newtn < 0) {
+ PRINT_ERROR("Illegal threads num value %ld", newtn);
+ res = -EINVAL;
+ goto out;
+ }
+
+ res = scst_device_sysfs_threads_data_store(dev, newtn,
+ dev->threads_pool_type);
+ if (res != 0)
+ goto out;
+
+ PRINT_INFO("Changed cmd threads num to %ld", newtn);
+
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute device_threads_num_attr =
+ __ATTR(threads_num, S_IRUGO | S_IWUSR,
+ scst_device_sysfs_threads_num_show,
+ scst_device_sysfs_threads_num_store);
+
+static ssize_t scst_device_sysfs_threads_pool_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos = 0;
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ if (dev->threads_num == 0) {
+ pos = sprintf(buf, "Async\n");
+ goto out;
+ } else if (dev->threads_num < 0) {
+ pos = sprintf(buf, "Not valid\n");
+ goto out;
+ }
+
+ switch (dev->threads_pool_type) {
+ case SCST_THREADS_POOL_PER_INITIATOR:
+ pos = sprintf(buf, "%s\n%s", SCST_THREADS_POOL_PER_INITIATOR_STR,
+ (dev->threads_pool_type != dev->handler->threads_pool_type) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ break;
+ case SCST_THREADS_POOL_SHARED:
+ pos = sprintf(buf, "%s\n%s", SCST_THREADS_POOL_SHARED_STR,
+ (dev->threads_pool_type != dev->handler->threads_pool_type) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ break;
+ default:
+ pos = sprintf(buf, "Unknown\n");
+ break;
+ }
+
+out:
+ return pos;
+}
+
+static ssize_t scst_device_sysfs_threads_pool_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_device *dev;
+ enum scst_dev_type_threads_pool_type newtpt;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ newtpt = scst_parse_threads_pool_type(buf, count);
+ if (newtpt == SCST_THREADS_POOL_TYPE_INVALID) {
+ PRINT_ERROR("Illegal threads pool type %s", buf);
+ res = -EINVAL;
+ goto out;
+ }
+
+ TRACE_DBG("buf %s, count %zd, newtpt %d", buf, count, newtpt);
+
+ res = scst_device_sysfs_threads_data_store(dev, dev->threads_num,
+ newtpt);
+ if (res != 0)
+ goto out;
+
+ PRINT_INFO("Changed cmd threads pool type to %d", newtpt);
+
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute device_threads_pool_type_attr =
+ __ATTR(threads_pool_type, S_IRUGO | S_IWUSR,
+ scst_device_sysfs_threads_pool_type_show,
+ scst_device_sysfs_threads_pool_type_store);
+
+static struct attribute *scst_device_attrs[] = {
+ &device_type_attr.attr,
+ &device_threads_num_attr.attr,
+ &device_threads_pool_type_attr.attr,
+ NULL,
+};
+
+static void scst_sysfs_device_release(struct kobject *kobj)
+{
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ /* Let's make lockdep happy */
+ up_write(&dev->dev_attr_rwsem);
+
+ scst_free_device(dev);
+ return;
+}
+
+int scst_create_devt_dev_sysfs(struct scst_device *dev)
+{
+ int retval = 0;
+ const struct attribute **pattr;
+
+ if (dev->handler == &scst_null_devtype)
+ goto out;
+
+ BUG_ON(!dev->handler->devt_kobj_initialized);
+
+ /*
+ * In case of errors there's no need for additional cleanup, because
+ * it will be done by the _put function() called by the caller.
+ */
+
+ retval = sysfs_create_link(&dev->dev_kobj,
+ &dev->handler->devt_kobj, "handler");
+ if (retval != 0) {
+ PRINT_ERROR("Can't create handler link for dev %s",
+ dev->virt_name);
+ goto out;
+ }
+
+ retval = sysfs_create_link(&dev->handler->devt_kobj,
+ &dev->dev_kobj, dev->virt_name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't create handler link for dev %s",
+ dev->virt_name);
+ goto out;
+ }
+
+ pattr = dev->handler->dev_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ retval = sysfs_create_file(&dev->dev_kobj, *pattr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add dev attr %s for dev %s",
+ (*pattr)->name, dev->virt_name);
+ goto out;
+ }
+ pattr++;
+ }
+ }
+
+out:
+ return retval;
+}
+
+void scst_devt_dev_sysfs_put(struct scst_device *dev)
+{
+ const struct attribute **pattr;
+
+ if (dev->handler == &scst_null_devtype)
+ goto out;
+
+ BUG_ON(!dev->handler->devt_kobj_initialized);
+
+ pattr = dev->handler->dev_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ sysfs_remove_file(&dev->dev_kobj, *pattr);
+ pattr++;
+ }
+ }
+
+ sysfs_remove_link(&dev->dev_kobj, "handler");
+ sysfs_remove_link(&dev->handler->devt_kobj, dev->virt_name);
+
+out:
+ return;
+}
+
+static ssize_t scst_dev_attr_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ int res;
+ struct kobj_attribute *kobj_attr;
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ if (down_read_trylock(&dev->dev_attr_rwsem) == 0) {
+ res = -ENOENT;
+ goto out;
+ }
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ res = kobj_attr->show(kobj, kobj_attr, buf);
+
+ up_read(&dev->dev_attr_rwsem);
+
+out:
+ return res;
+}
+
+static ssize_t scst_dev_attr_store(struct kobject *kobj, struct attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ struct kobj_attribute *kobj_attr;
+ struct scst_device *dev;
+
+ dev = container_of(kobj, struct scst_device, dev_kobj);
+
+ if (down_read_trylock(&dev->dev_attr_rwsem) == 0) {
+ res = -ENOENT;
+ goto out;
+ }
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ res = kobj_attr->store(kobj, kobj_attr, buf, count);
+
+ up_read(&dev->dev_attr_rwsem);
+
+out:
+ return res;
+}
+
+static struct sysfs_ops scst_dev_sysfs_ops = {
+ .show = scst_dev_attr_show,
+ .store = scst_dev_attr_store,
+};
+
+static struct kobj_type scst_device_ktype = {
+ .sysfs_ops = &scst_dev_sysfs_ops,
+ .release = scst_sysfs_device_release,
+ .default_attrs = scst_device_attrs,
+};
+
+int scst_create_device_sysfs(struct scst_device *dev)
+{
+ int retval = 0;
+
+ init_rwsem(&dev->dev_attr_rwsem);
+
+ dev->dev_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&dev->dev_kobj, &scst_device_ktype,
+ scst_devices_kobj, dev->virt_name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add device %s to sysfs", dev->virt_name);
+ goto out;
+ }
+
+ /*
+ * In case of errors there's no need for additional cleanup, because
+ * it will be done by the _put function() called by the caller.
+ */
+
+ dev->dev_exp_kobj = kobject_create_and_add("exported",
+ &dev->dev_kobj);
+ if (dev->dev_exp_kobj == NULL) {
+ PRINT_ERROR("Can't create exported link for device %s",
+ dev->virt_name);
+ retval = -ENOMEM;
+ goto out;
+ }
+
+ if (dev->scsi_dev != NULL) {
+ retval = sysfs_create_link(&dev->dev_kobj,
+ &dev->scsi_dev->sdev_dev.kobj, "scsi_device");
+ if (retval != 0) {
+ PRINT_ERROR("Can't create scsi_device link for dev %s",
+ dev->virt_name);
+ goto out;
+ }
+ }
+
+out:
+ return retval;
+}
+
+/*
+ * Must not be called under scst_mutex or there can be a deadlock with
+ * dev_attr_rwsem
+ */
+void scst_device_sysfs_put(struct scst_device *dev)
+{
+
+ if (dev->dev_kobj_initialized) {
+ kobject_del(dev->dev_exp_kobj);
+ kobject_put(dev->dev_exp_kobj);
+
+ kobject_del(&dev->dev_kobj);
+
+ down_write(&dev->dev_attr_rwsem);
+ kobject_put(&dev->dev_kobj);
+ } else
+ scst_free_device(dev);
+ return;
+}
+
+/*
+ * Target sessions directory implementation
+ */
+
+static ssize_t scst_sess_sysfs_commands_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ return sprintf(buf, "%i\n", atomic_read(&sess->sess_cmd_count));
+}
+
+static struct kobj_attribute session_commands_attr =
+ __ATTR(commands, S_IRUGO, scst_sess_sysfs_commands_show, NULL);
+
+static ssize_t scst_sess_sysfs_active_commands_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int res;
+ struct scst_session *sess;
+ int active_cmds = 0, t;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ for (t = TGT_DEV_HASH_SIZE-1; t >= 0; t--) {
+ struct list_head *sess_tgt_dev_list_head =
+ &sess->sess_tgt_dev_list_hash[t];
+ struct scst_tgt_dev *tgt_dev;
+ list_for_each_entry(tgt_dev, sess_tgt_dev_list_head,
+ sess_tgt_dev_list_entry) {
+ active_cmds += atomic_read(&tgt_dev->tgt_dev_cmd_count);
+ }
+ }
+
+ mutex_unlock(&scst_mutex);
+
+ res = sprintf(buf, "%i\n", active_cmds);
+
+out:
+ return res;
+}
+
+static struct kobj_attribute session_active_commands_attr =
+ __ATTR(active_commands, S_IRUGO, scst_sess_sysfs_active_commands_show,
+ NULL);
+
+static ssize_t scst_sess_sysfs_initiator_name_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ return scnprintf(buf, SCST_SYSFS_BLOCK_SIZE, "%s\n",
+ sess->initiator_name);
+}
+
+static struct kobj_attribute session_initiator_name_attr =
+ __ATTR(initiator_name, S_IRUGO, scst_sess_sysfs_initiator_name_show, NULL);
+
+static struct attribute *scst_session_attrs[] = {
+ &session_commands_attr.attr,
+ &session_active_commands_attr.attr,
+ &session_initiator_name_attr.attr,
+ NULL,
+};
+
+static void scst_sysfs_session_release(struct kobject *kobj)
+{
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ /* Let's make lockdep happy */
+ up_write(&sess->sess_attr_rwsem);
+
+ scst_release_session(sess);
+ return;
+}
+
+static ssize_t scst_sess_attr_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ int res;
+ struct kobj_attribute *kobj_attr;
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ if (down_read_trylock(&sess->sess_attr_rwsem) == 0) {
+ res = -ENOENT;
+ goto out;
+ }
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ res = kobj_attr->show(kobj, kobj_attr, buf);
+
+ up_read(&sess->sess_attr_rwsem);
+
+out:
+ return res;
+}
+
+static ssize_t scst_sess_attr_store(struct kobject *kobj, struct attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ struct kobj_attribute *kobj_attr;
+ struct scst_session *sess;
+
+ sess = container_of(kobj, struct scst_session, sess_kobj);
+
+ if (down_read_trylock(&sess->sess_attr_rwsem) == 0) {
+ res = -ENOENT;
+ goto out;
+ }
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ res = kobj_attr->store(kobj, kobj_attr, buf, count);
+
+ up_read(&sess->sess_attr_rwsem);
+
+out:
+ return res;
+}
+
+static struct sysfs_ops scst_sess_sysfs_ops = {
+ .show = scst_sess_attr_show,
+ .store = scst_sess_attr_store,
+};
+
+static struct kobj_type scst_session_ktype = {
+ .sysfs_ops = &scst_sess_sysfs_ops,
+ .release = scst_sysfs_session_release,
+ .default_attrs = scst_session_attrs,
+};
+
+/* scst_mutex supposed to be locked */
+int scst_create_sess_sysfs(struct scst_session *sess)
+{
+ int retval = 0;
+ struct scst_session *s;
+ const struct attribute **pattr;
+ char *name = (char *)sess->initiator_name;
+ int len = strlen(name) + 1, n = 1;
+
+restart:
+ list_for_each_entry(s, &sess->tgt->sess_list, sess_list_entry) {
+ if (!s->sess_kobj_initialized)
+ continue;
+
+ if (strcmp(name, kobject_name(&s->sess_kobj)) == 0) {
+ if (s == sess)
+ continue;
+
+ TRACE_DBG("Dublicated session from the same initiator "
+ "%s found", name);
+
+ if (name == sess->initiator_name) {
+ len = strlen(sess->initiator_name);
+ len += 20;
+ name = kmalloc(len, GFP_KERNEL);
+ if (name == NULL) {
+ PRINT_ERROR("Unable to allocate a "
+ "replacement name (size %d)",
+ len);
+ }
+ }
+
+ snprintf(name, len, "%s_%d", sess->initiator_name, n);
+ n++;
+ goto restart;
+ }
+ }
+
+ init_rwsem(&sess->sess_attr_rwsem);
+
+ sess->sess_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&sess->sess_kobj, &scst_session_ktype,
+ sess->tgt->tgt_sess_kobj, name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add session %s to sysfs", name);
+ goto out_free;
+ }
+
+ /*
+ * In case of errors there's no need for additional cleanup, because
+ * it will be done by the _put function() called by the caller.
+ */
+
+ pattr = sess->tgt->tgtt->sess_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ retval = sysfs_create_file(&sess->sess_kobj, *pattr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add sess attr %s for sess "
+ "for initiator %s", (*pattr)->name,
+ name);
+ goto out_free;
+ }
+ pattr++;
+ }
+ }
+
+ if (sess->acg == sess->tgt->default_acg)
+ retval = sysfs_create_link(&sess->sess_kobj,
+ sess->tgt->tgt_luns_kobj, "luns");
+ else
+ retval = sysfs_create_link(&sess->sess_kobj,
+ sess->acg->luns_kobj, "luns");
+
+out_free:
+ if (name != sess->initiator_name)
+ kfree(name);
+ return retval;
+}
+
+/*
+ * Must not be called under scst_mutex or there can be a deadlock with
+ * sess_attr_rwsem
+ */
+void scst_sess_sysfs_put(struct scst_session *sess)
+{
+
+ if (sess->sess_kobj_initialized) {
+ kobject_del(&sess->sess_kobj);
+
+ down_write(&sess->sess_attr_rwsem);
+ kobject_put(&sess->sess_kobj);
+ } else
+ scst_release_session(sess);
+ return;
+}
+
+/*
+ * Target luns directory implementation
+ */
+
+static void scst_acg_dev_release(struct kobject *kobj)
+{
+ struct scst_acg_dev *acg_dev;
+
+ acg_dev = container_of(kobj, struct scst_acg_dev, acg_dev_kobj);
+
+ scst_acg_dev_destroy(acg_dev);
+ return;
+}
+
+static ssize_t scst_lun_rd_only_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ struct scst_acg_dev *acg_dev;
+
+ acg_dev = container_of(kobj, struct scst_acg_dev, acg_dev_kobj);
+
+ if (acg_dev->rd_only || acg_dev->dev->rd_only)
+ return sprintf(buf, "%d\n%s\n", 1, SCST_SYSFS_KEY_MARK);
+ else
+ return sprintf(buf, "%d\n", 0);
+}
+
+static struct kobj_attribute lun_options_attr =
+ __ATTR(read_only, S_IRUGO, scst_lun_rd_only_show, NULL);
+
+static struct attribute *lun_attrs[] = {
+ &lun_options_attr.attr,
+ NULL,
+};
+
+static struct kobj_type acg_dev_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_acg_dev_release,
+ .default_attrs = lun_attrs,
+};
+
+int scst_create_acg_dev_sysfs(struct scst_acg *acg, unsigned int virt_lun,
+ struct kobject *parent)
+{
+ int retval;
+ struct scst_acg_dev *acg_dev = NULL, *acg_dev_tmp;
+ char str[20];
+
+ list_for_each_entry(acg_dev_tmp, &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ if (acg_dev_tmp->lun == virt_lun) {
+ acg_dev = acg_dev_tmp;
+ break;
+ }
+ }
+ if (acg_dev == NULL) {
+ PRINT_ERROR("%s", "acg_dev lookup for kobject creation failed");
+ retval = -EINVAL;
+ goto out;
+ }
+
+ snprintf(str, sizeof(str), "export%u",
+ acg_dev->dev->dev_exported_lun_num++);
+
+ kobject_get(&acg_dev->dev->dev_kobj);
+
+ acg_dev->acg_dev_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&acg_dev->acg_dev_kobj, &acg_dev_ktype,
+ parent, "%u", virt_lun);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add acg %s to sysfs", acg->acg_name);
+ goto out;
+ }
+
+ /*
+ * In case of errors there's no need for additional cleanup, because
+ * it will be done by the _put function() called by the caller.
+ */
+
+ retval = sysfs_create_link(acg_dev->dev->dev_exp_kobj,
+ &acg_dev->acg_dev_kobj, str);
+ if (retval != 0) {
+ PRINT_ERROR("Can't create acg %s LUN link", acg->acg_name);
+ goto out;
+ }
+
+ retval = sysfs_create_link(&acg_dev->acg_dev_kobj,
+ &acg_dev->dev->dev_kobj, "device");
+ if (retval != 0) {
+ PRINT_ERROR("Can't create acg %s device link", acg->acg_name);
+ goto out;
+ }
+
+out:
+ return retval;
+}
+
+static ssize_t __scst_luns_mgmt_store(struct scst_acg *acg,
+ struct kobject *kobj, const char *buf, size_t count)
+{
+ int res, virt = 0, read_only = 0, action;
+ char *buffer, *p, *e = NULL;
+ unsigned int host, channel = 0, id = 0, lun = 0, virt_lun;
+ struct scst_acg_dev *acg_dev = NULL, *acg_dev_tmp;
+ struct scst_device *d, *dev = NULL;
+
+#define SCST_LUN_ACTION_ADD 1
+#define SCST_LUN_ACTION_DEL 2
+#define SCST_LUN_ACTION_REPLACE 3
+#define SCST_LUN_ACTION_CLEAR 4
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+ p = buffer;
+
+ p = buffer;
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ if (strncasecmp("add", p, 3) == 0) {
+ p += 3;
+ action = SCST_LUN_ACTION_ADD;
+ } else if (strncasecmp("del", p, 3) == 0) {
+ p += 3;
+ action = SCST_LUN_ACTION_DEL;
+ } else if (!strncasecmp("replace", p, 7)) {
+ p += 7;
+ action = SCST_LUN_ACTION_REPLACE;
+ } else if (!strncasecmp("clear", p, 5)) {
+ p += 5;
+ action = SCST_LUN_ACTION_CLEAR;
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out_free;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_free_resume;
+ }
+
+ if (action != SCST_LUN_ACTION_CLEAR) {
+ if (!isspace(*p)) {
+ PRINT_ERROR("%s", "Syntax error");
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ while (isspace(*p) && *p != '\0')
+ p++;
+ e = p; /* save p */
+ host = simple_strtoul(p, &p, 0);
+ if (*p == ':') {
+ channel = simple_strtoul(p + 1, &p, 0);
+ id = simple_strtoul(p + 1, &p, 0);
+ lun = simple_strtoul(p + 1, &p, 0);
+ e = p;
+ } else {
+ virt++;
+ p = e; /* restore p */
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+ }
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if (virt) {
+ if (d->virt_id && !strcmp(d->virt_name, p)) {
+ dev = d;
+ TRACE_DBG("Virt device %p (%s) found",
+ dev, p);
+ break;
+ }
+ } else {
+ if (d->scsi_dev &&
+ d->scsi_dev->host->host_no == host &&
+ d->scsi_dev->channel == channel &&
+ d->scsi_dev->id == id &&
+ d->scsi_dev->lun == lun) {
+ dev = d;
+ TRACE_DBG("Dev %p (%d:%d:%d:%d) found",
+ dev, host, channel, id, lun);
+ break;
+ }
+ }
+ }
+ if (dev == NULL) {
+ if (virt) {
+ PRINT_ERROR("Virt device '%s' not found", p);
+ } else {
+ PRINT_ERROR("Device %d:%d:%d:%d not found",
+ host, channel, id, lun);
+ }
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ }
+
+ switch (action) {
+ case SCST_LUN_ACTION_ADD:
+ case SCST_LUN_ACTION_REPLACE:
+ {
+ bool dev_replaced = false;
+
+ e++;
+ while (isspace(*e) && *e != '\0')
+ e++;
+ virt_lun = simple_strtoul(e, &e, 0);
+
+ while (isspace(*e) && *e != '\0')
+ e++;
+
+ while (1) {
+ char *pp;
+ unsigned long val;
+ char *param = scst_get_next_token_str(&e);
+ if (param == NULL)
+ break;
+
+ p = scst_get_next_lexem(¶m);
+ if (*p == '\0') {
+ PRINT_ERROR("Syntax error at %s (device %s)",
+ param, dev->virt_name);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ pp = scst_get_next_lexem(¶m);
+ if (*pp == '\0') {
+ PRINT_ERROR("Parameter %s value missed for device %s",
+ p, dev->virt_name);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ if (scst_get_next_lexem(¶m)[0] != '\0') {
+ PRINT_ERROR("Too many parameter's %s values (device %s)",
+ p, dev->virt_name);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ res = strict_strtoul(pp, 0, &val);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %d "
+ "(device %s)", pp, res, dev->virt_name);
+ goto out_free_up;
+ }
+
+ if (!strcasecmp("read_only", p)) {
+ read_only = val;
+ TRACE_DBG("READ ONLY %d", read_only);
+ } else {
+ PRINT_ERROR("Unknown parameter %s (device %s)",
+ p, dev->virt_name);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ }
+
+ acg_dev = NULL;
+ list_for_each_entry(acg_dev_tmp, &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ if (acg_dev_tmp->lun == virt_lun) {
+ acg_dev = acg_dev_tmp;
+ break;
+ }
+ }
+
+ if (acg_dev != NULL) {
+ if (action == SCST_LUN_ACTION_ADD) {
+ PRINT_ERROR("virt lun %d already exists in "
+ "group %s", virt_lun, acg->acg_name);
+ res = -EEXIST;
+ goto out_free_up;
+ } else {
+ /* Replace */
+ res = scst_acg_remove_dev(acg, acg_dev->dev,
+ false);
+ if (res != 0)
+ goto out_free_up;
+
+ dev_replaced = true;
+ }
+ }
+
+ res = scst_acg_add_dev(acg, dev, virt_lun, read_only,
+ !dev_replaced);
+ if (res != 0)
+ goto out_free_up;
+
+ res = scst_create_acg_dev_sysfs(acg, virt_lun, kobj);
+ if (res != 0) {
+ PRINT_ERROR("%s", "Creation of acg_dev kobject failed");
+ goto out_remove_acg_dev;
+ }
+
+ if (dev_replaced) {
+ struct scst_tgt_dev *tgt_dev;
+
+ list_for_each_entry(tgt_dev, &dev->dev_tgt_dev_list,
+ dev_tgt_dev_list_entry) {
+ if ((tgt_dev->acg_dev->acg == acg) &&
+ (tgt_dev->lun == virt_lun)) {
+ TRACE_MGMT_DBG("INQUIRY DATA HAS CHANGED"
+ " on tgt_dev %p", tgt_dev);
+ scst_gen_aen_or_ua(tgt_dev,
+ SCST_LOAD_SENSE(scst_sense_inquery_data_changed));
+ }
+ }
+ }
+
+ break;
+ }
+ case SCST_LUN_ACTION_DEL:
+ res = scst_acg_remove_dev(acg, dev, true);
+ if (res != 0)
+ goto out_free_up;
+ break;
+ case SCST_LUN_ACTION_CLEAR:
+ PRINT_INFO("Removed all devices from group %s",
+ acg->acg_name);
+ list_for_each_entry_safe(acg_dev, acg_dev_tmp,
+ &acg->acg_dev_list,
+ acg_dev_list_entry) {
+ res = scst_acg_remove_dev(acg, acg_dev->dev,
+ list_is_last(&acg_dev->acg_dev_list_entry,
+ &acg->acg_dev_list));
+ if (res)
+ goto out_free_up;
+ }
+ break;
+ }
+
+ res = count;
+
+out_free_up:
+ mutex_unlock(&scst_mutex);
+
+out_free_resume:
+ scst_resume_activity();
+
+out_free:
+ kfree(buffer);
+
+out:
+ return res;
+
+out_remove_acg_dev:
+ scst_acg_remove_dev(acg, dev, true);
+ goto out_free_up;
+
+#undef SCST_LUN_ACTION_ADD
+#undef SCST_LUN_ACTION_DEL
+#undef SCST_LUN_ACTION_REPLACE
+#undef SCST_LUN_ACTION_CLEAR
+}
+
+static ssize_t scst_luns_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ static char *help = "Usage: echo \"add|del H:C:I:L lun [parameters]\" "
+ ">mgmt\n"
+ " echo \"add|del VNAME lun [parameters]\" "
+ ">mgmt\n"
+ " echo \"replace H:C:I:L lun [parameters]\" "
+ ">mgmt\n"
+ " echo \"replace VNAME lun [parameters]\" "
+ ">mgmt\n"
+ " echo \"clear\" >mgmt\n"
+ "\n"
+ "where parameters are one or more "
+ "param_name=value pairs separated by ';'\n"
+ "\nThe following parameters available: read_only.";
+
+ return sprintf(buf, help);
+}
+
+static ssize_t scst_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj->parent, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ res = __scst_luns_mgmt_store(acg, kobj, buf, count);
+ return res;
+}
+
+static ssize_t __scst_acg_addr_method_show(struct scst_acg *acg, char *buf)
+{
+ int res;
+
+ switch (acg->addr_method) {
+ case SCST_LUN_ADDR_METHOD_FLAT:
+ res = sprintf(buf, "FLAT\n%s\n", SCST_SYSFS_KEY_MARK);
+ break;
+ case SCST_LUN_ADDR_METHOD_PERIPHERAL:
+ res = sprintf(buf, "PERIPHERAL\n");
+ break;
+ default:
+ res = sprintf(buf, "UNKNOWN\n");
+ break;
+ }
+
+ return res;
+}
+
+static ssize_t __scst_acg_addr_method_store(struct scst_acg *acg,
+ const char *buf, size_t count)
+{
+ int res = count;
+
+ if (strncasecmp(buf, "FLAT", min_t(int, 4, count)) == 0)
+ acg->addr_method = SCST_LUN_ADDR_METHOD_FLAT;
+ else if (strncasecmp(buf, "PERIPHERAL", min_t(int, 10, count)) == 0)
+ acg->addr_method = SCST_LUN_ADDR_METHOD_PERIPHERAL;
+ else {
+ PRINT_ERROR("Unknown address method %s", buf);
+ res = -EINVAL;
+ }
+ return res;
+}
+
+static ssize_t scst_tgt_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ return __scst_acg_addr_method_show(acg, buf);
+}
+
+static ssize_t scst_tgt_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ res = __scst_acg_addr_method_store(acg, buf, count);
+ return res;
+}
+
+static ssize_t __scst_acg_io_grouping_type_show(struct scst_acg *acg, char *buf)
+{
+ int res;
+
+ switch (acg->acg_io_grouping_type) {
+ case SCST_IO_GROUPING_AUTO:
+ res = sprintf(buf, "%s\n", SCST_IO_GROUPING_AUTO_STR);
+ break;
+ case SCST_IO_GROUPING_THIS_GROUP_ONLY:
+ res = sprintf(buf, "%s\n%s\n",
+ SCST_IO_GROUPING_THIS_GROUP_ONLY_STR,
+ SCST_SYSFS_KEY_MARK);
+ break;
+ case SCST_IO_GROUPING_NEVER:
+ res = sprintf(buf, "%s\n%s\n", SCST_IO_GROUPING_NEVER_STR,
+ SCST_SYSFS_KEY_MARK);
+ break;
+ default:
+ res = sprintf(buf, "%d\n%s\n", acg->acg_io_grouping_type,
+ SCST_SYSFS_KEY_MARK);
+ break;
+ }
+
+ return res;
+}
+
+static ssize_t __scst_acg_io_grouping_type_store(struct scst_acg *acg,
+ const char *buf, size_t count)
+{
+ int res = 0;
+ int prev = acg->acg_io_grouping_type;
+ struct scst_acg_dev *acg_dev;
+
+ if (strncasecmp(buf, SCST_IO_GROUPING_AUTO_STR,
+ min_t(int, strlen(SCST_IO_GROUPING_AUTO_STR), count)) == 0)
+ acg->acg_io_grouping_type = SCST_IO_GROUPING_AUTO;
+ else if (strncasecmp(buf, SCST_IO_GROUPING_THIS_GROUP_ONLY_STR,
+ min_t(int, strlen(SCST_IO_GROUPING_THIS_GROUP_ONLY_STR), count)) == 0)
+ acg->acg_io_grouping_type = SCST_IO_GROUPING_THIS_GROUP_ONLY;
+ else if (strncasecmp(buf, SCST_IO_GROUPING_NEVER_STR,
+ min_t(int, strlen(SCST_IO_GROUPING_NEVER_STR), count)) == 0)
+ acg->acg_io_grouping_type = SCST_IO_GROUPING_NEVER;
+ else {
+ long io_grouping_type;
+ res = strict_strtoul(buf, 0, &io_grouping_type);
+ if ((res != 0) || (io_grouping_type <= 0)) {
+ PRINT_ERROR("Unknown or not allowed I/O grouping type "
+ "%s", buf);
+ res = -EINVAL;
+ goto out;
+ }
+ acg->acg_io_grouping_type = io_grouping_type;
+ }
+
+ if (prev == acg->acg_io_grouping_type)
+ goto out;
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_resume;
+ }
+
+ list_for_each_entry(acg_dev, &acg->acg_dev_list, acg_dev_list_entry) {
+ int rc;
+
+ scst_stop_dev_threads(acg_dev->dev);
+
+ rc = scst_create_dev_threads(acg_dev->dev);
+ if (rc != 0)
+ res = rc;
+ }
+
+ mutex_unlock(&scst_mutex);
+
+out_resume:
+ scst_resume_activity();
+
+out:
+ return res;
+}
+
+static ssize_t scst_tgt_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ return __scst_acg_io_grouping_type_show(acg, buf);
+}
+
+static ssize_t scst_tgt_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+ struct scst_tgt *tgt;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ acg = tgt->default_acg;
+
+ res = __scst_acg_io_grouping_type_store(acg, buf, count);
+ if (res != 0)
+ goto out;
+
+ res = count;
+
+out:
+ return res;
+}
+
+static int scst_create_acg_sysfs(struct scst_tgt *tgt,
+ struct scst_acg *acg)
+{
+ int retval = 0;
+
+ acg->acg_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&acg->acg_kobj, &acg_ktype,
+ tgt->tgt_ini_grp_kobj, acg->acg_name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add acg '%s' to sysfs", acg->acg_name);
+ goto out;
+ }
+
+ acg->luns_kobj = kobject_create_and_add("luns", &acg->acg_kobj);
+ if (acg->luns_kobj == NULL) {
+ PRINT_ERROR("Can't create luns kobj for tgt %s",
+ tgt->tgt_name);
+ retval = -ENOMEM;
+ goto out;
+ }
+
+ retval = sysfs_create_file(acg->luns_kobj, &scst_acg_luns_mgmt.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_luns_mgmt.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ acg->initiators_kobj = kobject_create_and_add("initiators",
+ &acg->acg_kobj);
+ if (acg->initiators_kobj == NULL) {
+ PRINT_ERROR("Can't create initiators kobj for tgt %s",
+ tgt->tgt_name);
+ retval = -ENOMEM;
+ goto out;
+ }
+
+ retval = sysfs_create_file(acg->initiators_kobj,
+ &scst_acg_ini_mgmt.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_ini_mgmt.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ retval = sysfs_create_file(&acg->acg_kobj, &scst_acg_addr_method.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_addr_method.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+ retval = sysfs_create_file(&acg->acg_kobj, &scst_acg_io_grouping_type.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add tgt attr %s for tgt %s",
+ scst_acg_io_grouping_type.attr.name, tgt->tgt_name);
+ goto out;
+ }
+
+out:
+ return retval;
+}
+
+void scst_acg_sysfs_put(struct scst_acg *acg)
+{
+
+ if (acg->acg_kobj_initialized) {
+ scst_clear_acg(acg);
+
+ kobject_del(acg->luns_kobj);
+ kobject_put(acg->luns_kobj);
+
+ kobject_del(acg->initiators_kobj);
+ kobject_put(acg->initiators_kobj);
+
+ kobject_del(&acg->acg_kobj);
+ kobject_put(&acg->acg_kobj);
+ } else
+ scst_destroy_acg(acg);
+ return;
+}
+
+static ssize_t scst_acg_addr_method_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ return __scst_acg_addr_method_show(acg, buf);
+}
+
+static ssize_t scst_acg_addr_method_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ res = __scst_acg_addr_method_store(acg, buf, count);
+ return res;
+}
+
+static ssize_t scst_acg_io_grouping_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ return __scst_acg_io_grouping_type_show(acg, buf);
+}
+
+static ssize_t scst_acg_io_grouping_type_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+
+ acg = container_of(kobj, struct scst_acg, acg_kobj);
+
+ res = __scst_acg_io_grouping_type_store(acg, buf, count);
+ if (res != 0)
+ goto out;
+
+ res = count;
+
+out:
+ return res;
+}
+
+static ssize_t scst_ini_group_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ static char *help = "Usage: echo \"create GROUP_NAME\" >mgmt\n"
+ " echo \"del GROUP_NAME\" >mgmt\n";
+
+ return sprintf(buf, help);
+}
+
+static ssize_t scst_ini_group_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res, action;
+ int len;
+ char *name;
+ char *buffer, *p, *e = NULL;
+ struct scst_acg *a, *acg = NULL;
+ struct scst_tgt *tgt;
+
+#define SCST_INI_GROUP_ACTION_CREATE 1
+#define SCST_INI_GROUP_ACTION_DEL 2
+
+ tgt = container_of(kobj->parent, struct scst_tgt, tgt_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+ p = buffer;
+
+ p = buffer;
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ if (strncasecmp("create ", p, 7) == 0) {
+ p += 7;
+ action = SCST_INI_GROUP_ACTION_CREATE;
+ } else if (strncasecmp("del ", p, 4) == 0) {
+ p += 4;
+ action = SCST_INI_GROUP_ACTION_DEL;
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out_free;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_free_resume;
+ }
+
+ while (isspace(*p) && *p != '\0')
+ p++;
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+
+ if (p[0] == '\0') {
+ PRINT_ERROR("%s", "Group name required");
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ list_for_each_entry(a, &tgt->tgt_acg_list, acg_list_entry) {
+ if (strcmp(a->acg_name, p) == 0) {
+ TRACE_DBG("group (acg) %p %s found",
+ a, a->acg_name);
+ acg = a;
+ break;
+ }
+ }
+
+ switch (action) {
+ case SCST_INI_GROUP_ACTION_CREATE:
+ TRACE_DBG("Creating group '%s'", p);
+ if (acg != NULL) {
+ PRINT_ERROR("acg name %s exist", p);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ len = strlen(p) + 1;
+ name = kmalloc(len, GFP_KERNEL);
+ if (name == NULL) {
+ PRINT_ERROR("%s", "Allocation of name failed");
+ res = -ENOMEM;
+ goto out_free_up;
+ }
+ strlcpy(name, p, len);
+
+ acg = scst_alloc_add_acg(tgt, name);
+ kfree(name);
+ if (acg == NULL)
+ goto out_free_up;
+
+ res = scst_create_acg_sysfs(tgt, acg);
+ if (res != 0)
+ goto out_free_acg;
+ break;
+ case SCST_INI_GROUP_ACTION_DEL:
+ TRACE_DBG("Deleting group '%s'", p);
+ if (acg == NULL) {
+ PRINT_ERROR("Group %s not found", p);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ if (!scst_acg_sess_is_empty(acg)) {
+ PRINT_ERROR("Group %s is not empty", acg->acg_name);
+ res = -EBUSY;
+ goto out_free_up;
+ }
+ scst_acg_sysfs_put(acg);
+ break;
+ }
+
+ res = count;
+
+out_free_up:
+ mutex_unlock(&scst_mutex);
+
+out_free_resume:
+ scst_resume_activity();
+
+out_free:
+ kfree(buffer);
+
+out:
+ return res;
+
+out_free_acg:
+ scst_acg_sysfs_put(acg);
+ goto out_free_up;
+
+#undef SCST_LUN_ACTION_CREATE
+#undef SCST_LUN_ACTION_DEL
+}
+
+static ssize_t scst_rel_tgt_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_tgt *tgt;
+ int res;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ res = sprintf(buf, "%d\n%s", tgt->rel_tgt_id,
+ (tgt->rel_tgt_id != 0) ? SCST_SYSFS_KEY_MARK "\n" : "");
+ return res;
+}
+
+static ssize_t scst_rel_tgt_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res = 0;
+ struct scst_tgt *tgt;
+ unsigned long rel_tgt_id;
+
+ if (buf == NULL)
+ goto out_err;
+
+ tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+
+ res = strict_strtoul(buf, 0, &rel_tgt_id);
+ if (res != 0)
+ goto out_err;
+
+ TRACE_DBG("Try to set relative target port id %d",
+ (uint16_t)rel_tgt_id);
+
+ if (rel_tgt_id < SCST_MIN_REL_TGT_ID ||
+ rel_tgt_id > SCST_MAX_REL_TGT_ID) {
+ if ((rel_tgt_id == 0) && !tgt->tgtt->is_target_enabled(tgt))
+ goto set;
+
+ PRINT_ERROR("Invalid relative port id %d",
+ (uint16_t)rel_tgt_id);
+ res = -EINVAL;
+ goto out;
+ }
+
+ if (tgt->tgtt->is_target_enabled(tgt) &&
+ rel_tgt_id != tgt->rel_tgt_id) {
+ if (!scst_is_relative_target_port_id_unique(rel_tgt_id, tgt)) {
+ PRINT_ERROR("Relative port id %d is not unique",
+ (uint16_t)rel_tgt_id);
+ res = -EBADSLT;
+ goto out;
+ }
+ }
+
+set:
+ tgt->rel_tgt_id = (uint16_t)rel_tgt_id;
+
+ res = count;
+
+out:
+ return res;
+
+out_err:
+ PRINT_ERROR("%s: Requested action not understood: %s", __func__, buf);
+ res = -EINVAL;
+ goto out;
+}
+
+int scst_create_acn_sysfs(struct scst_acg *acg, struct scst_acn *acn)
+{
+ int retval = 0;
+ int len;
+ struct kobj_attribute *attr = NULL;
+
+ acn->acn_attr = NULL;
+
+ attr = kzalloc(sizeof(struct kobj_attribute), GFP_KERNEL);
+ if (attr == NULL) {
+ PRINT_ERROR("Unable to allocate attributes for initiator '%s'",
+ acn->name);
+ retval = -ENOMEM;
+ goto out;
+ }
+
+ len = strlen(acn->name) + 1;
+ attr->attr.name = kzalloc(len, GFP_KERNEL);
+ if (attr->attr.name == NULL) {
+ PRINT_ERROR("Unable to allocate attributes for initiator '%s'",
+ acn->name);
+ retval = -ENOMEM;
+ goto out_free;
+ }
+ strlcpy((char *)attr->attr.name, acn->name, len);
+
+ attr->attr.owner = THIS_MODULE;
+ attr->attr.mode = S_IRUGO;
+ attr->show = scst_acn_file_show;
+ attr->store = NULL;
+
+ retval = sysfs_create_file(acg->initiators_kobj, &attr->attr);
+ if (retval != 0) {
+ PRINT_ERROR("Unable to create acn '%s' for group '%s'",
+ acn->name, acg->acg_name);
+ kfree(attr->attr.name);
+ goto out_free;
+ }
+
+ acn->acn_attr = attr;
+
+out:
+ return retval;
+
+out_free:
+ kfree(attr);
+ goto out;
+}
+
+void scst_acn_sysfs_del(struct scst_acg *acg, struct scst_acn *acn,
+ bool reassign)
+{
+
+ if (acn->acn_attr != NULL) {
+ sysfs_remove_file(acg->initiators_kobj,
+ &acn->acn_attr->attr);
+ kfree(acn->acn_attr->attr.name);
+ kfree(acn->acn_attr);
+ }
+ scst_acg_remove_acn(acn);
+ if (reassign)
+ scst_check_reassign_sessions();
+ return;
+}
+
+static ssize_t scst_acn_file_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return scnprintf(buf, SCST_SYSFS_BLOCK_SIZE, "%s\n",
+ attr->attr.name);
+}
+
+static ssize_t scst_acg_luns_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ struct scst_acg *acg;
+
+ acg = container_of(kobj->parent, struct scst_acg, acg_kobj);
+ res = __scst_luns_mgmt_store(acg, kobj, buf, count);
+ return res;
+}
+
+static ssize_t scst_acg_ini_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ static char *help = "Usage: echo \"add INITIATOR_NAME\" "
+ ">mgmt\n"
+ " echo \"del INITIATOR_NAME\" "
+ ">mgmt\n"
+ " echo \"move INITIATOR_NAME DEST_GROUP_NAME\" "
+ ">mgmt\n"
+ " echo \"clear\" "
+ ">mgmt\n";
+
+ return sprintf(buf, help);
+}
+
+static ssize_t scst_acg_ini_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res, action;
+ char *buffer, *p, *e = NULL;
+ char *name = NULL, *group = NULL;
+ struct scst_acg *acg = NULL, *acg_dest = NULL;
+ struct scst_tgt *tgt = NULL;
+ struct scst_acn *acn = NULL, *acn_tmp;
+
+#define SCST_ACG_ACTION_INI_ADD 1
+#define SCST_ACG_ACTION_INI_DEL 2
+#define SCST_ACG_ACTION_INI_CLEAR 3
+#define SCST_ACG_ACTION_INI_MOVE 4
+
+ acg = container_of(kobj->parent, struct scst_acg, acg_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+ p = buffer;
+
+ p = buffer;
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+
+ if (strncasecmp("add", p, 3) == 0) {
+ p += 3;
+ action = SCST_ACG_ACTION_INI_ADD;
+ } else if (strncasecmp("del", p, 3) == 0) {
+ p += 3;
+ action = SCST_ACG_ACTION_INI_DEL;
+ } else if (strncasecmp("clear", p, 5) == 0) {
+ p += 5;
+ action = SCST_ACG_ACTION_INI_CLEAR;
+ } else if (strncasecmp("move", p, 4) == 0) {
+ p += 4;
+ action = SCST_ACG_ACTION_INI_MOVE;
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ if (action != SCST_ACG_ACTION_INI_CLEAR)
+ if (!isspace(*p)) {
+ PRINT_ERROR("%s", "Syntax error");
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ res = scst_suspend_activity(true);
+ if (res != 0)
+ goto out_free;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_free_resume;
+ }
+
+ if (action != SCST_ACG_ACTION_INI_CLEAR)
+ while (isspace(*p) && *p != '\0')
+ p++;
+
+ switch (action) {
+ case SCST_ACG_ACTION_INI_ADD:
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+ name = p;
+
+ if (name[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid initiator name");
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ res = scst_acg_add_name(acg, name);
+ if (res != 0)
+ goto out_free_up;
+ break;
+ case SCST_ACG_ACTION_INI_DEL:
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+ name = p;
+
+ if (name[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid initiator name");
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ acn = scst_acg_find_name(acg, name);
+ if (acn == NULL) {
+ PRINT_ERROR("Unable to find "
+ "initiator '%s' in group '%s'",
+ name, acg->acg_name);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ scst_acn_sysfs_del(acg, acn, true);
+ break;
+ case SCST_ACG_ACTION_INI_CLEAR:
+ list_for_each_entry_safe(acn, acn_tmp, &acg->acn_list,
+ acn_list_entry) {
+ scst_acn_sysfs_del(acg, acn, false);
+ }
+ scst_check_reassign_sessions();
+ break;
+ case SCST_ACG_ACTION_INI_MOVE:
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ if (*e == '\0') {
+ PRINT_ERROR("%s", "Too few parameters");
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ *e = '\0';
+ name = p;
+
+ if (name[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid initiator name");
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ e++;
+ p = e;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = '\0';
+ group = p;
+
+ if (group[0] == '\0') {
+ PRINT_ERROR("%s", "Invalid group name");
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ TRACE_DBG("Move initiator '%s' to group '%s'",
+ name, group);
+
+ /*
+ * Better get tgt from hierarchy tgt_kobj -> tgt_ini_grp_kobj ->
+ * acg_kobj -> initiators_kobj than have direct pointer to tgt
+ * in struct acg and have a headache to care about its possible
+ * wrong dereference on the destruction time.
+ */
+ {
+ struct kobject *k;
+
+ /* acg_kobj */
+ k = kobj->parent;
+ if (k == NULL) {
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ /* tgt_ini_grp_kobj */
+ k = k->parent;
+ if (k == NULL) {
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ /* tgt_kobj */
+ k = k->parent;
+ if (k == NULL) {
+ res = -EINVAL;
+ goto out_free_up;
+ }
+
+ tgt = container_of(k, struct scst_tgt, tgt_kobj);
+ }
+
+ acn = scst_acg_find_name(acg, name);
+ if (acn == NULL) {
+ PRINT_ERROR("Unable to find "
+ "initiator '%s' in group '%s'",
+ name, acg->acg_name);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ acg_dest = scst_tgt_find_acg(tgt, group);
+ if (acg_dest == NULL) {
+ PRINT_ERROR("Unable to find group '%s' in target '%s'",
+ group, tgt->tgt_name);
+ res = -EINVAL;
+ goto out_free_up;
+ }
+ if (scst_acg_find_name(acg_dest, name) != NULL) {
+ PRINT_ERROR("Initiator '%s' already exists in group '%s'",
+ name, acg_dest->acg_name);
+ res = -EEXIST;
+ goto out_free_up;
+ }
+ scst_acn_sysfs_del(acg, acn, false);
+
+ res = scst_acg_add_name(acg_dest, name);
+ if (res != 0)
+ goto out_free_up;
+ break;
+ }
+
+ res = count;
+
+out_free_up:
+ mutex_unlock(&scst_mutex);
+
+out_free_resume:
+ scst_resume_activity();
+
+out_free:
+ kfree(buffer);
+
+out:
+ return res;
+
+#undef SCST_ACG_ACTION_INI_ADD
+#undef SCST_ACG_ACTION_INI_DEL
+#undef SCST_ACG_ACTION_INI_CLEAR
+#undef SCST_ACG_ACTION_INI_MOVE
+}
+
+/*
+ * SGV directory implementation
+ */
+
+static struct kobj_attribute sgv_stat_attr =
+ __ATTR(stats, S_IRUGO | S_IWUSR, sgv_sysfs_stat_show,
+ sgv_sysfs_stat_reset);
+
+static struct attribute *sgv_attrs[] = {
+ &sgv_stat_attr.attr,
+ NULL,
+};
+
+static void sgv_kobj_release(struct kobject *kobj)
+{
+ struct sgv_pool *pool;
+
+ pool = container_of(kobj, struct sgv_pool, sgv_kobj);
+
+ sgv_pool_destroy(pool);
+ return;
+}
+
+static struct kobj_type sgv_pool_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = sgv_kobj_release,
+ .default_attrs = sgv_attrs,
+};
+
+int scst_create_sgv_sysfs(struct sgv_pool *pool)
+{
+ int retval;
+
+ pool->sgv_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&pool->sgv_kobj, &sgv_pool_ktype,
+ scst_sgv_kobj, pool->name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add sgv pool %s to sysfs", pool->name);
+ goto out;
+ }
+
+out:
+ return retval;
+}
+
+/* pool can be dead upon exit from this function! */
+void scst_sgv_sysfs_put(struct sgv_pool *pool)
+{
+ if (pool->sgv_kobj_initialized) {
+ kobject_del(&pool->sgv_kobj);
+ kobject_put(&pool->sgv_kobj);
+ } else
+ sgv_pool_destroy(pool);
+ return;
+}
+
+static struct kobj_attribute sgv_global_stat_attr =
+ __ATTR(global_stats, S_IRUGO | S_IWUSR, sgv_sysfs_global_stat_show,
+ sgv_sysfs_global_stat_reset);
+
+static struct attribute *sgv_default_attrs[] = {
+ &sgv_global_stat_attr.attr,
+ NULL,
+};
+
+static struct kobj_type sgv_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_sysfs_release,
+ .default_attrs = sgv_default_attrs,
+};
+
+/*
+ * SCST sysfs root directory implementation
+ */
+
+static ssize_t scst_threads_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int count;
+
+ count = sprintf(buf, "%d\n%s", scst_main_cmd_threads.nr_threads,
+ (scst_main_cmd_threads.nr_threads != scst_threads) ?
+ SCST_SYSFS_KEY_MARK "\n" : "");
+ return count;
+}
+
+static ssize_t scst_threads_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ long oldtn, newtn, delta;
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ oldtn = scst_main_cmd_threads.nr_threads;
+
+ res = strict_strtoul(buf, 0, &newtn);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %d ", buf, res);
+ goto out_up;
+ }
+
+ if (newtn <= 0) {
+ PRINT_ERROR("Illegal threads num value %ld", newtn);
+ res = -EINVAL;
+ goto out_up;
+ }
+
+ delta = newtn - oldtn;
+ if (delta < 0)
+ scst_del_threads(&scst_main_cmd_threads, -delta);
+ else {
+ res = scst_add_threads(&scst_main_cmd_threads, NULL, NULL, delta);
+ if (res != 0)
+ goto out_up;
+ }
+
+ PRINT_INFO("Changed cmd threads num: old %ld, new %ld", oldtn, newtn);
+
+ res = count;
+
+out_up:
+ mutex_unlock(&scst_mutex);
+
+out:
+ return res;
+}
+
+static ssize_t scst_setup_id_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int count;
+
+ count = sprintf(buf, "0x%x%s\n", scst_setup_id,
+ (scst_setup_id == 0) ? "" : SCST_SYSFS_KEY_MARK "\n");
+ return count;
+}
+
+static ssize_t scst_setup_id_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ unsigned long val;
+
+ res = strict_strtoul(buf, 0, &val);
+ if (res != 0) {
+ PRINT_ERROR("strict_strtoul() for %s failed: %d ", buf, res);
+ goto out;
+ }
+
+ scst_setup_id = val;
+ PRINT_INFO("Changed scst_setup_id to %x", scst_setup_id);
+
+ res = count;
+
+out:
+ return res;
+}
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static void scst_read_trace_tlb(const struct scst_trace_log *tbl, char *buf,
+ unsigned long log_level, int *pos)
+{
+ const struct scst_trace_log *t = tbl;
+
+ if (t == NULL)
+ goto out;
+
+ while (t->token) {
+ if (log_level & t->val) {
+ *pos += sprintf(&buf[*pos], "%s%s",
+ (*pos == 0) ? "" : " | ",
+ t->token);
+ }
+ t++;
+ }
+out:
+ return;
+}
+
+static ssize_t scst_trace_level_show(const struct scst_trace_log *local_tbl,
+ unsigned long log_level, char *buf, const char *help)
+{
+ int pos = 0;
+
+ scst_read_trace_tlb(scst_trace_tbl, buf, log_level, &pos);
+ scst_read_trace_tlb(local_tbl, buf, log_level, &pos);
+
+ pos += sprintf(&buf[pos], "\n\n\nUsage:\n"
+ " echo \"all|none|default\" >trace_level\n"
+ " echo \"value DEC|0xHEX|0OCT\" >trace_level\n"
+ " echo \"add|del TOKEN\" >trace_level\n"
+ "\nwhere TOKEN is one of [debug, function, line, pid,\n"
+ " buff, mem, sg, out_of_mem,\n"
+ " special, scsi, mgmt, minor,\n"
+ " mgmt_dbg, scsi_serializing,\n"
+ " retry, recv_bot, send_bot, recv_top,\n"
+ " send_top%s]", help != NULL ? help : "");
+
+ return pos;
+}
+
+static ssize_t scst_main_trace_level_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return scst_trace_level_show(scst_local_trace_tbl, trace_flag,
+ buf, NULL);
+}
+
+static int scst_write_trace(const char *buf, size_t length,
+ unsigned long *log_level, unsigned long default_level,
+ const char *name, const struct scst_trace_log *tbl)
+{
+ int res = length;
+ int action;
+ unsigned long level = 0, oldlevel;
+ char *buffer, *p, *e;
+ const struct scst_trace_log *t;
+
+#define SCST_TRACE_ACTION_ALL 1
+#define SCST_TRACE_ACTION_NONE 2
+#define SCST_TRACE_ACTION_DEFAULT 3
+#define SCST_TRACE_ACTION_ADD 4
+#define SCST_TRACE_ACTION_DEL 5
+#define SCST_TRACE_ACTION_VALUE 6
+
+ if ((buf == NULL) || (length == 0)) {
+ res = -EINVAL;
+ goto out;
+ }
+
+ buffer = kmalloc(length+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ PRINT_ERROR("Unable to alloc intermediate buffer (size %zd)",
+ length+1);
+ res = -ENOMEM;
+ goto out;
+ }
+ memcpy(buffer, buf, length);
+ buffer[length] = '\0';
+
+ p = buffer;
+ if (!strncasecmp("all", p, 3)) {
+ action = SCST_TRACE_ACTION_ALL;
+ } else if (!strncasecmp("none", p, 4) || !strncasecmp("null", p, 4)) {
+ action = SCST_TRACE_ACTION_NONE;
+ } else if (!strncasecmp("default", p, 7)) {
+ action = SCST_TRACE_ACTION_DEFAULT;
+ } else if (!strncasecmp("add", p, 3)) {
+ p += 3;
+ action = SCST_TRACE_ACTION_ADD;
+ } else if (!strncasecmp("del", p, 3)) {
+ p += 3;
+ action = SCST_TRACE_ACTION_DEL;
+ } else if (!strncasecmp("value", p, 5)) {
+ p += 5;
+ action = SCST_TRACE_ACTION_VALUE;
+ } else {
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ switch (action) {
+ case SCST_TRACE_ACTION_ADD:
+ case SCST_TRACE_ACTION_DEL:
+ case SCST_TRACE_ACTION_VALUE:
+ if (!isspace(*p)) {
+ PRINT_ERROR("%s", "Syntax error");
+ res = -EINVAL;
+ goto out_free;
+ }
+ }
+
+ switch (action) {
+ case SCST_TRACE_ACTION_ALL:
+ level = TRACE_ALL;
+ break;
+ case SCST_TRACE_ACTION_DEFAULT:
+ level = default_level;
+ break;
+ case SCST_TRACE_ACTION_NONE:
+ level = TRACE_NULL;
+ break;
+ case SCST_TRACE_ACTION_ADD:
+ case SCST_TRACE_ACTION_DEL:
+ while (isspace(*p) && *p != '\0')
+ p++;
+ e = p;
+ while (!isspace(*e) && *e != '\0')
+ e++;
+ *e = 0;
+ if (tbl) {
+ t = tbl;
+ while (t->token) {
+ if (!strcasecmp(p, t->token)) {
+ level = t->val;
+ break;
+ }
+ t++;
+ }
+ }
+ if (level == 0) {
+ t = scst_trace_tbl;
+ while (t->token) {
+ if (!strcasecmp(p, t->token)) {
+ level = t->val;
+ break;
+ }
+ t++;
+ }
+ }
+ if (level == 0) {
+ PRINT_ERROR("Unknown token \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+ break;
+ case SCST_TRACE_ACTION_VALUE:
+ while (isspace(*p) && *p != '\0')
+ p++;
+ res = strict_strtoul(p, 0, &level);
+ if (res != 0) {
+ PRINT_ERROR("Invalid trace value \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+ break;
+ }
+
+ oldlevel = *log_level;
+
+ switch (action) {
+ case SCST_TRACE_ACTION_ADD:
+ *log_level |= level;
+ break;
+ case SCST_TRACE_ACTION_DEL:
+ *log_level &= ~level;
+ break;
+ default:
+ *log_level = level;
+ break;
+ }
+
+ PRINT_INFO("Changed trace level for \"%s\": old 0x%08lx, new 0x%08lx",
+ name, oldlevel, *log_level);
+
+out_free:
+ kfree(buffer);
+out:
+ return res;
+
+#undef SCST_TRACE_ACTION_ALL
+#undef SCST_TRACE_ACTION_NONE
+#undef SCST_TRACE_ACTION_DEFAULT
+#undef SCST_TRACE_ACTION_ADD
+#undef SCST_TRACE_ACTION_DEL
+#undef SCST_TRACE_ACTION_VALUE
+}
+
+static ssize_t scst_main_trace_level_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+
+ if (mutex_lock_interruptible(&scst_log_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = scst_write_trace(buf, count, &trace_flag,
+ SCST_DEFAULT_LOG_FLAGS, "scst", scst_local_trace_tbl);
+
+ mutex_unlock(&scst_log_mutex);
+
+out:
+ return res;
+}
+
+#endif /* defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_version_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+
+ sprintf(buf, "%s\n", SCST_VERSION_STRING);
+
+#ifdef CONFIG_SCST_STRICT_SERIALIZING
+ strcat(buf, "STRICT_SERIALIZING\n");
+#endif
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ strcat(buf, "EXTRACHECKS\n");
+#endif
+
+#ifdef CONFIG_SCST_TRACING
+ strcat(buf, "TRACING\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+ strcat(buf, "DEBUG\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_TM
+ strcat(buf, "DEBUG_TM\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_RETRY
+ strcat(buf, "DEBUG_RETRY\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_OOM
+ strcat(buf, "DEBUG_OOM\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG_SN
+ strcat(buf, "DEBUG_SN\n");
+#endif
+
+#ifdef CONFIG_SCST_USE_EXPECTED_VALUES
+ strcat(buf, "USE_EXPECTED_VALUES\n");
+#endif
+
+#ifdef CONFIG_SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ
+ strcat(buf, "ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ\n");
+#endif
+
+#ifdef CONFIG_SCST_STRICT_SECURITY
+ strcat(buf, "SCST_STRICT_SECURITY\n");
+#endif
+ return strlen(buf);
+}
+
+static struct kobj_attribute scst_threads_attr =
+ __ATTR(threads, S_IRUGO | S_IWUSR, scst_threads_show,
+ scst_threads_store);
+
+static struct kobj_attribute scst_setup_id_attr =
+ __ATTR(setup_id, S_IRUGO | S_IWUSR, scst_setup_id_show,
+ scst_setup_id_store);
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+static struct kobj_attribute scst_trace_level_attr =
+ __ATTR(trace_level, S_IRUGO | S_IWUSR, scst_main_trace_level_show,
+ scst_main_trace_level_store);
+#endif
+
+static struct kobj_attribute scst_version_attr =
+ __ATTR(version, S_IRUGO, scst_version_show, NULL);
+
+static struct attribute *scst_sysfs_root_default_attrs[] = {
+ &scst_threads_attr.attr,
+ &scst_setup_id_attr.attr,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ &scst_trace_level_attr.attr,
+#endif
+ &scst_version_attr.attr,
+ NULL,
+};
+
+static void scst_sysfs_root_release(struct kobject *kobj)
+{
+ complete_all(&scst_sysfs_root_release_completion);
+}
+
+static ssize_t scst_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ struct kobj_attribute *kobj_attr;
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ return kobj_attr->show(kobj, kobj_attr, buf);
+}
+
+static ssize_t scst_store(struct kobject *kobj, struct attribute *attr,
+ const char *buf, size_t count)
+{
+ struct kobj_attribute *kobj_attr;
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+
+ return kobj_attr->store(kobj, kobj_attr, buf, count);
+}
+
+struct sysfs_ops scst_sysfs_ops = {
+ .show = scst_show,
+ .store = scst_store,
+};
+
+static struct kobj_type scst_sysfs_root_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_sysfs_root_release,
+ .default_attrs = scst_sysfs_root_default_attrs,
+};
+
+static void scst_devt_free(struct kobject *kobj)
+{
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ complete_all(&devt->devt_kobj_release_compl);
+
+ scst_devt_cleanup(devt);
+ return;
+}
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+static ssize_t scst_devt_trace_level_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ return scst_trace_level_show(devt->trace_tbl,
+ devt->trace_flags ? *devt->trace_flags : 0, buf,
+ devt->trace_tbl_help);
+}
+
+static ssize_t scst_devt_trace_level_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ if (mutex_lock_interruptible(&scst_log_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ res = scst_write_trace(buf, count, devt->trace_flags,
+ devt->default_trace_flags, devt->name, devt->trace_tbl);
+
+ mutex_unlock(&scst_log_mutex);
+
+out:
+ return res;
+}
+
+static struct kobj_attribute devt_trace_attr =
+ __ATTR(trace_level, S_IRUGO | S_IWUSR,
+ scst_devt_trace_level_show, scst_devt_trace_level_store);
+
+#endif /* #if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING) */
+
+static ssize_t scst_devt_type_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ pos = sprintf(buf, "%d - %s\n", devt->type,
+ (unsigned)devt->type > ARRAY_SIZE(scst_dev_handler_types) ?
+ "unknown" : scst_dev_handler_types[devt->type]);
+
+ return pos;
+}
+
+static struct kobj_attribute scst_devt_type_attr =
+ __ATTR(type, S_IRUGO, scst_devt_type_show, NULL);
+
+static struct attribute *scst_devt_default_attrs[] = {
+ &scst_devt_type_attr.attr,
+ NULL,
+};
+
+static struct kobj_type scst_devt_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = scst_devt_free,
+ .default_attrs = scst_devt_default_attrs,
+};
+
+static ssize_t scst_devt_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ char *help = "Usage: echo \"add_device device_name [parameters]\" "
+ ">mgmt\n"
+ " echo \"del_device device_name\" >mgmt\n"
+ "%s"
+ "\n"
+ "where parameters are one or more "
+ "param_name=value pairs separated by ';'\n"
+ "%s%s";
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ if (devt->add_device_parameters_help != NULL)
+ return sprintf(buf, help,
+ (devt->mgmt_cmd_help) ? devt->mgmt_cmd_help : "",
+ "\nThe following parameters available: ",
+ devt->add_device_parameters_help);
+ else
+ return sprintf(buf, help,
+ (devt->mgmt_cmd_help) ? devt->mgmt_cmd_help : "",
+ "", "");
+}
+
+static ssize_t scst_devt_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ int res;
+ char *buffer, *p, *pp, *device_name;
+ struct scst_dev_type *devt;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ pp = buffer;
+ if (pp[strlen(pp) - 1] == '\n')
+ pp[strlen(pp) - 1] = '\0';
+
+ p = scst_get_next_lexem(&pp);
+
+ if (strcasecmp("add_device", p) == 0) {
+ device_name = scst_get_next_lexem(&pp);
+ if (*device_name == '\0') {
+ PRINT_ERROR("%s", "Device name required");
+ res = -EINVAL;
+ goto out_free;
+ }
+ res = devt->add_device(device_name, pp);
+ } else if (strcasecmp("del_device", p) == 0) {
+ device_name = scst_get_next_lexem(&pp);
+ if (*device_name == '\0') {
+ PRINT_ERROR("%s", "Device name required");
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ p = scst_get_next_lexem(&pp);
+ if (*p != '\0')
+ goto out_syntax_err;
+
+ res = devt->del_device(device_name);
+ } else if (devt->mgmt_cmd != NULL) {
+ scst_restore_token_str(p, pp);
+ res = devt->mgmt_cmd(buffer);
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ if (res == 0)
+ res = count;
+
+out_free:
+ kfree(buffer);
+
+out:
+ return res;
+
+out_syntax_err:
+ PRINT_ERROR("Syntax error on \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+}
+
+static struct kobj_attribute scst_devt_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_devt_mgmt_show,
+ scst_devt_mgmt_store);
+
+static ssize_t scst_devt_pass_through_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return sprintf(buf, "1");
+}
+
+static struct kobj_attribute scst_devt_pass_through =
+ __ATTR(pass_through, S_IRUGO, scst_devt_pass_through_show, NULL);
+
+static ssize_t scst_devt_pass_through_mgmt_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ char *help = "Usage: echo \"assign H:C:I:L\" >mgmt\n"
+ " echo \"unassign H:C:I:L\" >mgmt\n";
+ return sprintf(buf, help);
+}
+
+static ssize_t scst_devt_pass_through_mgmt_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ char *buffer, *p, *pp, *action;
+ struct scst_dev_type *devt;
+ unsigned long host, channel, id, lun;
+ struct scst_device *d, *dev = NULL;
+
+ devt = container_of(kobj, struct scst_dev_type, devt_kobj);
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ pp = buffer;
+ if (pp[strlen(pp) - 1] == '\n')
+ pp[strlen(pp) - 1] = '\0';
+
+ action = scst_get_next_lexem(&pp);
+ p = scst_get_next_lexem(&pp);
+ if (*p == '\0') {
+ PRINT_ERROR("%s", "Device required");
+ res = -EINVAL;
+ goto out_free;
+ }
+
+ ;
+ if (*scst_get_next_lexem(&pp) != '\0') {
+ PRINT_ERROR("%s", "Too many parameters");
+ res = -EINVAL;
+ goto out_syntax_err;
+ }
+
+ host = simple_strtoul(p, &p, 0);
+ if ((host == ULONG_MAX) || (*p != ':'))
+ goto out_syntax_err;
+ p++;
+ channel = simple_strtoul(p, &p, 0);
+ if ((channel == ULONG_MAX) || (*p != ':'))
+ goto out_syntax_err;
+ p++;
+ id = simple_strtoul(p, &p, 0);
+ if ((channel == ULONG_MAX) || (*p != ':'))
+ goto out_syntax_err;
+ p++;
+ lun = simple_strtoul(p, &p, 0);
+ if (lun == ULONG_MAX)
+ goto out_syntax_err;
+
+ TRACE_DBG("Dev %ld:%ld:%ld:%ld", host, channel, id, lun);
+
+ if (mutex_lock_interruptible(&scst_mutex) != 0) {
+ res = -EINTR;
+ goto out_free;
+ }
+
+ list_for_each_entry(d, &scst_dev_list, dev_list_entry) {
+ if ((d->virt_id == 0) &&
+ d->scsi_dev->host->host_no == host &&
+ d->scsi_dev->channel == channel &&
+ d->scsi_dev->id == id &&
+ d->scsi_dev->lun == lun) {
+ dev = d;
+ TRACE_DBG("Dev %p (%ld:%ld:%ld:%ld) found",
+ dev, host, channel, id, lun);
+ break;
+ }
+ }
+ if (dev == NULL) {
+ PRINT_ERROR("Device %ld:%ld:%ld:%ld not found",
+ host, channel, id, lun);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (dev->scsi_dev->type != devt->type) {
+ PRINT_ERROR("Type %d of device %s differs from type "
+ "%d of dev handler %s", dev->type,
+ dev->virt_name, devt->type, devt->name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (strcasecmp("assign", action) == 0)
+ res = scst_assign_dev_handler(dev, devt);
+ else if (strcasecmp("deassign", action) == 0) {
+ if (dev->handler != devt) {
+ PRINT_ERROR("Device %s is not assigned to handler %s",
+ dev->virt_name, devt->name);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+ res = scst_assign_dev_handler(dev, &scst_null_devtype);
+ } else {
+ PRINT_ERROR("Unknown action \"%s\"", action);
+ res = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (res == 0)
+ res = count;
+
+out_unlock:
+ mutex_unlock(&scst_mutex);
+
+out_free:
+ kfree(buffer);
+
+out:
+ return res;
+
+out_syntax_err:
+ PRINT_ERROR("Syntax error on \"%s\"", p);
+ res = -EINVAL;
+ goto out_free;
+}
+
+static struct kobj_attribute scst_devt_pass_through_mgmt =
+ __ATTR(mgmt, S_IRUGO | S_IWUSR, scst_devt_pass_through_mgmt_show,
+ scst_devt_pass_through_mgmt_store);
+
+int scst_create_devt_sysfs(struct scst_dev_type *devt)
+{
+ int retval;
+ struct kobject *parent;
+ const struct attribute **pattr;
+
+ init_completion(&devt->devt_kobj_release_compl);
+
+ if (devt->parent != NULL)
+ parent = &devt->parent->devt_kobj;
+ else
+ parent = scst_handlers_kobj;
+
+ devt->devt_kobj_initialized = 1;
+
+ retval = kobject_init_and_add(&devt->devt_kobj, &scst_devt_ktype,
+ parent, devt->name);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add devt %s to sysfs", devt->name);
+ goto out;
+ }
+
+ /*
+ * In case of errors there's no need for additional cleanup, because
+ * it will be done by the _put function() called by the caller.
+ */
+
+ if (devt->add_device != NULL) {
+ retval = sysfs_create_file(&devt->devt_kobj,
+ &scst_devt_mgmt.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add mgmt attr for dev handler %s",
+ devt->name);
+ goto out;
+ }
+ } else if (devt->pass_through) {
+ retval = sysfs_create_file(&devt->devt_kobj,
+ &scst_devt_pass_through_mgmt.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add mgmt attr for dev handler %s",
+ devt->name);
+ goto out;
+ }
+
+ retval = sysfs_create_file(&devt->devt_kobj,
+ &scst_devt_pass_through.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add pass_through attr for dev "
+ "handler %s", devt->name);
+ goto out;
+ }
+ }
+
+ pattr = devt->devt_attrs;
+ if (pattr != NULL) {
+ while (*pattr != NULL) {
+ retval = sysfs_create_file(&devt->devt_kobj, *pattr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add devt attr %s for dev "
+ "handler %s", (*pattr)->name,
+ devt->name);
+ goto out;
+ }
+ pattr++;
+ }
+ }
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ if (devt->trace_flags != NULL) {
+ retval = sysfs_create_file(&devt->devt_kobj,
+ &devt_trace_attr.attr);
+ if (retval != 0) {
+ PRINT_ERROR("Can't add devt trace_flag for dev "
+ "handler %s", devt->name);
+ goto out;
+ }
+ }
+#endif
+
+out:
+ return retval;
+}
+
+void scst_devt_sysfs_put(struct scst_dev_type *devt)
+{
+
+ if (devt->devt_kobj_initialized) {
+ int rc;
+
+ kobject_del(&devt->devt_kobj);
+ kobject_put(&devt->devt_kobj);
+
+ rc = wait_for_completion_timeout(&devt->devt_kobj_release_compl, HZ);
+ if (rc == 0) {
+ PRINT_INFO("Waiting for releasing sysfs entry "
+ "for dev handler template %s...", devt->name);
+ wait_for_completion(&devt->devt_kobj_release_compl);
+ PRINT_INFO("Done waiting for releasing sysfs entry "
+ "for dev handler template %s", devt->name);
+ }
+ } else
+ scst_devt_cleanup(devt);
+ return;
+}
+
+static DEFINE_MUTEX(scst_sysfs_user_info_mutex);
+
+/* All protected by scst_sysfs_user_info_mutex */
+static LIST_HEAD(scst_sysfs_user_info_list);
+static uint32_t scst_sysfs_info_cur_cookie;
+
+/* scst_sysfs_user_info_mutex supposed to be held */
+static struct scst_sysfs_user_info *scst_sysfs_user_find_info(uint32_t cookie)
+{
+ struct scst_sysfs_user_info *info, *res = NULL;
+
+ list_for_each_entry(info, &scst_sysfs_user_info_list,
+ info_list_entry) {
+ if (info->info_cookie == cookie) {
+ res = info;
+ break;
+ }
+ }
+ return res;
+}
+
+/**
+ * scst_sysfs_user_get_info() - get user_info
+ *
+ * Finds the user_info based on cookie and mark it as received the reply by
+ * setting for it flag info_being_executed.
+ *
+ * Returns found entry or NULL.
+ */
+struct scst_sysfs_user_info *scst_sysfs_user_get_info(uint32_t cookie)
+{
+ struct scst_sysfs_user_info *res = NULL;
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ res = scst_sysfs_user_find_info(cookie);
+ if (res != NULL) {
+ if (!res->info_being_executed)
+ res->info_being_executed = 1;
+ }
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_sysfs_user_get_info);
+
+/**
+ ** Helper functionality to help target drivers and dev handlers support
+ ** sending events to user space and wait for their completion in a safe
+ ** manner. See samples how to use it in iscsi-scst or scst_user.
+ **/
+
+/**
+ * scst_sysfs_user_add_info() - create and add user_info in the global list
+ *
+ * Creates an info structure and adds it in the info_list.
+ * Returns 0 and out_info on success, error code otherwise.
+ */
+int scst_sysfs_user_add_info(struct scst_sysfs_user_info **out_info)
+{
+ int res = 0;
+ struct scst_sysfs_user_info *info;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (info == NULL) {
+ PRINT_ERROR("Unable to allocate sysfs user info (size %zd)",
+ sizeof(*info));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ while ((info->info_cookie == 0) ||
+ (scst_sysfs_user_find_info(info->info_cookie) != NULL))
+ info->info_cookie = scst_sysfs_info_cur_cookie++;
+
+ init_completion(&info->info_completion);
+
+ list_add_tail(&info->info_list_entry, &scst_sysfs_user_info_list);
+ info->info_in_list = 1;
+
+ *out_info = info;
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_sysfs_user_add_info);
+
+/**
+ * scst_sysfs_user_del_info - delete and frees user_info
+ */
+void scst_sysfs_user_del_info(struct scst_sysfs_user_info *info)
+{
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ if (info->info_in_list)
+ list_del(&info->info_list_entry);
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+
+ kfree(info);
+ return;
+}
+EXPORT_SYMBOL_GPL(scst_sysfs_user_del_info);
+
+/*
+ * Returns true if the reply received and being processed by another part of
+ * the kernel, false otherwise. Also removes the user_info from the list to
+ * fix for the user space that it missed the timeout.
+ */
+static bool scst_sysfs_user_info_executing(struct scst_sysfs_user_info *info)
+{
+ bool res;
+
+ mutex_lock(&scst_sysfs_user_info_mutex);
+
+ res = info->info_being_executed;
+
+ if (info->info_in_list) {
+ list_del(&info->info_list_entry);
+ info->info_in_list = 0;
+ }
+
+ mutex_unlock(&scst_sysfs_user_info_mutex);
+ return res;
+}
+
+/**
+ * scst_wait_info_completion() - wait an user space event's completion
+ *
+ * Waits for the info request been completed by user space at most timeout
+ * jiffies. If the reply received before timeout and being processed by
+ * another part of the kernel, i.e. scst_sysfs_user_info_executing()
+ * returned true, waits for it to complete indefinitely.
+ *
+ * Returns status of the request completion.
+ */
+int scst_wait_info_completion(struct scst_sysfs_user_info *info,
+ unsigned long timeout)
+{
+ int res, rc;
+
+ TRACE_DBG("Waiting for info %p completion", info);
+
+ while (1) {
+ rc = wait_for_completion_interruptible_timeout(
+ &info->info_completion, timeout);
+ if (rc > 0) {
+ TRACE_DBG("Waiting for info %p finished with %d",
+ info, rc);
+ break;
+ } else if (rc == 0) {
+ if (!scst_sysfs_user_info_executing(info)) {
+ PRINT_ERROR("Timeout waiting for user "
+ "space event %p", info);
+ res = -EBUSY;
+ goto out;
+ } else {
+ /* Req is being executed in the kernel */
+ TRACE_DBG("Keep waiting for info %p completion",
+ info);
+ wait_for_completion(&info->info_completion);
+ break;
+ }
+ } else if (rc != -ERESTARTSYS) {
+ res = rc;
+ PRINT_ERROR("wait_for_completion() failed: %d",
+ res);
+ goto out;
+ } else {
+ TRACE_DBG("Waiting for info %p finished with %d, "
+ "retrying", info, rc);
+ }
+ }
+
+ TRACE_DBG("info %p, status %d", info, info->info_status);
+ res = info->info_status;
+
+out:
+ return res;
+}
+EXPORT_SYMBOL_GPL(scst_wait_info_completion);
+
+int __init scst_sysfs_init(void)
+{
+ int retval = 0;
+
+ retval = kobject_init_and_add(&scst_sysfs_root_kobj,
+ &scst_sysfs_root_ktype, kernel_kobj, "%s", "scst_tgt");
+ if (retval != 0)
+ goto sysfs_root_add_error;
+
+ scst_targets_kobj = kobject_create_and_add("targets",
+ &scst_sysfs_root_kobj);
+ if (scst_targets_kobj == NULL)
+ goto targets_kobj_error;
+
+ scst_devices_kobj = kobject_create_and_add("devices",
+ &scst_sysfs_root_kobj);
+ if (scst_devices_kobj == NULL)
+ goto devices_kobj_error;
+
+ scst_sgv_kobj = kzalloc(sizeof(*scst_sgv_kobj), GFP_KERNEL);
+ if (scst_sgv_kobj == NULL)
+ goto sgv_kobj_error;
+
+ retval = kobject_init_and_add(scst_sgv_kobj, &sgv_ktype,
+ &scst_sysfs_root_kobj, "%s", "sgv");
+ if (retval != 0)
+ goto sgv_kobj_add_error;
+
+ scst_handlers_kobj = kobject_create_and_add("handlers",
+ &scst_sysfs_root_kobj);
+ if (scst_handlers_kobj == NULL)
+ goto handlers_kobj_error;
+
+out:
+ return retval;
+
+handlers_kobj_error:
+ kobject_del(scst_sgv_kobj);
+
+sgv_kobj_add_error:
+ kobject_put(scst_sgv_kobj);
+
+sgv_kobj_error:
+ kobject_del(scst_devices_kobj);
+ kobject_put(scst_devices_kobj);
+
+devices_kobj_error:
+ kobject_del(scst_targets_kobj);
+ kobject_put(scst_targets_kobj);
+
+targets_kobj_error:
+ kobject_del(&scst_sysfs_root_kobj);
+
+sysfs_root_add_error:
+ kobject_put(&scst_sysfs_root_kobj);
+
+ if (retval == 0)
+ retval = -EINVAL;
+ goto out;
+}
+
+void scst_sysfs_cleanup(void)
+{
+
+ PRINT_INFO("%s", "Exiting SCST sysfs hierarchy...");
+
+ kobject_del(scst_sgv_kobj);
+ kobject_put(scst_sgv_kobj);
+
+ kobject_del(scst_devices_kobj);
+ kobject_put(scst_devices_kobj);
+
+ kobject_del(scst_targets_kobj);
+ kobject_put(scst_targets_kobj);
+
+ kobject_del(scst_handlers_kobj);
+ kobject_put(scst_handlers_kobj);
+
+ kobject_del(&scst_sysfs_root_kobj);
+ kobject_put(&scst_sysfs_root_kobj);
+
+ wait_for_completion(&scst_sysfs_root_release_completion);
+ /*
+ * There is a race, when in the release() schedule happens just after
+ * calling complete(), so if we exit and unload scst module immediately,
+ * there will be oops there. So let's give it a chance to quit
+ * gracefully. Unfortunately, current kobjects implementation
+ * doesn't allow better ways to handle it.
+ */
+ msleep(3000);
+
+ PRINT_INFO("%s", "Exiting SCST sysfs hierarchy done");
+ return;
+}
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 9/12/1/5] SCST debugging support
[not found] ` <4BC44D08.4060907@vlnb.net>
` (7 preceding siblings ...)
2010-04-13 13:06 ` [PATCH][RFC 8/12/1/5] SCST sysfs interface Vladislav Bolkhovitin
@ 2010-04-13 13:06 ` Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 10/12/1/5] SCST external modules support Vladislav Bolkhovitin
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:06 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains SCST debugging support routines.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
drivers/scst/scst_debug.c | 136 ++++++++++++++++++++++
include/scst/scst_debug.h | 276 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 412 insertions(+)
diff -uprN orig/linux-2.6.33/include/scst/scst_debug.h linux-2.6.33/include/scst/scst_debug.h
--- orig/linux-2.6.33/include/scst/scst_debug.h
+++ linux-2.6.33/include/scst/scst_debug.h
@@ -0,0 +1,276 @@
+/*
+ * include/scst_debug.h
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Contains macroses for execution tracing and error reporting
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __SCST_DEBUG_H
+#define __SCST_DEBUG_H
+
+#include <generated/autoconf.h> /* for CONFIG_* */
+
+#include <linux/bug.h> /* for WARN_ON_ONCE */
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+#define EXTRACHECKS_BUG_ON(a) BUG_ON(a)
+#define EXTRACHECKS_WARN_ON(a) WARN_ON(a)
+#define EXTRACHECKS_WARN_ON_ONCE(a) WARN_ON_ONCE(a)
+#else
+#define EXTRACHECKS_BUG_ON(a) do { } while (0)
+#define EXTRACHECKS_WARN_ON(a) do { } while (0)
+#define EXTRACHECKS_WARN_ON_ONCE(a) do { } while (0)
+#endif
+
+#define TRACE_NULL 0x00000000
+#define TRACE_DEBUG 0x00000001
+#define TRACE_FUNCTION 0x00000002
+#define TRACE_LINE 0x00000004
+#define TRACE_PID 0x00000008
+#define TRACE_BUFF 0x00000020
+#define TRACE_MEMORY 0x00000040
+#define TRACE_SG_OP 0x00000080
+#define TRACE_OUT_OF_MEM 0x00000100
+#define TRACE_MINOR 0x00000200 /* less important events */
+#define TRACE_MGMT 0x00000400
+#define TRACE_MGMT_DEBUG 0x00000800
+#define TRACE_SCSI 0x00001000
+#define TRACE_SPECIAL 0x00002000 /* filtering debug, etc */
+#define TRACE_FLOW_CONTROL 0x00004000 /* flow control in action */
+#define TRACE_ALL 0xffffffff
+/* Flags 0xXXXX0000 are local for users */
+
+#define TRACE_MINOR_AND_MGMT_DBG (TRACE_MINOR|TRACE_MGMT_DEBUG)
+
+#ifndef KERN_CONT
+#define KERN_CONT ""
+#endif
+
+/*
+ * Note: in the next two printk() statements the KERN_CONT macro is only
+ * present to suppress a checkpatch warning (KERN_CONT is defined as "").
+ */
+#define PRINT(log_flag, format, args...) \
+ printk(log_flag format "\n", ## args)
+#define PRINTN(log_flag, format, args...) \
+ printk(log_flag format, ## args)
+
+#ifdef LOG_PREFIX
+#define __LOG_PREFIX LOG_PREFIX
+#else
+#define __LOG_PREFIX NULL
+#endif
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+#ifndef CONFIG_SCST_DEBUG
+#define ___unlikely(a) (a)
+#else
+#define ___unlikely(a) unlikely(a)
+#endif
+
+/*
+ * We don't print prefix for debug traces to not put additional preasure
+ * on the logging system in case of a lot of logging.
+ */
+
+extern int debug_print_prefix(unsigned long trace_flag,
+ const char *prefix, const char *func, int line);
+extern void debug_print_buffer(const void *data, int len);
+
+#define TRACE(trace, format, args...) \
+do { \
+ if (___unlikely(trace_flag & (trace))) { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, \
+ __func__, __LINE__); \
+ PRINT(KERN_CONT, format, args); \
+ } \
+} while (0)
+
+#ifdef CONFIG_SCST_DEBUG
+
+#define PRINT_BUFFER(message, buff, len) \
+do { \
+ PRINT(KERN_INFO, "%s:%s:", __func__, message); \
+ debug_print_buffer(buff, len); \
+} while (0)
+
+#else
+
+#define PRINT_BUFFER(message, buff, len) \
+do { \
+ PRINT(KERN_INFO, "%s:", message); \
+ debug_print_buffer(buff, len); \
+} while (0)
+
+#endif
+
+#define PRINT_BUFF_FLAG(flag, message, buff, len) \
+do { \
+ if (___unlikely(trace_flag & (flag))) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, "%s:", message); \
+ debug_print_buffer(buff, len); \
+ } \
+} while (0)
+
+#else /* CONFIG_SCST_DEBUG || CONFIG_SCST_TRACING */
+
+#define TRACE(trace, args...) do {} while (0)
+#define PRINT_BUFFER(message, buff, len) do {} while (0)
+#define PRINT_BUFF_FLAG(flag, message, buff, len) do {} while (0)
+
+#endif /* CONFIG_SCST_DEBUG || CONFIG_SCST_TRACING */
+
+#ifdef CONFIG_SCST_DEBUG
+
+#define TRACE_DBG_FLAG(trace, format, args...) \
+do { \
+ if (trace_flag & (trace)) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, format, args); \
+ } \
+} while (0)
+
+#define TRACE_MEM(args...) TRACE_DBG_FLAG(TRACE_MEMORY, args)
+#define TRACE_SG(args...) TRACE_DBG_FLAG(TRACE_SG_OP, args)
+#define TRACE_DBG(args...) TRACE_DBG_FLAG(TRACE_DEBUG, args)
+#define TRACE_DBG_SPECIAL(args...) TRACE_DBG_FLAG(TRACE_DEBUG|TRACE_SPECIAL, args)
+#define TRACE_MGMT_DBG(args...) TRACE_DBG_FLAG(TRACE_MGMT_DEBUG, args)
+#define TRACE_MGMT_DBG_SPECIAL(args...) \
+ TRACE_DBG_FLAG(TRACE_MGMT_DEBUG|TRACE_SPECIAL, args)
+
+#define TRACE_BUFFER(message, buff, len) \
+do { \
+ if (trace_flag & TRACE_BUFF) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, "%s:", message); \
+ debug_print_buffer(buff, len); \
+ } \
+} while (0)
+
+#define TRACE_BUFF_FLAG(flag, message, buff, len) \
+do { \
+ if (trace_flag & (flag)) { \
+ debug_print_prefix(trace_flag, NULL, __func__, __LINE__);\
+ PRINT(KERN_CONT, "%s:", message); \
+ debug_print_buffer(buff, len); \
+ } \
+} while (0)
+
+#define PRINT_LOG_FLAG(log_flag, format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, format, args); \
+} while (0)
+
+#define PRINT_WARNING(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, "***WARNING***: " format, args); \
+} while (0)
+
+#define PRINT_ERROR(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, "***ERROR***: " format, args); \
+} while (0)
+
+#define PRINT_CRIT_ERROR(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, "***CRITICAL ERROR***: " format, args); \
+} while (0)
+
+#define PRINT_INFO(format, args...) \
+do { \
+ debug_print_prefix(trace_flag, __LOG_PREFIX, __func__, __LINE__);\
+ PRINT(KERN_CONT, format, args); \
+} while (0)
+
+#else /* CONFIG_SCST_DEBUG */
+
+#define TRACE_MEM(format, args...) do {} while (0)
+#define TRACE_SG(format, args...) do {} while (0)
+#define TRACE_DBG(format, args...) do {} while (0)
+#define TRACE_DBG_FLAG(format, args...) do {} while (0)
+#define TRACE_DBG_SPECIAL(format, args...) do {} while (0)
+#define TRACE_MGMT_DBG(format, args...) do {} while (0)
+#define TRACE_MGMT_DBG_SPECIAL(format, args...) do {} while (0)
+#define TRACE_BUFFER(message, buff, len) do {} while (0)
+#define TRACE_BUFF_FLAG(flag, message, buff, len) do {} while (0)
+
+#ifdef LOG_PREFIX
+
+#define PRINT_INFO(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: " format, LOG_PREFIX, args); \
+} while (0)
+
+#define PRINT_WARNING(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: ***WARNING***: " \
+ format, LOG_PREFIX, args); \
+} while (0)
+
+#define PRINT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: ***ERROR***: " \
+ format, LOG_PREFIX, args); \
+} while (0)
+
+#define PRINT_CRIT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_INFO, "%s: ***CRITICAL ERROR***: " \
+ format, LOG_PREFIX, args); \
+} while (0)
+
+#else
+
+#define PRINT_INFO(format, args...) \
+do { \
+ PRINT(KERN_INFO, format, args); \
+} while (0)
+
+#define PRINT_WARNING(format, args...) \
+do { \
+ PRINT(KERN_INFO, "***WARNING***: " \
+ format, args); \
+} while (0)
+
+#define PRINT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_ERR, "***ERROR***: " \
+ format, args); \
+} while (0)
+
+#define PRINT_CRIT_ERROR(format, args...) \
+do { \
+ PRINT(KERN_CRIT, "***CRITICAL ERROR***: " \
+ format, args); \
+} while (0)
+
+#endif /* LOG_PREFIX */
+
+#endif /* CONFIG_SCST_DEBUG */
+
+#if defined(CONFIG_SCST_DEBUG) && defined(CONFIG_DEBUG_SLAB)
+#define SCST_SLAB_FLAGS (SLAB_RED_ZONE | SLAB_POISON)
+#else
+#define SCST_SLAB_FLAGS 0L
+#endif
+
+#endif /* __SCST_DEBUG_H */
diff -uprN orig/linux-2.6.33/drivers/scst/scst_debug.c linux-2.6.33/drivers/scst/scst_debug.c
--- orig/linux-2.6.33/drivers/scst/scst_debug.c
+++ linux-2.6.33/drivers/scst/scst_debug.c
@@ -0,0 +1,136 @@
+/*
+ * scst_debug.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Contains helper functions for execution tracing and error reporting.
+ * Intended to be included in main .c file.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "scst.h"
+#include "scst_debug.h"
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+
+#define TRACE_BUF_SIZE 512
+
+static char trace_buf[TRACE_BUF_SIZE];
+static DEFINE_SPINLOCK(trace_buf_lock);
+
+static inline int get_current_tid(void)
+{
+ /* Code should be the same as in sys_gettid() */
+ if (in_interrupt()) {
+ /*
+ * Unfortunately, task_pid_vnr() isn't IRQ-safe, so otherwise
+ * it can oops. ToDo.
+ */
+ return 0;
+ }
+ return task_pid_vnr(current);
+}
+
+/**
+ * debug_print_prefix() - print debug prefix for a log line
+ *
+ * Prints, if requested by trace_flag, debug prefix for each log line
+ */
+int debug_print_prefix(unsigned long trace_flag,
+ const char *prefix, const char *func, int line)
+{
+ int i = 0;
+ unsigned long flags;
+ int pid = get_current_tid();
+
+ spin_lock_irqsave(&trace_buf_lock, flags);
+
+ trace_buf[0] = '\0';
+
+ if (trace_flag & TRACE_PID)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE, "[%d]: ", pid);
+ if (prefix != NULL)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%s: ",
+ prefix);
+ if (trace_flag & TRACE_FUNCTION)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%s:", func);
+ if (trace_flag & TRACE_LINE)
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%i:", line);
+
+ PRINTN(KERN_INFO, "%s", trace_buf);
+
+ spin_unlock_irqrestore(&trace_buf_lock, flags);
+
+ return i;
+}
+EXPORT_SYMBOL(debug_print_prefix);
+
+/**
+ * debug_print_buffer() - print a buffer
+ *
+ * Prints in the log data from the buffer
+ */
+void debug_print_buffer(const void *data, int len)
+{
+ int z, z1, i;
+ const unsigned char *buf = (const unsigned char *) data;
+ unsigned long flags;
+
+ if (buf == NULL)
+ return;
+
+ spin_lock_irqsave(&trace_buf_lock, flags);
+
+ PRINT(KERN_INFO, " (h)___0__1__2__3__4__5__6__7__8__9__A__B__C__D__E__F");
+ for (z = 0, z1 = 0, i = 0; z < len; z++) {
+ if (z % 16 == 0) {
+ if (z != 0) {
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i,
+ " ");
+ for (; (z1 < z) && (i < TRACE_BUF_SIZE - 1);
+ z1++) {
+ if ((buf[z1] >= 0x20) &&
+ (buf[z1] < 0x80))
+ trace_buf[i++] = buf[z1];
+ else
+ trace_buf[i++] = '.';
+ }
+ trace_buf[i] = '\0';
+ PRINT(KERN_INFO, "%s", trace_buf);
+ i = 0;
+ }
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i,
+ "%4x: ", z);
+ }
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, "%02x ",
+ buf[z]);
+ }
+
+ i += snprintf(&trace_buf[i], TRACE_BUF_SIZE - i, " ");
+ for (; (z1 < z) && (i < TRACE_BUF_SIZE - 1); z1++) {
+ if ((buf[z1] > 0x20) && (buf[z1] < 0x80))
+ trace_buf[i++] = buf[z1];
+ else
+ trace_buf[i++] = '.';
+ }
+ trace_buf[i] = '\0';
+
+ PRINT(KERN_INFO, "%s", trace_buf);
+
+ spin_unlock_irqrestore(&trace_buf_lock, flags);
+ return;
+}
+EXPORT_SYMBOL(debug_print_buffer);
+
+#endif /* CONFIG_SCST_DEBUG || CONFIG_SCST_TRACING */
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 10/12/1/5] SCST external modules support
[not found] ` <4BC44D08.4060907@vlnb.net>
` (8 preceding siblings ...)
2010-04-13 13:06 ` [PATCH][RFC 9/12/1/5] SCST debugging support Vladislav Bolkhovitin
@ 2010-04-13 13:06 ` Vladislav Bolkhovitin
9 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:06 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains SCST external modules support routines.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
scst_module.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/scst_module.c linux-2.6.33/drivers/scst/scst_module.c
--- orig/linux-2.6.33/drivers/scst/scst_module.c
+++ linux-2.6.33/drivers/scst/scst_module.c
@@ -0,0 +1,63 @@
+/*
+ * scst_module.c
+ *
+ * Copyright (C) 2004 - 2010 Vladislav Bolkhovitin <vst@vlnb.net>
+ * Copyright (C) 2004 - 2005 Leonid Stoljar
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * Support for loading target modules. The usage is similar to scsi_module.c
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+
+#include <scst.h>
+
+static int __init init_this_scst_driver(void)
+{
+ int res;
+
+ res = scst_register_target_template(&driver_target_template);
+ TRACE_DBG("scst_register_target_template() returned %d", res);
+ if (res < 0)
+ goto out;
+
+#ifdef SCST_REGISTER_INITIATOR_DRIVER
+ driver_template.module = THIS_MODULE;
+ scsi_register_module(MODULE_SCSI_HA, &driver_template);
+ TRACE_DBG("driver_template.present=%d",
+ driver_template.present);
+ if (driver_template.present == 0) {
+ res = -ENODEV;
+ MOD_DEC_USE_COUNT;
+ goto out;
+ }
+#endif
+
+out:
+ return res;
+}
+
+static void __exit exit_this_scst_driver(void)
+{
+
+#ifdef SCST_REGISTER_INITIATOR_DRIVER
+ scsi_unregister_module(MODULE_SCSI_HA, &driver_template);
+#endif
+
+ scst_unregister_target_template(&driver_target_template);
+ return;
+}
+
+module_init(init_this_scst_driver);
+module_exit(exit_this_scst_driver);
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 11/12/1/5] SCST core's README
[not found] ` <4BC45841.2090707@vlnb.net>
@ 2010-04-13 13:07 ` Vladislav Bolkhovitin
2010-04-13 13:07 ` [PATCH][RFC 12/12/1/5] SCST sysfs rules Vladislav Bolkhovitin
1 sibling, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:07 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains SCST core's README.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
README.scst | 1379 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 1379 insertions(+)
diff -uprN orig/linux-2.6.33/Documentation/scst/README.scst linux-2.6.33/Documentation/scst/README.scst
--- orig/linux-2.6.33/Documentation/scst/README.scst
+++ linux-2.6.33/Documentation/scst/README.scst
@@ -0,0 +1,1379 @@
+Generic SCSI target mid-level for Linux (SCST)
+==============================================
+
+SCST is designed to provide unified, consistent interface between SCSI
+target drivers and Linux kernel and simplify target drivers development
+as much as possible. Detail description of SCST's features and internals
+could be found in "Generic SCSI Target Middle Level for Linux" document
+SCST's Internet page http://scst.sourceforge.net.
+
+SCST supports the following I/O modes:
+
+ * Pass-through mode with one to many relationship, i.e. when multiple
+ initiators can connect to the exported pass-through devices, for
+ the following SCSI devices types: disks (type 0), tapes (type 1),
+ processors (type 3), CDROMs (type 5), MO disks (type 7), medium
+ changers (type 8) and RAID controllers (type 0xC)
+
+ * FILEIO mode, which allows to use files on file systems or block
+ devices as virtual remotely available SCSI disks or CDROMs with
+ benefits of the Linux page cache
+
+ * BLOCKIO mode, which performs direct block IO with a block device,
+ bypassing page-cache for all operations. This mode works ideally with
+ high-end storage HBAs and for applications that either do not need
+ caching between application and disk or need the large block
+ throughput
+
+ * User space mode using scst_user device handler, which allows to
+ implement in the user space virtual SCSI devices in the SCST
+ environment
+
+ * "Performance" device handlers, which provide in pseudo pass-through
+ mode a way for direct performance measurements without overhead of
+ actual data transferring from/to underlying SCSI device
+
+In addition, SCST supports advanced per-initiator access and devices
+visibility management, so different initiators could see different set
+of devices with different access permissions. See below for details.
+
+Installation
+------------
+
+To see your devices remotely, you need to add them to at least "Default"
+security group (see below how). By default, no local devices are seen
+remotely. There must be LUN 0 in each security group, i.e. LUs
+numeration must not start from, e.g., 1. Otherwise you will see no
+devices on remote initiators and SCST core will write into the kernel
+log message: "tgt_dev for LUN 0 not found, command to unexisting LU?"
+
+It is highly recommended to use scstadmin utility for configuring
+devices and security groups.
+
+If you experience problems during modules load or running, check your
+kernel logs (or run dmesg command for the few most recent messages).
+
+IMPORTANT: Without loading appropriate device handler, corresponding devices
+========= will be invisible for remote initiators, which could lead to holes
+ in the LUN addressing, so automatic device scanning by remote SCSI
+ mid-level could not notice the devices. Therefore you will have
+ to add them manually via
+ 'echo "- - -" >/sys/class/scsi_host/hostX/scan',
+ where X - is the host number.
+
+IMPORTANT: Working of target and initiator on the same host is
+========= supported, except the following 2 cases: swap over target exported
+ device and using a writable mmap over a file from target
+ exported device. The latter means you can't mount a file
+ system over target exported device. In other words, you can
+ freely use any sg, sd, st, etc. devices imported from target
+ on the same host, but you can't mount file systems or put
+ swap on them. This is a limitation of Linux memory/cache
+ manager, because in this case an OOM deadlock like: system
+ needs some memory -> it decides to clear some cache -> cache
+ needs to write on target exported device -> initiator sends
+ request to the target -> target needs memory -> system needs
+ even more memory -> deadlock.
+
+IMPORTANT: In the current version simultaneous access to local SCSI devices
+========= via standard high-level SCSI drivers (sd, st, sg, etc.) and
+ SCST's target drivers is unsupported. Especially it is
+ important for execution via sg and st commands that change
+ the state of devices and their parameters, because that could
+ lead to data corruption. If any such command is done, at
+ least related device handler(s) must be restarted. For block
+ devices READ/WRITE commands using direct disk handler look to
+ be safe.
+
+IMPORTANT: Some versions of Windows have a bug, which makes them consider
+========= response of READ CAPACITY(16) longer than 12 bytes as a faulty one.
+ As the result, such Windows'es refuse to see SCST exported
+ devices >2TB in size. This is fixed by MS in latter Windows
+ versions, probably, by some hotfix. But if you're using such
+ buggy Windows and experience this problem, change in
+ scst_vdisk.c::vdisk_exec_read_capacity16() "#if 1" to "#if 0".
+
+Usage in failover mode
+----------------------
+
+It is recommended to use TEST UNIT READY ("tur") command to check if
+SCST target is alive in MPIO configurations.
+
+Device handlers
+---------------
+
+Device specific drivers (device handlers) are plugins for SCST, which
+help SCST to analyze incoming requests and determine parameters,
+specific to various types of devices. If an appropriate device handler
+for a SCSI device type isn't loaded, SCST doesn't know how to handle
+devices of this type, so they will be invisible for remote initiators
+(more precisely, "LUN not supported" sense code will be returned).
+
+In addition to device handlers for real devices, there are VDISK, user
+space and "performance" device handlers.
+
+VDISK device handler works over files on file systems and makes from
+them virtual remotely available SCSI disks or CDROM's. In addition, it
+allows to work directly over a block device, e.g. local IDE or SCSI disk
+or ever disk partition, where there is no file systems overhead. Using
+block devices comparing to sending SCSI commands directly to SCSI
+mid-level via scsi_do_req()/scsi_execute_async() has advantage that data
+are transferred via system cache, so it is possible to fully benefit from
+caching and read ahead performed by Linux's VM subsystem. The only
+disadvantage here that in the FILEIO mode there is superfluous data
+copying between the cache and SCST's buffers. This issue is going to be
+addressed in the next release. Virtual CDROM's are useful for remote
+installation. See below for details how to setup and use VDISK device
+handler.
+
+SCST user space device handler provides an interface between SCST and
+the user space, which allows to create pure user space devices. The
+simplest example, where one would want it is if he/she wants to write a
+VTL. With scst_user he/she can write it purely in the user space. Or one
+would want it if he/she needs some sophisticated for kernel space
+processing of the passed data, like encrypting them or making snapshots.
+
+"Performance" device handlers for disks, MO disks and tapes in their
+exec() method skip (pretend to execute) all READ and WRITE operations
+and thus provide a way for direct link performance measurements without
+overhead of actual data transferring from/to underlying SCSI device.
+
+NOTE: Since "perf" device handlers on READ operations don't touch the
+==== commands' data buffer, it is returned to remote initiators as it
+ was allocated, without even being zeroed. Thus, "perf" device
+ handlers impose some security risk, so use them with caution.
+
+Compilation options
+-------------------
+
+There are the following compilation options, that could be change using
+your favorite kernel configuration Makefile target, e.g. "make xconfig":
+
+ - CONFIG_SCST_DEBUG - if defined, turns on some debugging code,
+ including some logging. Makes the driver considerably bigger and slower,
+ producing large amount of log data.
+
+ - CONFIG_SCST_TRACING - if defined, turns on ability to log events. Makes the
+ driver considerably bigger and leads to some performance loss.
+
+ - CONFIG_SCST_EXTRACHECKS - if defined, adds extra validity checks in
+ the various places.
+
+ - CONFIG_SCST_USE_EXPECTED_VALUES - if not defined (default), initiator
+ supplied expected data transfer length and direction will be used only for
+ verification purposes to return error or warn in case if one of them
+ is invalid. Instead, locally decoded from SCSI command values will be
+ used. This is necessary for security reasons, because otherwise a
+ faulty initiator can crash target by supplying invalid value in one
+ of those parameters. This is especially important in case of
+ pass-through mode. If CONFIG_SCST_USE_EXPECTED_VALUES is defined, initiator
+ supplied expected data transfer length and direction will override
+ the locally decoded values. This might be necessary if internal SCST
+ commands translation table doesn't contain SCSI command, which is
+ used in your environment. You can know that if you have messages like
+ "Unknown opcode XX for YY. Should you update scst_scsi_op_table?" in
+ your kernel log and your initiator returns an error. Also report
+ those messages in the SCST mailing list
+ scst-devel@lists.sourceforge.net. Note, that not all SCSI transports
+ support supplying expected values.
+
+ - CONFIG_SCST_DEBUG_TM - if defined, turns on task management functions
+ debugging, when on LUN 6 some of the commands will be delayed for
+ about 60 sec., so making the remote initiator send TM functions, eg
+ ABORT TASK and TARGET RESET. Also define
+ CONFIG_SCST_TM_DBG_GO_OFFLINE symbol in the Makefile if you want that
+ the device eventually become completely unresponsive, or otherwise to
+ circle around ABORTs and RESETs code. Needs CONFIG_SCST_DEBUG turned
+ on.
+
+ - CONFIG_SCST_STRICT_SERIALIZING - if defined, makes SCST send all commands to
+ underlying SCSI device synchronously, one after one. This makes task
+ management more reliable, with cost of some performance penalty. This
+ is mostly actual for stateful SCSI devices like tapes, where the
+ result of command's execution depends from device's settings defined
+ by previous commands. Disk and RAID devices are stateless in the most
+ cases. The current SCSI core in Linux doesn't allow to abort all
+ commands reliably if they sent asynchronously to a stateful device.
+ Turned off by default, turn it on if you use stateful device(s) and
+ need as much error recovery reliability as possible. As a side effect
+ of CONFIG_SCST_STRICT_SERIALIZING, on kernels below 2.6.30 no kernel
+ patching is necessary for pass-through device handlers (scst_disk,
+ etc.).
+
+ - CONFIG_SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ - if defined, it will be
+ allowed to submit pass-through commands to real SCSI devices via the SCSI
+ middle layer using scsi_execute_async() function from soft IRQ
+ context (tasklets). This used to be the default, but currently it
+ seems the SCSI middle layer starts expecting only thread context on
+ the IO submit path, so it is disabled now by default. Enabling it
+ will decrease amount of context switches and improve performance. It
+ is more or less safe, in the worst case, if in your configuration the
+ SCSI middle layer really doesn't expect SIRQ context in
+ scsi_execute_async() function, you will get a warning message in the
+ kernel log.
+
+ - CONFIG_SCST_STRICT_SECURITY - if defined, makes SCST zero allocated data
+ buffers. Undefining it (default) considerably improves performance
+ and eases CPU load, but could create a security hole (information
+ leakage), so enable it, if you have strict security requirements.
+
+ - CONFIG_SCST_ABORT_CONSIDER_FINISHED_TASKS_AS_NOT_EXISTING - if defined,
+ in case when TASK MANAGEMENT function ABORT TASK is trying to abort a
+ command, which has already finished, remote initiator, which sent the
+ ABORT TASK request, will receive TASK NOT EXIST (or ABORT FAILED)
+ response for the ABORT TASK request. This is more logical response,
+ since, because the command finished, attempt to abort it failed, but
+ some initiators, particularly VMware iSCSI initiator, consider TASK
+ NOT EXIST response as if the target got crazy and try to RESET it.
+ Then sometimes get crazy itself. So, this option is disabled by
+ default.
+
+ - CONFIG_SCST_MEASURE_LATENCY - if defined, provides in /sys/kernel/scst_tgt
+ and below statisctics about average commands processing latency. You
+ can clear already measured results by writing 0 in the corresponding
+ file. Note, you need a non-preemptible kernel to have correct
+ results.
+
+HIGHMEM kernel configurations are fully supported, but not recommended
+for performance reasons, except for scst_user, where they are not
+supported, because this module deals with user supplied memory on a
+zero-copy manner. If you need to use it, consider change VMSPLIT option
+or use 64-bit system configuration instead.
+
+For changing VMSPLIT option (CONFIG_VMSPLIT to be precise) you should in
+"make menuconfig" command set the following variables:
+
+ - General setup->Configure standard kernel features (for small systems): ON
+
+ - General setup->Prompt for development and/or incomplete code/drivers: ON
+
+ - Processor type and features->High Memory Support: OFF
+
+ - Processor type and features->Memory split: according to amount of
+ memory you have. If it is less than 800MB, you may not touch this
+ option at all.
+
+Module parameters
+-----------------
+
+Module scst supports the following parameters:
+
+ - scst_threads - allows to set count of SCST's threads. By default it
+ is CPU count.
+
+ - scst_max_cmd_mem - sets maximum amount of memory in Mb allowed to be
+ consumed by the SCST commands for data buffers at any given time. By
+ default it is approximately TotalMem/4.
+
+SCST sysfs interface
+--------------------
+
+Root of SCST sysfs interface is /sys/kernel/scst_tgt. It has the
+following entries:
+
+ - devices - this is a root subdirectory for all SCST devices
+
+ - handlers - this is a root subdirectory for all SCST dev handlers
+
+ - sgv - this is a root subdirectory for all SCST SGV caches
+
+ - targets - this is a root subdirectory for all SCST targets
+
+ - setup_id - allows to read and write SCST setup ID. This ID can be
+ used in cases, when the same SCST configuration should be installed
+ on several targets, but exported from those targets devices should
+ have different IDs and SNs. For instance, VDISK dev handler uses this
+ ID to generate T10 vendor specific identifier and SN of the devices.
+
+ - threads - allows to read and set number of global SCST I/O threads.
+ Those threads used with async. dev handlers, for instance, vdisk
+ BLOCKIO or NULLIO.
+
+ - trace_level - allows to enable and disable various tracing
+ facilities. See content of this file for help how to use it.
+
+ - version - read-only attribute, which allows to see version of
+ SCST and enabled optional features.
+
+Each SCST sysfs file (attribute) can contain in the last line mark
+"[key]". It is automatically added mark used to allow scstadmin to see
+which attributes it should save in the config file. You can ignore it.
+
+"Devices" subdirectory contains subdirectories for each SCST devices.
+
+Content of each device's subdirectory is dev handler specific. See
+documentation for your dev handlers for more info about it as well as
+SysfsRules file for more info about common to all dev handlers rules.
+Standard SCST dev handlers have at least the following common entries:
+
+ - exported - subdirectory containing links to all LUNs where this
+ device was exported.
+
+ - handler - if dev handler determined for this device, this link points
+ to it. The handler can be not set for pass-through devices.
+
+ - threads_num - shows and allows to set number of threads in this device's
+ threads pool. If 0 - no threads will be created, and global SCST
+ threads pool will be used. If <0 - creation of the threads pool is
+ prohibited.
+
+ - threads_pool_type - shows and allows to sets threads pool type.
+ Possible values: "per_initiator" and "shared". When the value is
+ "per_initiator" (default), each session from each initiator will use
+ separate dedicated pool of threads. When the value is "shared", all
+ sessions from all initiators will share the same per-device pool of
+ threads. Valid only if threads_num attribute >0.
+
+ - type - SCSI type of this device
+
+See below for more information about other entries of this subdirectory
+of the standard SCST dev handlers.
+
+"Handlers" subdirectory contains subdirectories for each SCST dev
+handler.
+
+Content of each handler's subdirectory is dev handler specific. See
+documentation for your dev handlers for more info about it as well as
+SysfsRules file for more info about common to all dev handlers rules.
+Standard SCST dev handlers have at least the following common entries:
+
+ - mgmt - this entry allows to create virtual devices and their
+ attributes (for virtual devices dev handlers) or assign/unassign real
+ SCSI devices to/from this dev handler (for pass-through dev
+ handlers).
+
+ - pass_through - if exists, it contains 1 and this dev handler is a
+ pass-through dev handler.
+
+ - trace_level - allows to enable and disable various tracing
+ facilities. See content of this file for help how to use it.
+
+ - type - SCSI type of devices served by this dev handler.
+
+See below for more information about other entries of this subdirectory
+of the standard SCST dev handlers.
+
+"Sgv" subdirectory contains statistic information of SCST SGV caches. It
+has the following entries:
+
+ - None, one or more subdirectories for each existing SGV cache.
+
+ - global_stats - file containing global SGV caches statistics.
+
+Each SGV cache's subdirectory has the following item:
+
+ - stats - file containing statistics for this SGV caches.
+
+"Targets" subdirectory contains subdirectories for each SCST target.
+
+Content of each target's subdirectory is target specific. See
+documentation for your target for more info about it as well as
+SysfsRules file for more info about common to all targets rules.
+Every target should have at least the following entries:
+
+ - ini_groups - subdirectory, which contains and allows to define
+ initiator-oriented access control information, see below.
+
+ - luns - subdirectory, which contains list of available LUNs in the
+ target-oriented access control and allows to define it, see below.
+
+ - sessions - subdirectory containing connected to this target sessions.
+
+ - enabled - using this attribute you can enable or disable this target/
+ It allows to finish configuring it before it starts accepting new
+ connections. 0 by default.
+
+ - addr_method - used LUNs addressing method. Possible values:
+ "Peripheral" and "Flat". Most initiators work well with Peripheral
+ addressing method (default), but some (HP-UX, for instance) may
+ require Flat method. This attribute is also available in the
+ initiators security groups, so you can assign the addressing method
+ on per-initiator basis.
+
+ - io_grouping_type - defines how I/O from sessions to this target are
+ grouped together. This I/O grouping is very important for
+ performance. By setting this attribute in a right value, you can
+ considerably increase performance of your setup. This grouping is
+ performed only if you use CFQ I/O scheduler on the target and for
+ devices with threads_num >= 0 and, if threads_num > 0, with
+ threads_pool_type "per_initiator". Possible values:
+ "this_group_only", "never", "auto", or I/O group number >0. When the
+ value is "this_group_only" all I/O from all sessions in this target
+ will be grouped together. When the value is "never", I/O from
+ different sessions will not be grouped together, i.e. all sessions in
+ this target will have separate dedicated I/O groups. When the value
+ is "auto" (default), all I/O from initiators with the same name
+ (iSCSI initiator name, for instance) in all targets will be grouped
+ together with a separate dedicated I/O group for each initiator name.
+ For iSCSI this mode works well, but other transports usually use
+ different initiator names for different sessions, so using such
+ transports in MPIO configurations you should either use value
+ "this_group_only", or an explicit I/O group number. This attribute is
+ also available in the initiators security groups, so you can assign
+ the I/O grouping on per-initiator basis. See below for more info how
+ to use this attribute.
+
+ - rel_tgt_id - allows to read or write SCSI Relative Target Port
+ Identifier attribute. This identifier is used to identify SCSI Target
+ Ports by some SCSI commands, mainly by Persistent Reservations
+ commands. This identifier must be unique among all SCST targets, but
+ for convenience SCST allows disabled targets to have not unique
+ rel_tgt_id. In this case SCST will not allow to enable this target
+ until rel_tgt_id becomes unique. This attribute initialized unique by
+ SCST by default.
+
+A target driver may have also the following entries:
+
+ - "hw_target" - if the target driver supports both hardware and virtual
+ targets (for instance, an FC adapter supporting NPIV, which has
+ hardware targets for its physical ports as well as virtual NPIV
+ targets), this read only attribute for all hardware targets will
+ exist and contain value 1.
+
+Subdirectory "sessions" contains one subdirectory for each connected
+session with name equal to name of the connected initiator.
+
+Each session subdirectory contains the following entries:
+
+ - initiator_name - contains initiator name
+
+ - force_close - optional write-only attribute, which allows to force
+ close this session.
+
+ - active_commands - contains number of active, i.e. not yet or being
+ executed, SCSI commands in this session.
+
+ - commands - contains overall number of SCSI commands in this session.
+
+ - other target driver specific attributes and subdirectories.
+
+See below description of the VDISK's sysfs interface for samples.
+
+Access and devices visibility management (LUN masking) - sysfs interface
+------------------------------------------------------------------------
+
+Access and devices visibility management allows for an initiator or
+group of initiators to see different devices with different LUNs
+with necessary access permissions.
+
+SCST supports two modes of access control:
+
+1. Target-oriented. In this mode you define for each target a default
+set of LUNs, which are accessible to all initiators, connected to that
+target. This is a regular access control mode, which people usually mean
+thinking about access control in general. For instance, in IET this is
+the only supported mode.
+
+2. Initiator-oriented. In this mode you define which LUNs are accessible
+for each initiator. In this mode you should create for each set of one
+or more initiators, which should access to the same set of devices with
+the same LUNs, a separate security group, then add to it devices and
+names of allowed initiator(s).
+
+Both modes can be used simultaneously. In this case the
+initiator-oriented mode has higher priority, than the target-oriented,
+i.e. initiators are at first searched in all defined security groups for
+this target and, if none matches, the default target's set of LUNs is
+used. This set of LUNs might be empty, then the initiator will not see
+any LUNs from the target.
+
+You can at any time find out which set of LUNs each session is assigned
+to by looking where link
+/sys/kernel/scst_tgt/targets/target_driver/target_name/sessions/initiator_name/luns
+points to.
+
+To configure the target-oriented access control SCST provides the
+following interface. Each target's sysfs subdirectory
+(/sys/kernel/scst_tgt/targets/target_driver/target_name) has "luns"
+subdirectory. This subdirectory contains the list of already defined
+target-oriented access control LUNs for this target as well as file
+"mgmt". This file has the following commands, which you can send to it,
+for instance, using "echo" shell command. You can always get a small
+help about supported commands by looking inside this file. "Parameters"
+are one or more param_name=value pairs separated by ';'.
+
+ - "add H:C:I:L lun [parameters]" - adds a pass-through device with
+ host:channel:id:lun with LUN "lun". Optionally, the device could be
+ marked as read only by using parameter "read_only". The recommended
+ way to find out H:C:I:L numbers is use of lsscsi utility.
+
+ - "replace H:C:I:L lun [parameters]" - replaces by pass-through device
+ with host:channel:id:lun existing with LUN "lun" device with
+ generation of INQUIRY DATA HAS CHANGED Unit Attention. If the old
+ device doesn't exist, this command acts as the "add" command.
+ Optionally, the device could be marked as read only by using
+ parameter "read_only". The recommended way to find out H:C:I:L
+ numbers is use of lsscsi utility.
+
+ - "del H:C:I:L" - deletes a pass-through device with host:channel:id:lun
+ The recommended way to find out H:C:I:L numbers is use of lsscsi
+ utility.
+
+ - "add VNAME lun [parameters]" - adds a virtual device with name VNAME
+ with LUN "lun". Optionally, the device could be marked as read only
+ by using parameter "read_only".
+
+ - "replace VNAME lun [parameters]" - replaces by virtual device
+ with name VNAME existing with LUN "lun" device with generation of
+ INQUIRY DATA HAS CHANGED Unit Attention. If the old device doesn't
+ exist, this command acts as the "add" command. Optionally, the device
+ could be marked as read only by using parameter "read_only".
+
+ - "del VNAME" - deletes a virtual device with name VNAME.
+
+ - "clear" - clears the list of devices
+
+To configure the initiator-oriented access control SCST provides the
+following interface. Each target's sysfs subdirectory
+(/sys/kernel/scst_tgt/targets/target_driver/target_name) has "ini_groups"
+subdirectory. This subdirectory contains the list of already defined
+security groups for this target as well as file "mgmt". This file has
+the following commands, which you can send to it, for instance, using
+"echo" shell command. You can always get a small help about supported
+commands by looking inside this file.
+
+ - "create GROUP_NAME" - creates a new security group.
+
+ - "del GROUP_NAME" - deletes a new security group.
+
+Each security group's subdirectory contains 2 subdirectories: initiators
+and luns.
+
+Each "initiators" subdirectory contains list of added to this groups
+initiator as well as as well as file "mgmt". This file has the following
+commands, which you can send to it, for instance, using "echo" shell
+command. You can always get a small help about supported commands by
+looking inside this file.
+
+ - "add INITIATOR_NAME" - adds initiator with name INITIATOR_NAME to the
+ group.
+
+ - "del INITIATOR_NAME" - deletes initiator with name INITIATOR_NAME
+ from the group.
+
+ - "move INITIATOR_NAME DEST_GROUP_NAME" moves initiator with name
+ INITIATOR_NAME from the current group to group with name
+ DEST_GROUP_NAME.
+
+ - "clear" - deletes all initiators from this group.
+
+For "add" and "del" commands INITIATOR_NAME can be a simple DOS-type
+patterns, containing '*' and '?' symbols. '*' means match all any
+symbols, '?' means match only any single symbol. For instance,
+"blah.xxx" will match "bl?h.*".
+
+Each "luns" subdirectory contains the list of already defined LUNs for
+this group as well as file "mgmt". Content of this file as well as list
+of available in it commands is fully identical to the "luns"
+subdirectory of the target-oriented access control.
+
+Examples:
+
+ - echo "create INI" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/mgmt -
+ creates security group INI for target iqn.2006-10.net.vlnb:tgt1.
+
+ - echo "add 2:0:1:0 11" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI/luns/mgmt -
+ adds a pass-through device sitting on host 2, channel 0, ID 1, LUN 0
+ to group with name INI as LUN 11.
+
+ - echo "add disk1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI/luns/mgmt -
+ adds a virtual disk with name disk1 to group with name INI as LUN 0.
+
+ - echo "add 21:*:e0:?b:83:*" >/sys/kernel/scst_tgt/targets/21:00:00:a0:8c:54:52:12/ini_groups/INI/initiators/mgmt -
+ adds a pattern to group with name INI to Fibre Channel target with
+ WWN 21:00:00:a0:8c:54:52:12, which matches WWNs of Fibre Channel
+ initiator ports.
+
+Consider you need to have an iSCSI target with name
+"iqn.2007-05.com.example:storage.disk1.sys1.xyz", which should export
+virtual device "dev1" with LUN 0 and virtual device "dev2" with LUN 1,
+but initiator with name
+"iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" should see only
+virtual device "dev2" read only with LUN 0. To achieve that you should
+do the following commands:
+
+# echo "iqn.2007-05.com.example:storage.disk1.sys1.xyz" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+# echo "add dev1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/luns/mgmt
+# echo "add dev2 1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/luns/mgmt
+# echo "create SPEC_INI" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/ini_groups/mgmt
+# echo "add dev2 0 read_only=1" \
+ >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/ini_groups/SPEC_INI/luns/mgmt
+# echo "iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" \
+ >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example:storage.disk1.sys1.xyz/ini_groups/SPEC_INI/initiators/mgmt
+
+For Fibre Channel or SAS in the above example you should use target's
+and initiator ports WWNs instead of iSCSI names.
+
+It is highly recommended to use scstadmin utility instead of described
+in this section low level interface.
+
+IMPORTANT
+=========
+
+There must be LUN 0 in each set of LUNs, i.e. LUs numeration must not
+start from, e.g., 1. Otherwise you will see no devices on remote
+initiators and SCST core will write into the kernel log message: "tgt_dev
+for LUN 0 not found, command to unexisting LU?"
+
+IMPORTANT
+=========
+
+All the access control must be fully configured BEFORE the corresponding
+target is enabled! When you enable a target, it will immediately start
+accepting new connections, hence creating new sessions, and those new
+sessions will be assigned to security groups according to the
+*currently* configured access control settings. For instance, to
+the default target's set of LUNs, instead of "HOST004" group as you may
+need, because "HOST004" doesn't exist yet. So, one must configure all
+the security groups before new connections from the initiators are
+created, i.e. before the target enabled.
+
+VDISK device handler
+--------------------
+
+VDISK has 4 built-in dev handlers: vdisk_fileio, vdisk_blockio,
+vdisk_nullio and vcdrom. Roots of their sysfs interface are
+/sys/kernel/scst_tgt/handlers/handler_name, e.g. for vdisk_fileio:
+/sys/kernel/scst_tgt/handlers/vdisk_fileio. Each root has the following
+entries:
+
+ - None, one or more links to devices with name equal to names
+ of the corresponding devices.
+
+ - trace_level - allows to enable and disable various tracing
+ facilities. See content of this file for help how to use it.
+
+ - mgmt - main management entry, which allows to add/delete VDISK
+ devices with the corresponding type.
+
+The "mgmt" file has the following commands, which you can send to it,
+for instance, using "echo" shell command. You can always get a small
+help about supported commands by looking inside this file. "Parameters"
+are one or more param_name=value pairs separated by ';'.
+
+ - echo "add_device device_name [parameters]" - adds a virtual device
+ with name device_name and specified parameters (see below)
+
+ - echo "del_device device_name" - deletes a virtual device with name
+ device_name.
+
+Handler vdisk_fileio provides FILEIO mode to create virtual devices.
+This mode uses as backend files and accesses to them using regular
+read()/write() file calls. This allows to use full power of Linux page
+cache. The following parameters possible for vdisk_fileio:
+
+ - filename - specifies path and file name of the backend file. The path
+ must be absolute.
+
+ - blocksize - specifies block size used by this virtual device. The
+ block size must be power of 2 and >= 512 bytes. Default is 512.
+
+ - write_through - disables write back caching. Note, this option
+ has sense only if you also *manually* disable write-back cache in
+ *all* your backstorage devices and make sure it's actually disabled,
+ since many devices are known to lie about this mode to get better
+ benchmark results. Default is 0.
+
+ - read_only - read only. Default is 0.
+
+ - o_direct - disables both read and write caching. This mode isn't
+ currently fully implemented, you should use user space fileio_tgt
+ program in O_DIRECT mode instead (see below).
+
+ - nv_cache - enables "non-volatile cache" mode. In this mode it is
+ assumed that the target has a GOOD UPS with ability to cleanly
+ shutdown target in case of power failure and it is software/hardware
+ bugs free, i.e. all data from the target's cache are guaranteed
+ sooner or later to go to the media. Hence all data synchronization
+ with media operations, like SYNCHRONIZE_CACHE, are ignored in order
+ to bring more performance. Also in this mode target reports to
+ initiators that the corresponding device has write-through cache to
+ disable all write-back cache workarounds used by initiators. Use with
+ extreme caution, since in this mode after a crash of the target
+ journaled file systems don't guarantee the consistency after journal
+ recovery, therefore manual fsck MUST be ran. Note, that since usually
+ the journal barrier protection (see "IMPORTANT" note below) turned
+ off, enabling NV_CACHE could change nothing from data protection
+ point of view, since no data synchronization with media operations
+ will go from the initiator. This option overrides "write_through"
+ option. Disabled by default.
+
+ - removable - with this flag set the device is reported to remote
+ initiators as removable.
+
+Handler vdisk_blockio provides BLOCKIO mode to create virtual devices.
+This mode performs direct block I/O with a block device, bypassing the
+page cache for all operations. This mode works ideally with high-end
+storage HBAs and for applications that either do not need caching
+between application and disk or need the large block throughput. See
+below for more info.
+
+The following parameters possible for vdisk_blockio: filename,
+blocksize, read_only, removable. See vdisk_fileio above for description
+of those parameters.
+
+Handler vdisk_nullio provides NULLIO mode to create virtual devices. In
+this mode no real I/O is done, but success returned to initiators.
+Intended to be used for performance measurements at the same way as
+"*_perf" handlers. The following parameters possible for vdisk_nullio:
+blocksize, read_only, removable. See vdisk_fileio above for description
+of those parameters.
+
+Handler vcdrom allows emulation of a virtual CDROM device using an ISO
+file as backend. It doesn't have any parameters.
+
+For example:
+
+echo "add_device disk1 filename=/disk1; blocksize=4096; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
+
+will create a FILEIO virtual device disk1 with backend file /disk1
+with block size 4K and NV_CACHE enabled.
+
+Each vdisk_fileio's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name:
+
+ - filename - contains path and file name of the backend file.
+
+ - blocksize - contains block size used by this virtual device.
+
+ - write_through - contains status of write back caching of this virtual
+ device.
+
+ - read_only - contains read only status of this virtual device.
+
+ - o_direct - contains O_DIRECT status of this virtual device.
+
+ - nv_cache - contains NV_CACHE status of this virtual device.
+
+ - removable - contains removable status of this virtual device.
+
+ - size_mb - contains size of this virtual device in MB.
+
+ - t10_dev_id - contains and allows to set T10 vendor specific
+ identifier for Device Identification VPD page (0x83) of INQUIRY data.
+ By default VDISK handler always generates t10_dev_id for every new
+ created device at creation time based on the device name and
+ scst_vdisk_ID scst_vdisk.ko module parameter (see below).
+
+ - usn - contains the virtual device's serial number of INQUIRY data. It
+ is created at the device creation time based on the device name and
+ scst_vdisk_ID scst_vdisk.ko module parameter (see below).
+
+ - type - contains SCSI type of this virtual device.
+
+ - resync_size - write only attribute, which makes vdisk_fileio to
+ rescan size of the backend file. It is useful if you changed it, for
+ instance, if you resized it.
+
+For example:
+
+/sys/kernel/scst_tgt/devices/disk1
+|-- blocksize
+|-- exported
+| |-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/0
+| |-- export1 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/INI/luns/0
+| |-- export2 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/0
+| |-- export3 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI1/luns/0
+| |-- export4 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/INI2/luns/0
+|-- filename
+|-- handler -> ../../handlers/vdisk_fileio
+|-- nv_cache
+|-- o_direct
+|-- read_only
+|-- removable
+|-- resync_size
+|-- size_mb
+|-- t10_dev_id
+|-- threads_num
+|-- threads_pool_type
+|-- type
+|-- usn
+`-- write_through
+
+Each vdisk_blockio's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name: blocksize, filename,
+read_only, removable, resync_size, size_mb, t10_dev_id, threads_num,
+threads_pool_type, type, usn. See above description of those parameters.
+
+Each vdisk_nullio's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name: blocksize, read_only,
+removable, size_mb, t10_dev_id, threads_num, threads_pool_type, type,
+usn. See above description of those parameters.
+
+Each vcdrom's device has the following attributes in
+/sys/kernel/scst_tgt/devices/device_name: filename, size_mb,
+t10_dev_id, threads_num, threads_pool_type, type, usn. See above
+description of those parameters. Exception is filename attribute. For
+vcdrom it is writable. Writing to it allows to virtually insert or
+change virtual CD media in the virtual CDROM device. For example:
+
+ - echo "/image.iso" >/sys/kernel/scst_tgt/devices/cdrom/filename - will
+ insert file /image.iso as virtual media to the virtual CDROM cdrom.
+
+ - echo "" >/sys/kernel/scst_tgt/devices/cdrom/filename - will remove
+ "media" from the virtual CDROM cdrom.
+
+Additionally to the sysfs interface VDISK handler has module parameter
+"num_threads", which specifies count of I/O threads for each VDISK's
+device. If you have a workload, which tends to produce rather random
+accesses (e.g. DB-like), you should increase this count to a bigger
+value, like 32. If you have a rather sequential workload, you should
+decrease it to a lower value, like number of CPUs on the target or even
+1. Due to some limitations of Linux I/O subsystem, increasing number of
+I/O threads too much leads to sequential performance drop, especially
+with deadline scheduler, so decreasing it can improve sequential
+performance. The default provides a good compromise between random and
+sequential accesses.
+
+You shouldn't be afraid to have too many VDISK I/O threads if you have
+many VDISK devices. Kernel threads consume very little amount of
+resources (several KBs) and only necessary threads will be used by SCST,
+so the threads will not trash your system.
+
+CAUTION: If you partitioned/formatted your device with block size X, *NEVER*
+======== ever try to export and then mount it (even accidentally) with another
+ block size. Otherwise you can *instantly* damage it pretty
+ badly as well as all your data on it. Messages on initiator
+ like: "attempt to access beyond end of device" is the sign of
+ such damage.
+
+ Moreover, if you want to compare how well different block sizes
+ work for you, you **MUST** EVERY TIME AFTER CHANGING BLOCK SIZE
+ **COMPLETELY** **WIPE OFF** ALL THE DATA FROM THE DEVICE. In
+ other words, THE **WHOLE** DEVICE **MUST** HAVE ONLY **ZEROS**
+ AS THE DATA AFTER YOU SWITCH TO NEW BLOCK SIZE. Switching block
+ sizes isn't like switching between FILEIO and BLOCKIO, after
+ changing block size all previously written with another block
+ size data MUST BE ERASED. Otherwise you will have a full set of
+ very weird behaviors, because blocks addressing will be
+ changed, but initiators in most cases will not have a
+ possibility to detect that old addresses written on the device
+ in, e.g., partition table, don't refer anymore to what they are
+ intended to refer.
+
+IMPORTANT: Some disk and partition table management utilities don't support
+========= block sizes >512 bytes, therefore make sure that your favorite one
+ supports it. Currently only cfdisk is known to work only with
+ 512 bytes blocks, other utilities like fdisk on Linux or
+ standard disk manager on Windows are proved to work well with
+ non-512 bytes blocks. Note, if you export a disk file or
+ device with some block size, different from one, with which
+ it was already partitioned, you could get various weird
+ things like utilities hang up or other unexpected behavior.
+ Hence, to be sure, zero the exported file or device before
+ the first access to it from the remote initiator with another
+ block size. On Window initiator make sure you "Set Signature"
+ in the disk manager on the imported from the target drive
+ before doing any other partitioning on it. After you
+ successfully mounted a file system over non-512 bytes block
+ size device, the block size stops matter, any program will
+ work with files on such file system.
+
+Caching
+-------
+
+By default for performance reasons VDISK FILEIO devices use write back
+caching policy. This is generally safe for modern applications who
+prepared to work in the write back caching environments, so know when to
+flush cache to keep their data consistent and minimize damage caused in
+case of power/hardware/software failures by lost in the cache data.
+
+For instance, journaled file systems flush cache on each meta data
+update, so they survive power/hardware/software failures pretty well.
+Note, Linux IO subsystem guarantees it work reliably only using data
+protection barriers, which, for instance, for Ext3 turned off by default
+(see http://lwn.net/Articles/283161). Some info about barriers from the
+XFS point of view could be found at
+http://oss.sgi.com/projects/xfs/faq.html#wcache. On Linux initiators for
+Ext3 and ReiserFS file systems the barrier protection could be turned on
+using "barrier=1" and "barrier=flush" mount options correspondingly. You
+can check if the barriers turn on or off by looking in /proc/mounts.
+Windows and, AFAIK, other UNIX'es don't need any special explicit
+options and do necessary barrier actions on write-back caching devices
+by default.
+
+But even in case of journaled file systems your unsaved cached data will
+still be lost in case of power/hardware/software failures, so you may
+need to supply your target server with a good UPS with possibility to
+gracefully shutdown your target on power shortage or disable write back
+caching using WRITE_THROUGH flag. Note, on some real-life workloads
+write through caching might perform better, than write back one with the
+barrier protection turned on. Also note that without barriers enabled
+(i.e. by default) Linux doesn't provide a guarantee that after
+sync()/fsync() all written data really hit permanent storage. They can
+be stored in the cache of your backstorage devices and, hence, lost on a
+power failure event. Thus, ever with write-through cache mode, you still
+either need to enable barriers on your backend file system on the target
+(for devices this is, indeed, impossible), or need a good UPS to protect
+yourself from not committed data loss.
+
+To limit this data loss you can use files in /proc/sys/vm to limit
+amount of unflushed data in the system cache.
+
+BLOCKIO VDISK mode
+------------------
+
+This module works best for these types of scenarios:
+
+1) Data that are not aligned to 4K sector boundaries and <4K block sizes
+are used, which is normally found in virtualization environments where
+operating systems start partitions on odd sectors (Windows and it's
+sector 63).
+
+2) Large block data transfers normally found in database loads/dumps and
+streaming media.
+
+3) Advanced relational database systems that perform their own caching
+which prefer or demand direct IO access and, because of the nature of
+their data access, can actually see worse performance with
+non-discriminate caching.
+
+4) Multiple layers of targets were the secondary and above layers need
+to have a consistent view of the primary targets in order to preserve
+data integrity which a page cache backed IO type might not provide
+reliably.
+
+Also it has an advantage over FILEIO that it doesn't copy data between
+the system cache and the commands data buffers, so it saves a
+considerable amount of CPU power and memory bandwidth.
+
+IMPORTANT: Since data in BLOCKIO and FILEIO modes are not consistent between
+========= them, if you try to use a device in both those modes simultaneously,
+ you will almost instantly corrupt your data on that device.
+
+Pass-through mode
+-----------------
+
+In the pass-through mode (i.e. using the pass-through device handlers
+scst_disk, scst_tape, etc) SCSI commands, coming from remote initiators,
+are passed to local SCSI devices on target as is, without any
+modifications.
+
+In the SYSFS interface all real SCSI devices are listed in
+/sys/kernel/scst_tgt/devices in form host:channel:id:lun numbers, for
+instance 1:0:0:0. The recommended way to match those numbers to your
+devices is use of lsscsi utility.
+
+When a pass-through dev handler is loaded it assigns itself to all
+existing SCSI devices of its SCSI type. If you later want to unassign some
+SCSI device from it or assign it to another dev handler you can use the
+following interface.
+
+Each pass-through dev handler has in its root subdirectory
+/sys/kernel/scst_tgt/handlers/handler_name, e.g.
+/sys/kernel/scst_tgt/handlers/dev_disk, "mgmt" file. It allows the
+following commands. They can be sent to it using, e.g., echo command.
+
+ - "assign" - this command assigns SCSI device with
+host:channel:id:lun numbers to this dev handler.
+
+echo "assign 1:0:0:0" >mgmt
+
+will assign SCSI device 1:0:0:0 to this dev handler.
+
+ - "unassign" - this command unassigns SCSI device with
+host:channel:id:lun numbers from this dev handler.
+
+As usually, on read the "mgmt" file returns small help about available
+commands.
+
+As any other hardware, the local SCSI hardware can not handle commands
+with amount of data and/or segments count in scatter-gather array bigger
+some values. Therefore, when using the pass-through mode you should note
+that values for maximum number of segments and maximum amount of
+transferred data for each SCSI command on devices on initiators can not
+be bigger, than corresponding values of the corresponding SCSI devices
+on the target. Otherwise you will see symptoms like small transfers work
+well, but large ones stall and messages like: "Unable to complete
+command due to SG IO count limitation" are printed in the kernel logs.
+
+You can't control from the user space limit of the scatter-gather
+segments, but for block devices usually it is sufficient if you set on
+the initiators /sys/block/DEVICE_NAME/queue/max_sectors_kb in the same
+or lower value as in /sys/block/DEVICE_NAME/queue/max_hw_sectors_kb for
+the corresponding devices on the target.
+
+For not-block devices SCSI commands are usually generated directly by
+applications, so, if you experience large transfers stalls, you should
+check documentation for your application how to limit the transfer
+sizes.
+
+Another way to solve this issue is to build SG entries with more than 1
+page each. See the following patch as an example:
+http://scst.sourceforge.net/sgv_big_order_alloc.diff
+
+User space mode using scst_user dev handler
+-------------------------------------------
+
+User space program fileio_tgt uses interface of scst_user dev handler
+and allows to see how it works in various modes. Fileio_tgt provides
+mostly the same functionality as scst_vdisk handler with the most
+noticeable difference that it supports O_DIRECT mode. O_DIRECT mode is
+basically the same as BLOCKIO, but also supports files, so for some
+loads it could be significantly faster, than the regular FILEIO access.
+All the words about BLOCKIO from above apply to O_DIRECT as well. See
+fileio_tgt's README file for more details.
+
+Performance
+-----------
+
+SCST from the very beginning has been designed and implemented to
+provide the best possible performance. Since there is no "one fit all"
+the best performance configuration for different setups and loads, SCST
+provides extensive set of settings to allow to tune it for the best
+performance in each particular case. You don't have to necessary use
+those settings. If you don't, SCST will do very good job to autotune for
+you, so the resulting performance will, in average, be better
+(sometimes, much better) than with other SCSI targets. But in some cases
+you can by manual tuning improve it even more.
+
+Before doing any performance measurements note that performance results
+are very much dependent from your type of load, so it is crucial that
+you choose access mode (FILEIO, BLOCKIO, O_DIRECT, pass-through), which
+suits your needs the best.
+
+In order to get the maximum performance you should:
+
+1. For SCST:
+
+ - Disable in Makefile CONFIG_SCST_STRICT_SERIALIZING, CONFIG_SCST_EXTRACHECKS,
+ CONFIG_SCST_TRACING, CONFIG_SCST_DEBUG*, CONFIG_SCST_STRICT_SECURITY
+
+ - For pass-through devices enable
+ CONFIG_SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ.
+
+2. For target drivers:
+
+ - Disable in Makefiles CONFIG_SCST_EXTRACHECKS, CONFIG_SCST_TRACING,
+ CONFIG_SCST_DEBUG*
+
+3. For device handlers, including VDISK:
+
+ - Disable in Makefile CONFIG_SCST_TRACING and CONFIG_SCST_DEBUG.
+
+IMPORTANT: Some of the above compilation options in the SCST SVN enabled
+========= by default, i.e. the development version of SCST is optimized
+ for development and bug hunting, not for performance. For it
+ you can set the above options, except
+ CONFIG_SCST_ALLOW_PASSTHROUGH_IO_SUBMIT_IN_SIRQ, in the
+ needed values by command "make debug2perf" performed in
+ trunk/.
+
+4. Make sure you have io_grouping_type option set correctly, especially
+in the following cases:
+
+ - Several initiators share your target's backstorage. It can be a
+ shared LU using some cluster FS, like VMFS, as well as can be
+ different LUs located on the same backstorage (RAID array). For
+ instance, if you have 3 initiators and each of them using its own
+ dedicated FILEIO device file from the same RAID-6 array on the
+ target.
+
+ In this case for the best performance you should have
+ io_grouping_type option set in value "never" in all the LUNs' targets
+ and security groups.
+
+ - Your initiator connected to your target in MPIO mode. In this case for
+ the best performance you should:
+
+ * Either connect all the sessions from the initiator to a single
+ target or security group and have io_grouping_type option set in
+ value "this_group_only" in the target or security group,
+
+ * Or, if it isn't possible to connect all the sessions from the
+ initiator to a single target or security group, assign the same
+ numeric io_grouping_type value for each target/security group this
+ initiator connected to. The exact value itself doesn't matter,
+ important only that all the targets/security groups use the same
+ value.
+
+Don't forget, io_grouping_type makes sense only if you use CFQ I/O
+scheduler on the target and for devices with threads_num >= 0 and, if
+threads_num > 0, with threads_pool_type "per_initiator".
+
+You can check if in your setup io_grouping_type set correctly as well as
+if the "auto" io_grouping_type value works for you by tests like the
+following:
+
+ - For not MPIO case you can run single thread sequential reading, e.g.
+ using buffered dd, from one initiator, then run the same single
+ thread sequential reading from the second initiator in parallel. If
+ io_grouping_type is set correctly the aggregate throughput measured
+ on the target should only slightly decrease as well as all initiators
+ should have nearly equal share of it. If io_grouping_type is not set
+ correctly, the aggregate throughput and/or throughput on any
+ initiator will decrease significantly, in 2 times or even more. For
+ instance, you have 80MB/s single thread sequential reading from the
+ target on any initiator. When then both initiators are reading in
+ parallel you should see on the target aggregate throughput something
+ like 70-75MB/s with correct io_grouping_type and something like
+ 35-40MB/s or 8-10MB/s on any initiator with incorrect.
+
+ - For the MPIO case it's quite easier. With incorrect io_grouping_type
+ you simply won't see performance increase from adding the second
+ session (assuming your hardware is capable to transfer data through
+ both sessions in parallel), or can even see a performance decrease.
+
+5. If you are going to use your target in an VM environment, for
+instance as a shared storage with VMware, make sure all your VMs
+connected to the target via *separate* sessions. For instance, for iSCSI
+it means that each VM has own connection to the target, not all VMs
+connected using a single connection. You can check it using SCST sysfs
+interface. For other transports you should use available facilities,
+like NPIV for Fibre Channel, to make separate sessions for each VM. If
+you miss it, you can greatly loose performance of parallel access to
+your target from different VMs. This isn't related to the case if your
+VMs are using the same shared storage, like with VMFS, for instance. In
+this case all your VM hosts will be connected to the target via separate
+sessions, which is enough.
+
+6. For other target and initiator software parts:
+
+ - Make sure you applied on your kernel all available SCST patches.
+ If for your kernel version this patch doesn't exist, it is strongly
+ recommended to upgrade your kernel to version, for which this patch
+ exists.
+
+ - Don't enable debug/hacking features in the kernel, i.e. use them as
+ they are by default.
+
+ - The default kernel read-ahead and queuing settings are optimized
+ for locally attached disks, therefore they are not optimal if they
+ attached remotely (SCSI target case), which sometimes could lead to
+ unexpectedly low throughput. You should increase read-ahead size to at
+ least 512KB or even more on all initiators and the target.
+
+ You should also limit on all initiators maximum amount of sectors per
+ SCSI command. This tuning is also recommended on targets with large
+ read-ahead values. To do it on Linux, run:
+
+ echo “64” > /sys/block/sdX/queue/max_sectors_kb
+
+ where specify instead of X your imported from target device letter,
+ like 'b', i.e. sdb.
+
+ To increase read-ahead size on Linux, run:
+
+ blockdev --setra N /dev/sdX
+
+ where N is a read-ahead number in 512-byte sectors and X is a device
+ letter like above.
+
+ Note: you need to set read-ahead setting for device sdX again after
+ you changed the maximum amount of sectors per SCSI command for that
+ device.
+
+ Note2: you need to restart SCST after you changed read-ahead settings
+ on the target.
+
+ - You may need to increase amount of requests that OS on initiator
+ sends to the target device. To do it on Linux initiators, run
+
+ echo “64” > /sys/block/sdX/queue/nr_requests
+
+ where X is a device letter like above.
+
+ You may also experiment with other parameters in /sys/block/sdX
+ directory, they also affect performance. If you find the best values,
+ please share them with us.
+
+ - On the target use CFQ IO scheduler. In most cases it has performance
+ advantage over other IO schedulers, sometimes huge (2+ times
+ aggregate throughput increase).
+
+ - It is recommended to turn the kernel preemption off, i.e. set
+ the kernel preemption model to "No Forced Preemption (Server)".
+
+ - Looks like XFS is the best filesystem on the target to store device
+ files, because it allows considerably better linear write throughput,
+ than ext3.
+
+7. For hardware on target.
+
+ - Make sure that your target hardware (e.g. target FC or network card)
+ and underlaying IO hardware (e.g. IO card, like SATA, SCSI or RAID to
+ which your disks connected) don't share the same PCI bus. You can
+ check it using lspci utility. They have to work in parallel, so it
+ will be better if they don't compete for the bus. The problem is not
+ only in the bandwidth, which they have to share, but also in the
+ interaction between cards during that competition. This is very
+ important, because in some cases if target and backend storage
+ controllers share the same PCI bus, it could lead up to 5-10 times
+ less performance, than expected. Moreover, some motherboard (by
+ Supermicro, particularly) have serious stability issues if there are
+ several high speed devices on the same bus working in parallel. If
+ you have no choice, but PCI bus sharing, set in the BIOS PCI latency
+ as low as possible.
+
+8. If you use VDISK IO module in FILEIO mode, NV_CACHE option will
+provide you the best performance. But using it make sure you use a good
+UPS with ability to shutdown the target on the power failure.
+
+Baseline performance numbers you can find in those measurements:
+http://lkml.org/lkml/2009/3/30/283.
+
+IMPORTANT: If you use on initiator some versions of Windows (at least W2K)
+========= you can't get good write performance for VDISK FILEIO devices with
+ default 512 bytes block sizes. You could get about 10% of the
+ expected one. This is because of the partition alignment, which
+ is (simplifying) incompatible with how Linux page cache
+ works, so for each write the corresponding block must be read
+ first. Use 4096 bytes block sizes for VDISK devices and you
+ will have the expected write performance. Actually, any OS on
+ initiators, not only Windows, will benefit from block size
+ max(PAGE_SIZE, BLOCK_SIZE_ON_UNDERLYING_FS), where PAGE_SIZE
+ is the page size, BLOCK_SIZE_ON_UNDERLYING_FS is block size
+ on the underlying FS, on which the device file located, or 0,
+ if a device node is used. Both values are from the target.
+ See also important notes about setting block sizes >512 bytes
+ for VDISK FILEIO devices above.
+
+9. In some cases, for instance working with SSD devices, which consume 100%
+of a single CPU load for data transfers in their internal threads, to
+maximize IOPS it can be needed to assign for those threads dedicated
+CPUs using Linux CPU affinity facilities. No IRQ processing should be
+done on those CPUs. Check that using /proc/interrupts. See taskset
+command and Documentation/IRQ-affinity.txt in your kernel's source tree
+for how to assign IRQ affinity to tasks and IRQs.
+
+The reason for that is that processing of coming commands in SIRQ
+context might be done on the same CPUs as SSD devices' threads doing data
+transfers. As the result, those threads won't receive all the processing
+power of those CPUs and perform worse.
+
+Work if target's backstorage or link is too slow
+------------------------------------------------
+
+Under high I/O load, when your target's backstorage gets overloaded, or
+working over a slow link between initiator and target, when the link
+can't serve all the queued commands on time, you can experience I/O
+stalls or see in the kernel log abort or reset messages.
+
+At first, consider the case of too slow target's backstorage. On some
+seek intensive workloads even fast disks or RAIDs, which able to serve
+continuous data stream on 500+ MB/s speed, can be as slow as 0.3 MB/s.
+Another possible cause for that can be MD/LVM/RAID on your target as in
+http://lkml.org/lkml/2008/2/27/96 (check the whole thread as well).
+
+Thus, in such situations simply processing of one or more commands takes
+too long time, hence initiator decides that they are stuck on the target
+and tries to recover. Particularly, it is known that the default amount
+of simultaneously queued commands (48) is sometimes too high if you do
+intensive writes from VMware on a target disk, which uses LVM in the
+snapshot mode. In this case value like 16 or even 8-10 depending of your
+backstorage speed could be more appropriate.
+
+Unfortunately, currently SCST lacks dynamic I/O flow control, when the
+queue depth on the target is dynamically decreased/increased based on
+how slow/fast the backstorage speed comparing to the target link. So,
+there are 6 possible actions, which you can do to workaround or fix this
+issue in this case:
+
+1. Ignore incoming task management (TM) commands. It's fine if there are
+not too many of them, so average performance isn't hurt and the
+corresponding device isn't getting put offline, i.e. if the backstorage
+isn't too slow.
+
+2. Decrease /sys/block/sdX/device/queue_depth on the initiator in case
+if it's Linux (see below how) or/and SCST_MAX_TGT_DEV_COMMANDS constant
+in scst_priv.h file until you stop seeing incoming TM commands.
+ISCSI-SCST driver also has its own iSCSI specific parameter for that,
+see its README file.
+
+To decrease device queue depth on Linux initiators you can run command:
+
+# echo Y >/sys/block/sdX/device/queue_depth
+
+where Y is the new number of simultaneously queued commands, X - your
+imported device letter, like 'a' for sda device. There are no special
+limitations for Y value, it can be any value from 1 to possible maximum
+(usually, 32), so start from dividing the current value on 2, i.e. set
+16, if /sys/block/sdX/device/queue_depth contains 32.
+
+3. Increase the corresponding timeout on the initiator. For Linux it is
+located in
+/sys/devices/platform/host*/session*/target*:0:0/*:0:0:1/timeout. It can
+be done automatically by an udev rule. For instance, the following
+rule will increase it to 300 seconds:
+
+SUBSYSTEM=="scsi", KERNEL=="[0-9]*:[0-9]*", ACTION=="add", ATTR{type}=="0|7|14", ATTR{timeout}="300"
+
+By default, this timeout is 30 or 60 seconds, depending on your distribution.
+
+4. Try to avoid such seek intensive workloads.
+
+5. Increase speed of the target's backstorage.
+
+6. Implement in SCST dynamic I/O flow control. This will be an ultimate
+solution. See "Dynamic I/O flow control" section on
+http://scst.sourceforge.net/contributing.html page for possible
+implementation idea.
+
+Next, consider the case of too slow link between initiator and target,
+when the initiator tries to simultaneously push N commands to the target
+over it. In this case time to serve those commands, i.e. send or receive
+data for them over the link, can be more, than timeout for any single
+command, hence one or more commands in the tail of the queue can not be
+served on time less than the timeout, so the initiator will decide that
+they are stuck on the target and will try to recover.
+
+To workaround/fix this issue in this case you can use ways 1, 2, 3, 6
+above or (7): increase speed of the link between target and initiator.
+But for some initiators implementations for WRITE commands there might
+be cases when target has no way to detect the issue, so dynamic I/O flow
+control will not be able to help. In those cases you could also need on
+the initiator(s) to either decrease the queue depth (way 2), or increase
+the corresponding timeout (way 3).
+
+Note, that logged messages about QUEUE_FULL status are quite different
+by nature. This is a normal work, just SCSI flow control in action.
+Simply don't enable "mgmt_minor" logging level, or, alternatively, if
+you are confident in the worst case performance of your back-end storage
+or initiator-target link, you can increase SCST_MAX_TGT_DEV_COMMANDS in
+scst_priv.h to 64. Usually initiators don't try to push more commands on
+the target.
+
+Credits
+-------
+
+Thanks to:
+
+ * Mark Buechler <mark.buechler@gmail.com> for a lot of useful
+ suggestions, bug reports and help in debugging.
+
+ * Ming Zhang <mingz@ele.uri.edu> for fixes and comments.
+
+ * Nathaniel Clark <nate@misrule.us> for fixes and comments.
+
+ * Calvin Morrow <calvin.morrow@comcast.net> for testing and useful
+ suggestions.
+
+ * Hu Gang <hugang@soulinfo.com> for the original version of the
+ LSI target driver.
+
+ * Erik Habbinga <erikhabbinga@inphase-tech.com> for fixes and support
+ of the LSI target driver.
+
+ * Ross S. W. Walker <rswwalker@hotmail.com> for the original block IO
+ code and Vu Pham <huongvp@yahoo.com> who updated it for the VDISK dev
+ handler.
+
+ * Michael G. Byrnes <michael.byrnes@hp.com> for fixes.
+
+ * Alessandro Premoli <a.premoli@andxor.it> for fixes
+
+ * Nathan Bullock <nbullock@yottayotta.com> for fixes.
+
+ * Terry Greeniaus <tgreeniaus@yottayotta.com> for fixes.
+
+ * Krzysztof Blaszkowski <kb@sysmikro.com.pl> for many fixes and bug reports.
+
+ * Jianxi Chen <pacers@users.sourceforge.net> for fixing problem with
+ devices >2TB in size
+
+ * Bart Van Assche <bart.vanassche@gmail.com> for a lot of help
+
+ * Daniel Debonzi <debonzi@linux.vnet.ibm.com> for a big part of SCST sysfs tree
+ implementation
+
+Vladislav Bolkhovitin <vst@vlnb.net>, http://scst.sourceforge.net
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 12/12/1/5] SCST sysfs rules
[not found] ` <4BC45841.2090707@vlnb.net>
2010-04-13 13:07 ` [PATCH][RFC 11/12/1/5] SCST core's README Vladislav Bolkhovitin
@ 2010-04-13 13:07 ` Vladislav Bolkhovitin
1 sibling, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:07 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Vu Pham, Bart Van Assche, James Smart, Joe Eykholt, Andy Yan,
linux-driver
This patch contains SCST sysfs rules.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
SysfsRules | 795 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 795 insertions(+)
diff -uprN orig/linux-2.6.33/Documentation/scst/SysfsRules linux-2.6.33/Documentation/scst/SysfsRules
--- orig/linux-2.6.33/Documentation/scst/SysfsRules
+++ linux-2.6.33/Documentation/scst/SysfsRules
@@ -0,0 +1,795 @@
+ SCST SYSFS interface rules
+ ==========================
+
+This file describes SYSFS interface rules, which all SCST target
+drivers, dev handlers and management utilities MUST follow. This allows
+to have a simple, self-documented, target drivers and dev handlers
+independent management interface.
+
+Words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+document are to be interpreted as described in RFC 2119.
+
+In this document "key attribute" means a configuration attribute with
+not default value, which must be configured during the target driver's
+initialization. A key attribute MUST have in the last line keyword
+"[key]". If a default value set to a key attribute, it becomes a regular
+none-key attribute. For instance, iSCSI target has attribute DataDigest.
+Default value for this attribute is "None". It value "CRC32C" is set to
+this attribute, it will become a key attribute. If value "None" is again
+set, this attribute will become back to a none-key attribute.
+
+Each user configurable attribute with a not default value MUST be marked
+as key attribute.
+
+I. Rules for target drivers
+===========================
+
+SCST core for each target driver (struct scst_tgt_template) creates a
+root subdirectory in /sys/kernel/scst_tgt/targets with name
+scst_tgt_template.name (called "target_driver_name" further in this
+document).
+
+For each target (struct scst_tgt) SCST core creates a root subdirectory
+in /sys/kernel/scst_tgt/targets/target_driver_name with name
+scst_tgt.tgt_name (called "target_name" further in this document).
+
+There are 2 type of targets possible: hardware and virtual targets.
+Hardware targets are targets corresponding to real hardware, for
+instance, a Fibre Channel adapter's port. Virtual targets are hardware
+independent targets, which can be dynamically added or removed, for
+instance, an iSCSI target, or NPIV Fibre Channel target.
+
+A target driver supporting virtual targets must support "mgmt" attribute
+and "add_target"/"del_target" commands.
+
+If target driver supports both hardware and virtual targets (for
+instance, an FC adapter supporting NPIV, which has hardware targets for
+its physical ports as well as virtual NPIV targets), it MUST create each
+hardware target with hw_target mark to make SCST core create "hw_target"
+attribute (see below).
+
+Attributes for target drivers
+-----------------------------
+
+A target driver MAY support in its root subdirectory the following
+optional attributes. Target drivers MAY also support there other
+read-only or read-writable attributes.
+
+1. "enabled" - this attribute MUST allow to enable and disable target
+driver as a whole, i.e. if disabled, the target driver MUST NOT accept
+new connections. The goal of this attribute is to allow the target
+driver's initial configuration. For instance, iSCSI target may need to
+have discovery user names and passwords set before it starts serving
+discovery connections.
+
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+On read it MUST return 0, if the target driver is disabled, and 1, if it
+is enabled.
+
+On write it MUST accept '0' character as request to disable and '1' as
+request to enable, but MAY also accept other driver specific commands.
+
+During disabling the target driver MAY close already connected sessions
+in all targets, but this is OPTIONAL.
+
+MUST be 0 by default.
+
+2. "trace_level" - this attribute SHOULD allow to change log level of this
+driver.
+
+This attribute SHOULD have read and write permissions for superuser and be
+read-only for other users.
+
+On read it SHOULD return a help text about available command and log levels.
+
+On write it SHOULD accept commands to change log levels according to the
+help text.
+
+For example:
+
+out_of_mem | minor | pid | line | function | special | mgmt | mgmt_dbg | flow_control | conn
+
+Usage:
+ echo "all|none|default" >trace_level
+ echo "value DEC|0xHEX|0OCT" >trace_level
+ echo "add|del TOKEN" >trace_level
+
+where TOKEN is one of [debug, function, line, pid,
+ entryexit, buff, mem, sg, out_of_mem,
+ special, scsi, mgmt, minor,
+ mgmt_dbg, scsi_serializing,
+ retry, recv_bot, send_bot, recv_top,
+ send_top, d_read, d_write, conn, conn_dbg, iov, pdu, net_page]
+
+3. "version" - this read-only for all attribute SHOULD return version of
+the target driver and some info about its enabled compile time facilities.
+
+For example:
+
+2.0.0
+EXTRACHECKS
+DEBUG
+
+4. "mgmt" - this attribute MUST allow to add and delete targets, if
+virtual targets are supported by this driver, as well as it MAY allow to
+add and delete the target driver's or its targets' attributes.
+
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+On read it MUST return a help string describing available commands and
+parameters.
+
+For example:
+
+Usage: echo "add_target target_name [parameters]" >mgmt
+ echo "del_target target_name" >mgmt
+ echo "add_attribute IncomingUser name password" >mgmt
+ echo "del_attribute IncomingUser name" >mgmt
+ echo "add_attribute OutgoingUser name password" >mgmt
+ echo "del_attribute OutgoingUser name" >mgmt
+ echo "add_target_attribute target_name IncomingUser name password" >mgmt
+ echo "del_target_attribute target_name IncomingUser name" >mgmt
+ echo "add_target_attribute target_name OutgoingUser name password" >mgmt
+ echo "del_target_attribute target_name OutgoingUser name" >mgmt
+
+where parameters are one or more param_name=value pairs separated by ';'
+
+4.1. "add_target" - if supported, this command MUST add new target with
+name "target_name" and specified optional or required parameters. Each
+parameter MUST be in form "parameter=value". All parameters MUST be
+separated by ';' symbol.
+
+All target drivers supporting creation of virtual targets MUST support
+this command.
+
+All target drivers supporting "add_target" command MUST support all
+read-only targets' key attributes as parameters to "add_target" command
+with the attributes' names as parameters' names and the attributes'
+values as parameters' values.
+
+For example:
+
+echo "add_target TARGET1 parameter1=1; parameter2=2" >mgmt
+
+will add target with name "TARGET1" and parameters with names
+"parameter1" and "parameter2" with values 1 and 2 correspondingly.
+
+4.2. "del_target" - if supported, this command MUST delete target with
+name "target_name". If "add_target" command is supported "del_target"
+MUST also be supported.
+
+4.3. "add_attribute" - if supported, this command MUST add a target
+driver's attribute with the specified name and one or more values.
+
+All target drivers supporting run time creation of the target driver's
+key attributes MUST support this command.
+
+For example, for iSCSI target:
+
+echo "add_attribute IncomingUser name password" >mgmt
+
+will add for discovery sessions an incoming user (attribute
+/sys/kernel/scst_tgt/targets/iscsi/IncomingUser) with name "name" and
+password "password".
+
+4.4. "del_attribute" - if supported, this command MUST delete target
+driver's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. For instance, iSCSI target might have several
+incoming users. If not needed, target driver might ignore the values.
+
+If "add_attribute" command is supported "del_attribute" MUST
+also be supported.
+
+4.5. "add_target_attribute" - if supported, this command MUST add new
+attribute for the specified target with the specified name and one or
+more values.
+
+All target drivers supporting run time creation of targets' key
+attributes MUST support this command.
+
+For example:
+
+echo "add_target_attribute iqn.2006-10.net.vlnb:tgt IncomingUser name password" >mgmt
+
+will add for target with name "iqn.2006-10.net.vlnb:tgt" an incoming
+user (attribute
+/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/IncomingUser)
+with name "name" and password "password".
+
+4.6. "del_target_attribute" - if supported, this command MUST delete
+target's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. For instance, iSCSI target might have several
+incoming users. If not needed, target driver might ignore the values.
+
+If "add_target_attribute" command is supported "del_target_attribute"
+MUST also be supported.
+
+Attributes for targets
+----------------------
+
+Each target MAY support in its root subdirectory the following optional
+attributes. Target drivers MAY also support there other read-only or
+read-writable attributes.
+
+1. "enabled" - this attribute MUST allow to enable and disable the
+corresponding target, i.e. if disabled, the target MUST NOT accept new
+connections. The goal of this attribute is to allow the target's initial
+configuration. For instance, each target needs to have its LUNs setup
+before it starts serving initiators. Another example is iSCSI target,
+which may need to have initialized a number of iSCSI parameters before
+it starts accepting new iSCSI connections.
+
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+On read it MUST return 0, if the target is disabled, and 1, if it is
+enabled.
+
+On write it MUST accept '0' character as request to disable and '1' as
+request to enable. Other requests MUST be rejected.
+
+SCST core provides some facilities, which MUST be used to implement this
+attribute.
+
+During disabling the target driver MAY close already connected sessions
+to the target, but this is OPTIONAL.
+
+MUST be 0 by default.
+
+SCST core will automatically create for all targets the following
+attributes:
+
+1. "rel_tgt_id" - allows to read or write SCSI Relative Target Port
+Identifier attribute.
+
+2. "hw_target" - allows to distinguish hardware and virtual targets, if
+the target driver supports both.
+
+To provide OPTIONAL force close session functionality target drivers
+MUST implement it using "force_close" write only session's attribute,
+which on write to it MUST close the corresponding session.
+
+See SCST core's README for more info about those attributes.
+
+II. Rules for dev handlers
+==========================
+
+There are 2 types of dev handlers: parent dev handlers and children dev
+handlers. The children dev handlers depend from the parent dev handlers.
+
+SCST core for each parent dev handler (struct scst_dev_type with
+parent member with value NULL) creates a root subdirectory in
+/sys/kernel/scst_tgt/handlers with name scst_dev_type.name (called
+"dev_handler_name" further in this document).
+
+Parent dev handlers can have one or more subdirectories for children dev
+handlers with names scst_dev_type.name of them.
+
+Only one level of the dev handlers' parent/children hierarchy is
+allowed. Parent dev handlers, which support children dev handlers, MUST
+NOT handle devices and MUST be only placeholders for the children dev
+handlers.
+
+Further in this document children dev handlers or parent dev handlers,
+which don't support children, will be called "end level dev handlers".
+
+Each end level dev handler MUST be either for pass-through, or for
+virtual devices.
+
+End level dev handlers can be recognized by existence of the "mgmt"
+attribute.
+
+For each device (struct scst_device) SCST core creates a root
+subdirectory in /sys/kernel/scst_tgt/devices/device_name with name
+scst_device.virt_name (called "device_name" further in this document).
+
+Attributes for dev handlers
+---------------------------
+
+Each pass-through dev handler MUST have in its root subdirectory
+"pass-through" and "mgmt" attributes and the "mgmt" attribute MUST
+support "assign" and "unassign" commands as described below.
+
+Each virtual devices dev handler MUST have it in its root subdirectory
+"mgmt" attribute, which MUST support "add_device" and "del_device"
+attributes as described below.
+
+Parent dev handlers and end level dev handlers without parents MAY
+support in its root subdirectory the following optional attributes. They
+MAY also support there other read-only or read-writable attributes.
+
+1. "trace_level" - this attribute SHOULD allow to change log level of this
+driver.
+
+This attribute SHOULD have read and write permissions for superuser and be
+read-only for other users.
+
+On read it SHOULD return a help text about available command and log levels.
+
+On write it SHOULD accept commands to change log levels according to the
+help text.
+
+For example:
+
+out_of_mem | minor | pid | line | function | special | mgmt | mgmt_dbg
+
+Usage:
+ echo "all|none|default" >trace_level
+ echo "value DEC|0xHEX|0OCT" >trace_level
+ echo "add|del TOKEN" >trace_level
+
+where TOKEN is one of [debug, function, line, pid,
+ entryexit, buff, mem, sg, out_of_mem,
+ special, scsi, mgmt, minor,
+ mgmt_dbg, scsi_serializing,
+ retry, recv_bot, send_bot, recv_top,
+ send_top]
+
+2. "version" - this read-only for all attribute SHOULD return version of
+the dev handler and some info about its enabled compile time facilities.
+
+For example:
+
+2.0.0
+EXTRACHECKS
+DEBUG
+
+End level dev handlers in their root subdirectories MUST support "mgmt"
+attribute and MAY support other read-only or read-writable attributes.
+This attribute MUST have read and write permissions for superuser and be
+read-only for other users.
+
+Attribute "mgmt" for virtual devices dev handlers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For virtual devices dev handlers "mgmt" attribute MUST allow to add and
+delete devices as well as it MAY allow to add and delete the dev
+handler's or its devices' attributes.
+
+On read it MUST return a help string describing available commands and
+parameters.
+
+For example:
+
+Usage: echo "add_device device_name [parameters]" >mgmt
+ echo "del_device device_name" >mgmt
+ echo "add_attribute AttributeX ValueX" >mgmt
+ echo "del_attribute AttributeX ValueX" >mgmt
+ echo "add_attribute AttributeY ValueY1 ValueY2" >mgmt
+ echo "del_attribute AttributeY" >mgmt
+ echo "add_device_attribute device_name AttributeDX ValueDX" >mgmt
+ echo "del_device_attribute device_name AttributeDX ValueDX" >mgmt
+ echo "add_device_attribute device_name AttributeDY ValueY1 ValueY2" >mgmt
+ echo "del_device_attribute device_name AttributeDY" >mgmt
+
+where parameters are one or more param_name=value pairs separated by ';'
+
+The following parameters available: filename, blocksize, write_through, nv_cache, o_direct, read_only, removable
+
+1. "add_device" - this command MUST add new device with name
+"device_name" and specified optional or required parameters. Each
+parameter MUST be in form "parameter=value". All parameters MUST be
+separated by ';' symbol.
+
+All dev handlers supporting "add_device" command MUST support all
+read-only devices' key attributes as parameters to "add_device" command
+with the attributes' names as parameters' names and the attributes'
+values as parameters' values.
+
+For example:
+
+echo "add_device device1 parameter1=1; parameter2=2" >mgmt
+
+will add device with name "device1" and parameters with names
+"parameter1" and "parameter2" with values 1 and 2 correspondingly.
+
+2. "del_device" - this command MUST delete device with name
+"device_name".
+
+3. "add_attribute" - if supported, this command MUST add a device
+driver's attribute with the specified name and one or more values.
+
+All dev handlers supporting run time creation of the dev handler's
+key attributes MUST support this command.
+
+For example:
+
+echo "add_attribute AttributeX ValueX" >mgmt
+
+will add attribute
+/sys/kernel/scst_tgt/handlers/dev_handler_name/AttributeX with value ValueX.
+
+4. "del_attribute" - if supported, this command MUST delete device
+driver's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. If not needed, dev handler might ignore the
+values.
+
+If "add_attribute" command is supported "del_attribute" MUST also be
+supported.
+
+5. "add_device_attribute" - if supported, this command MUST add new
+attribute for the specified device with the specified name and one or
+more values.
+
+All dev handlers supporting run time creation of devices' key attributes
+MUST support this command.
+
+For example:
+
+echo "add_device_attribute device1 AttributeDX ValueDX" >mgmt
+
+will add for device with name "device1" attribute
+/sys/kernel/scst_tgt/devices/device_name/AttributeDX) with value
+ValueDX.
+
+6. "del_device_attribute" - if supported, this command MUST delete
+device's attribute with the specified name and values. The values MUST
+be specified, because in some cases attributes MAY internally be
+distinguished by values. If not needed, dev handler might ignore the
+values.
+
+If "add_device_attribute" command is supported "del_device_attribute"
+MUST also be supported.
+
+Attribute "mgmt" for pass-through devices dev handlers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For pass-through devices dev handlers "mgmt" attribute MUST allow to
+assign and unassign this dev handler to existing SCSI devices.
+
+On read it MUST return a help string describing available commands and
+parameters.
+
+For example:
+
+Usage: echo "assign H:C:I:L" >mgmt
+ echo "unassign H:C:I:L" >mgmt
+
+1. "assign" - this command MUST assign SCSI device with
+host:channel:id:lun numbers to this dev handler.
+
+All pass-through dev handlers MUST support this command.
+
+For example:
+
+echo "assign 1:0:0:0" >mgmt
+
+will assign SCSI device 1:0:0:0 to this dev handler.
+
+2. "unassign" - this command MUST unassign SCSI device with
+host:channel:id:lun numbers from this dev handler.
+
+SCST core will automatically create for all dev handlers the following
+attributes:
+
+1. "pass_through" - if the dev handler is a handler for pass-through
+devices, SCST core create this attribute. This attribute will have value
+1.
+
+2. "type" - SCSI type of device this dev handler can handle.
+
+See SCST core's README for more info about those attributes.
+
+Attributes for devices
+----------------------
+
+Each device MAY support in its root subdirectory any read-only or
+read-writable attributes.
+
+SCST core will automatically create for all devices the following
+attributes:
+
+1. "type" - SCSI type of this device
+
+See SCST core's README for more info about those attributes.
+
+III. Rules for management utilities
+===================================
+
+Rules summary
+-------------
+
+A management utility (scstadmin) SHOULD NOT keep any knowledge specific
+to any device, dev handler, target or target driver. It SHOULD only know
+the common SCST SYSFS rules, which all dev handlers and target drivers
+MUST follow. Namely:
+
+Common rules:
+~~~~~~~~~~~~~
+
+1. All key attributes MUST be marked by mark "[key]" in the last line of
+the attribute.
+
+2. All not key attributes don't matter and SHOULD be ignored.
+
+For target drivers and targets:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. If target driver supports adding new targets, it MUST have "mgmt"
+attribute, which MUST support "add_target" and "del_target" commands as
+specified above.
+
+2. If target driver supports run time adding new key attributes, it MUST
+have "mgmt" attribute, which MUST support "add_attribute" and
+"del_attribute" commands as specified above.
+
+3. If target driver supports both hardware and virtual targets, all its
+hardware targets MUST have "hw_target" attribute with value 1.
+
+4. If target has read-only key attributes, the add_target command MUST
+support them as parameters.
+
+5. If target supports run time adding new key attributes, the target
+driver MUST have "mgmt" attribute, which MUST support
+"add_target_attribute" and "del_target_attribute" commands as specified
+above.
+
+6. Both target drivers and targets MAY support "enable" attribute. If
+supported, after configuring the corresponding target driver or target
+"1" MUST be written to this attribute in the following order: at first,
+for all targets of the target driver, then for the target driver.
+
+For devices and dev handlers:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Each dev handler in its root subdirectory MUST have "mgmt" attribute.
+
+2. All pass-through dev handlers MUST have "pass_through" attribute.
+
+3. Each virtual devices dev handler MUST support "add_device" and
+"del_device" commands to the "mgmt" attribute as specified above.
+
+4. Each pass-through dev handler MUST support "assign" and "unassign"
+commands to the "mgmt" attribute as specified above.
+
+5. If dev handler driver supports run time adding new key attributes, it
+MUST support "add_attribute" and "del_attribute" commands to the "mgmt"
+attribute as specified above.
+
+6. All device handlers have links in the root subdirectory pointing to
+their devices.
+
+7. If device has read-only key attributes, the "add_device" command MUST
+support them as parameters.
+
+8. If device supports run time adding new key attributes, its dev
+handler MUST support "add_device_attribute" and "del_device_attribute"
+commands to the "mgmt" attribute as specified above.
+
+9. Each device has "handler" link to its dev handler's root
+subdirectory.
+
+Algorithm to convert current SCST configuration to config file
+--------------------------------------------------------------
+
+A management utility SHOULD use the following algorithm when converting
+current SCST configuration to a config file:
+
+1. Scan all attributes in /sys/kernel/scst_tgt (not requirsive) and store
+all found key attributes.
+
+2. Scan all subdirectories of /sys/kernel/scst_tgt/handlers. Each
+subdirectory with "mgmt" attribute is a root subdirectory of a dev
+handler with name the name of the subdirectory. For each found dev
+handler do the following:
+
+2.1. Store the dev handler's name. Store also its path to the root
+subdirectory, if it isn't default (/sys/kernel/scst_tgt/handlers/handler_name).
+
+2.2. Store all dev handler's key attributes.
+
+2.3. Go through all links in the root subdirectory pointing to
+/sys/kernel/scst_tgt/devices and for each device:
+
+2.3.1. For virtual devices dev handlers:
+
+2.3.1.1. Store the name of the device.
+
+2.3.1.2. Store all key attributes. Mark all read only key attributes
+during storing, they will be parameters for the device's creation.
+
+2.3.2. For pass-through devices dev handlers:
+
+2.3.2.1. Store the H:C:I:L name of the device. Optionally, instead of
+the name unique T10 vendor device ID found using command:
+
+sg_inq -p 0x83 /dev/sdX
+
+can be stored. It will allow to reliably find out this device if on the
+next reboot it will have another host:channel:id:lin numbers. The sdX
+device can be found as the last letters after ':' in
+/sys/kernel/scst_tgt/devices/H:C:I:L/scsi_device/device/block:sdX.
+
+3. Go through all subdirectories in /sys/kernel/scst_tgt/targets. For
+each target driver:
+
+3.1. Store the name of the target driver.
+
+3.2. Store all its key attributes.
+
+3.3. Go through all target's subdirectories. For each target:
+
+3.3.1. Store the name of the target.
+
+3.3.2. Mark if the target is hardware or virtual target. The target is a
+hardware target if it has "hw_target" attribute or its target driver
+doesn't have "mgmt" attribute.
+
+3.3.3. Store all key attributes. Mark all read only key attributes
+during storing, they will be parameters for the target's creation.
+
+3.3.4. Scan all "luns" subdirectory and store:
+
+ - LUN.
+
+ - LU's device name.
+
+ - Key attributes.
+
+3.3.5. Scan all "ini_groups" subdirectories. For each group store the following:
+
+ - The group's name.
+
+ - The group's LUNs (the same info as for 3.3.4).
+
+ - The group's initiators.
+
+3.3.6. Store value of "enabled" attribute, if it exists.
+
+3.4. Store value of "enabled" attribute, if it exists.
+
+Algorithm to initialize SCST from config file
+---------------------------------------------
+
+A management utility SHOULD use the following algorithm when doing
+initial SCST configuration from a config file. All necessary kernel
+modules and user space programs supposed to be already loaded, hence all
+dev handlers' entries in /sys/kernel/scst_tgt/handlers as well as all
+entries for hardware targets already created.
+
+1. Set stored values for all stored global (/sys/kernel/scst_tgt)
+attributes.
+
+2. For each dev driver:
+
+2.1. Set stored values for all already existing stored attributes.
+
+2.2. Create not existing stored attributes using "add_attribute" command.
+
+2.3. For virtual devices dev handlers for each stored device:
+
+2.3.1. Create the device using "add_device" command using marked read
+only attributes as parameters.
+
+2.3.2. Set stored values for all already existing stored attributes.
+
+2.3.3. Create not existing stored attributes using
+"add_device_attribute" command.
+
+2.4. For pass-through dev handlers for each stores device:
+
+2.4.1. Assign the corresponding pass-through device to this dev handler
+using "assign" command, if it isn't already auto assigned to it.
+
+3. For each target driver:
+
+3.1. Set stored values for all already existing stored attributes.
+
+3.2. Create not existing stored attributes using "add_attribute" command.
+
+3.3. For each target:
+
+3.3.1. For virtual targets:
+
+3.3.1.1. Create the target using "add_target" command using marked read
+only attributes as parameters.
+
+3.3.1.2. Set stored values for all already existing stored attributes.
+
+3.3.1.3. Create not existing stored attributes using
+"add_target_attribute" command.
+
+3.3.2. For hardware targets for each target:
+
+3.3.2.1. Set stored values for all already existing stored attributes.
+
+3.3.2.2. Create not existing stored attributes using
+"add_target_attribute" command.
+
+3.3.3. Setup LUNs
+
+3.3.4. Setup ini_groups, their LUNs and initiators' names.
+
+3.3.5. If this target supports enabling, enable it.
+
+3.4. If this target driver supports enabling, enable it.
+
+Algorithm to apply changes in config file to currently running SCST
+-------------------------------------------------------------------
+
+A management utility SHOULD use the following algorithm when applying
+changes in config file to currently running SCST.
+
+Not all changes can be applied on enabled targets or enabled target
+drivers. From other side, for some target drivers enabling/disabling is
+a very long and disruptive operation, which should be performed as rare
+as possible. Thus, the management utility SHOULD support additional
+option, which, if set, will make it to disable all affected targets
+before doing any change with them.
+
+1. Scan all attributes in /sys/kernel/scst_tgt (not requirsive) and
+compare stored and actual key attributes. Apply all changes.
+
+2. Scan all subdirectories of /sys/kernel/scst_tgt/handlers. Each
+subdirectory with "mgmt" attribute is a root subdirectory of a dev
+handler with name the name of the subdirectory. For each found dev
+handler do the following:
+
+2.1. Compare stored and actual key attributes. Apply all changes. Create
+new attributes using "add_attribute" commands and delete not needed any
+more attributes using "del_attribute" command.
+
+2.2. Compare existing devices (links in the root subdirectory pointing
+to /sys/kernel/scst_tgt/devices) and stored devices in the config file.
+Delete all not needed devices and create new devices.
+
+2.3. For all existing devices:
+
+2.3.1. Compare stored and actual key attributes. Apply all changes.
+Create new attributes using "add_device_attribute" commands and delete
+not needed any more attributes using "del_device_attribute" command.
+
+2.3.2. If any read only key attribute for virtual device should be
+changed, delete the devices and recreate it.
+
+2.3.3. For pass-through devices dev handlers reassign handlers if
+necessary.
+
+3. Go through all subdirectories in /sys/kernel/scst_tgt/targets. For
+each target driver:
+
+3.1. If this target driver should be disabled, disable it.
+
+3.2. Compare stored and actual key attributes. Apply all changes. Create
+new attributes using "add_attribute" commands and delete not needed any
+more attributes using "del_attribute" command.
+
+3.3. Go through all target's subdirectories. Compare existing and stored
+targets. Delete all not needed targets and create new targets.
+
+3.4. For all existing targets:
+
+3.4.1. If this target should be disabled, disable it.
+
+3.4.2. Compare stored and actual key attributes. Apply all changes.
+Create new attributes using "add_target_attribute" commands and delete
+not needed any more attributes using "del_target_attribute" command.
+
+3.4.3. If any read only key attribute for virtual target should be
+changed, delete the target and recreate it.
+
+3.4.4. Scan all "luns" subdirectory and apply necessary changes, using
+"replace" commands to replace one LUN by another, if needed.
+
+3.4.5. Scan all "ini_groups" subdirectories and apply necessary changes,
+using "replace" commands to replace one LUN by another and "move"
+command to move initiator from one group to another, if needed. It MUST
+be done in the following order:
+
+ - Necessary initiators deleted, if they aren't going to be moved
+
+ - LUNs updated
+
+ - Necessary initiators added or moved
+
+3.4.6. If this target should be enabled, enable it.
+
+3.5. If this target driver should be enabled, enable it.
+
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 0/4/4/5] iSCSI-SCST
[not found] <4BC44A49.7070307@vlnb.net>
` (2 preceding siblings ...)
[not found] ` <4BC45841.2090707@vlnb.net>
@ 2010-04-13 13:09 ` Vladislav Bolkhovitin
[not found] ` <4BC45AFA.7060403@vlnb.net>
` (2 subsequent siblings)
6 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:09 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Bart Van Assche
This patchset contains iSCSI-SCST iSCSI target together with documentation.
Vlad
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 1/4/4/5] iSCSI-SCST's Makefile and Kconfig
[not found] ` <4BC45AFA.7060403@vlnb.net>
@ 2010-04-13 13:10 ` Vladislav Bolkhovitin
0 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:10 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Bart Van Assche
This patch contains iSCSI-SCST's Makefile and Kconfig.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
Kconfig | 25 +++++++++++++++++++++++++
Makefile | 6 ++++++
2 files changed, 31 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/Makefile linux-2.6.33/drivers/scst/iscsi-scst/Makefile
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/Makefile
+++ linux-2.6.33/drivers/scst/iscsi-scst/Makefile
@@ -0,0 +1,6 @@
+EXTRA_CFLAGS += -Iinclude/scst
+
+iscsi-scst-y := iscsi.o nthread.o config.o digest.o \
+ conn.o session.o target.o event.o param.o
+
+obj-$(CONFIG_SCST_ISCSI) += iscsi-scst.o
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/Kconfig linux-2.6.33/drivers/scst/iscsi-scst/Kconfig
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/Kconfig
+++ linux-2.6.33/drivers/scst/iscsi-scst/Kconfig
@@ -0,0 +1,25 @@
+config SCST_ISCSI
+ tristate "ISCSI Target"
+ depends on SCST && INET
+ default SCST
+ help
+ ISCSI target driver for SCST framework. The iSCSI protocol has been
+ defined in RFC 3720. To use it you should download from
+ http://scst.sourceforge.net the user space part of it.
+
+config SCST_ISCSI_DEBUG_DIGEST_FAILURES
+ bool "Simulate iSCSI digest failures"
+ depends on SCST_ISCSI
+ help
+ Simulates iSCSI digest failures in random places. Even when iSCSI
+ traffic is sent over a TCP connection, the 16-bit TCP checksum is too
+ weak for the requirements of a storage protocol. Furthermore, there
+ are also instances where the TCP checksum does not protect iSCSI
+ data, as when data is corrupted while being transferred on a PCI bus
+ or while in memory. The iSCSI protocol therefore defines a 32-bit CRC
+ digest on iSCSI packets in order to detect data corruption on an
+ end-to-end basis. CRCs can be used on iSCSI PDU headers and/or data.
+ Enabling this option allows to test digest failure recovery in the
+ iSCSI initiator that is talking to SCST.
+
+ If unsure, say "N".
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 2/4/4/5] iSCSI-SCST's header files
[not found] ` <4BC45E87.6000901@vlnb.net>
@ 2010-04-13 13:10 ` Vladislav Bolkhovitin
0 siblings, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:10 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Bart Van Assche
This patch contains iSCSI-SCST's header files.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
drivers/scst/iscsi-scst/digest.h | 31 +
drivers/scst/iscsi-scst/iscsi.h | 709 ++++++++++++++++++++++++++++++++++++
drivers/scst/iscsi-scst/iscsi_dbg.h | 60 +++
drivers/scst/iscsi-scst/iscsi_hdr.h | 519 ++++++++++++++++++++++++++
include/scst/iscsi_scst.h | 207 ++++++++++
include/scst/iscsi_scst_itf_ver.h | 2
include/scst/iscsi_scst_ver.h | 18
diff -uprN orig/linux-2.6.33/include/scst/iscsi_scst.h linux-2.6.33/include/scst/iscsi_scst.h
--- orig/linux-2.6.33/include/scst/iscsi_scst.h
+++ linux-2.6.33/include/scst/iscsi_scst.h
@@ -0,0 +1,207 @@
+/*
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _ISCSI_SCST_U_H
+#define _ISCSI_SCST_U_H
+
+#ifndef __KERNEL__
+#include <sys/uio.h>
+#endif
+
+#include "iscsi_scst_ver.h"
+#include "iscsi_scst_itf_ver.h"
+
+/* The maximum length of 223 bytes in the RFC. */
+#define ISCSI_NAME_LEN 256
+
+#define ISCSI_LISTEN_PORT 3260
+
+#define SCSI_ID_LEN 24
+
+#ifndef aligned_u64
+#define aligned_u64 uint64_t __attribute__((aligned(8)))
+#endif
+
+#define ISCSI_MAX_ATTR_NAME_LEN 50
+#define ISCSI_MAX_ATTR_VALUE_LEN 512
+
+enum {
+ key_initial_r2t,
+ key_immediate_data,
+ key_max_connections,
+ key_max_recv_data_length,
+ key_max_xmit_data_length,
+ key_max_burst_length,
+ key_first_burst_length,
+ key_default_wait_time,
+ key_default_retain_time,
+ key_max_outstanding_r2t,
+ key_data_pdu_inorder,
+ key_data_sequence_inorder,
+ key_error_recovery_level,
+ key_header_digest,
+ key_data_digest,
+ key_ofmarker,
+ key_ifmarker,
+ key_ofmarkint,
+ key_ifmarkint,
+ session_key_last,
+};
+
+enum {
+ key_queued_cmnds,
+ key_rsp_timeout,
+ key_nop_in_interval,
+ key_max_sessions,
+ target_key_last,
+};
+
+enum {
+ key_session,
+ key_target,
+};
+
+struct iscsi_kern_target_info {
+ u32 tid;
+ u32 cookie;
+ char name[ISCSI_NAME_LEN];
+ u32 attrs_num;
+ aligned_u64 attrs_ptr;
+};
+
+struct iscsi_kern_session_info {
+ u32 tid;
+ aligned_u64 sid;
+ char initiator_name[ISCSI_NAME_LEN];
+ u32 exp_cmd_sn;
+ s32 session_params[session_key_last];
+ s32 target_params[target_key_last];
+};
+
+#define DIGEST_ALL (DIGEST_NONE | DIGEST_CRC32C)
+#define DIGEST_NONE (1 << 0)
+#define DIGEST_CRC32C (1 << 1)
+
+struct iscsi_kern_conn_info {
+ u32 tid;
+ aligned_u64 sid;
+
+ u32 cid;
+ u32 stat_sn;
+ u32 exp_stat_sn;
+ int fd;
+};
+
+struct iscsi_kern_attr {
+ u32 mode;
+ char name[ISCSI_MAX_ATTR_NAME_LEN];
+};
+
+struct iscsi_kern_mgmt_cmd_res_info {
+ u32 tid;
+ u32 cookie;
+ u32 req_cmd;
+ u32 result;
+ char value[ISCSI_MAX_ATTR_VALUE_LEN];
+};
+
+struct iscsi_kern_params_info {
+ u32 tid;
+ aligned_u64 sid;
+
+ u32 params_type;
+ u32 partial;
+
+ s32 session_params[session_key_last];
+ s32 target_params[target_key_last];
+};
+
+enum iscsi_kern_event_code {
+ E_ADD_TARGET,
+ E_DEL_TARGET,
+ E_MGMT_CMD,
+ E_ENABLE_TARGET,
+ E_DISABLE_TARGET,
+ E_GET_ATTR_VALUE,
+ E_SET_ATTR_VALUE,
+ E_CONN_CLOSE,
+};
+
+struct iscsi_kern_event {
+ u32 tid;
+ aligned_u64 sid;
+ u32 cid;
+ u32 code;
+ u32 cookie;
+ char target_name[ISCSI_NAME_LEN];
+ u32 param1_size;
+ u32 param2_size;
+};
+
+struct iscsi_kern_register_info {
+ union {
+ aligned_u64 version;
+ struct {
+ int max_data_seg_len;
+ int max_queued_cmds;
+ };
+ };
+};
+
+struct iscsi_kern_attr_info {
+ u32 tid;
+ u32 cookie;
+ struct iscsi_kern_attr attr;
+};
+
+#define DEFAULT_NR_QUEUED_CMNDS 32
+#define MIN_NR_QUEUED_CMNDS 1
+#define MAX_NR_QUEUED_CMNDS 256
+
+#define DEFAULT_RSP_TIMEOUT 30
+#define MIN_RSP_TIMEOUT 10
+#define MAX_RSP_TIMEOUT 65535
+
+#define DEFAULT_NOP_IN_INTERVAL 30
+#define MIN_NOP_IN_INTERVAL 0
+#define MAX_NOP_IN_INTERVAL 65535
+
+#define NETLINK_ISCSI_SCST 25
+
+#define REGISTER_USERD _IOWR('s', 0, struct iscsi_kern_register_info)
+#define ADD_TARGET _IOW('s', 1, struct iscsi_kern_target_info)
+#define DEL_TARGET _IOW('s', 2, struct iscsi_kern_target_info)
+#define ADD_SESSION _IOW('s', 3, struct iscsi_kern_session_info)
+#define DEL_SESSION _IOW('s', 4, struct iscsi_kern_session_info)
+#define ADD_CONN _IOW('s', 5, struct iscsi_kern_conn_info)
+#define DEL_CONN _IOW('s', 6, struct iscsi_kern_conn_info)
+#define ISCSI_PARAM_SET _IOW('s', 7, struct iscsi_kern_params_info)
+#define ISCSI_PARAM_GET _IOWR('s', 8, struct iscsi_kern_params_info)
+
+#define ISCSI_ATTR_ADD _IOW('s', 9, struct iscsi_kern_attr_info)
+#define ISCSI_ATTR_DEL _IOW('s', 10, struct iscsi_kern_attr_info)
+#define MGMT_CMD_CALLBACK _IOW('s', 11, struct iscsi_kern_mgmt_cmd_res_info)
+
+static inline int iscsi_is_key_internal(int key)
+{
+ switch (key) {
+ case key_max_xmit_data_length:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+#endif
diff -uprN orig/linux-2.6.33/include/scst/iscsi_scst_ver.h linux-2.6.33/include/scst/iscsi_scst_ver.h
--- orig/linux-2.6.33/include/scst/iscsi_scst_ver.h
+++ linux-2.6.33/include/scst/iscsi_scst_ver.h
@@ -0,0 +1,18 @@
+/*
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#define ISCSI_VERSION_STRING_SUFFIX
+
+#define ISCSI_VERSION_STRING "2.0.0/0.4.17r214" ISCSI_VERSION_STRING_SUFFIX
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/digest.h linux-2.6.33/drivers/scst/iscsi-scst/digest.h
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/digest.h
+++ linux-2.6.33/drivers/scst/iscsi-scst/digest.h
@@ -0,0 +1,31 @@
+/*
+ * iSCSI digest handling.
+ *
+ * Copyright (C) 2004 Xiranet Communications GmbH <arne.redlich@xiranet.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ISCSI_DIGEST_H__
+#define __ISCSI_DIGEST_H__
+
+extern void digest_alg_available(int *val);
+
+extern int digest_init(struct iscsi_conn *conn);
+
+extern int digest_rx_header(struct iscsi_cmnd *cmnd);
+extern int digest_rx_data(struct iscsi_cmnd *cmnd);
+
+extern void digest_tx_header(struct iscsi_cmnd *cmnd);
+extern void digest_tx_data(struct iscsi_cmnd *cmnd);
+
+#endif /* __ISCSI_DIGEST_H__ */
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi_dbg.h linux-2.6.33/drivers/scst/iscsi-scst/iscsi_dbg.h
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi_dbg.h
+++ linux-2.6.33/drivers/scst/iscsi-scst/iscsi_dbg.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef ISCSI_DBG_H
+#define ISCSI_DBG_H
+
+#define LOG_PREFIX "iscsi-scst"
+
+#include <scst_debug.h>
+
+#define TRACE_D_WRITE 0x80000000
+#define TRACE_CONN_OC 0x40000000
+#define TRACE_D_IOV 0x20000000
+#define TRACE_D_DUMP_PDU 0x10000000
+#define TRACE_NET_PG 0x08000000
+#define TRACE_CONN_OC_DBG 0x04000000
+
+#ifdef CONFIG_SCST_DEBUG
+#define ISCSI_DEFAULT_LOG_FLAGS (TRACE_FUNCTION | TRACE_LINE | TRACE_PID | \
+ TRACE_OUT_OF_MEM | TRACE_MGMT | TRACE_MGMT_DEBUG | \
+ TRACE_MINOR | TRACE_SPECIAL | TRACE_CONN_OC)
+#else
+#define ISCSI_DEFAULT_LOG_FLAGS (TRACE_OUT_OF_MEM | TRACE_MGMT | \
+ TRACE_SPECIAL)
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+struct iscsi_pdu;
+struct iscsi_cmnd;
+extern void iscsi_dump_pdu(struct iscsi_pdu *pdu);
+extern unsigned long iscsi_get_flow_ctrl_or_mgmt_dbg_log_flag(
+ struct iscsi_cmnd *cmnd);
+#else
+#define iscsi_dump_pdu(x) do {} while (0)
+#define iscsi_get_flow_ctrl_or_mgmt_dbg_log_flag(x) do {} while (0)
+#endif
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+extern unsigned long iscsi_trace_flag;
+#define trace_flag iscsi_trace_flag
+#endif
+
+#define TRACE_CONN_CLOSE(args...) TRACE_DBG_FLAG(TRACE_DEBUG|TRACE_CONN_OC, args)
+#define TRACE_CONN_CLOSE_DBG(args...) TRACE(TRACE_CONN_OC_DBG, args)
+#define TRACE_NET_PAGE(args...) TRACE_DBG_FLAG(TRACE_NET_PG, args)
+#define TRACE_WRITE(args...) TRACE_DBG_FLAG(TRACE_DEBUG|TRACE_D_WRITE, args)
+
+#endif
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi.h linux-2.6.33/drivers/scst/iscsi-scst/iscsi.h
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi.h
+++ linux-2.6.33/drivers/scst/iscsi-scst/iscsi.h
@@ -0,0 +1,709 @@
+/*
+ * Copyright (C) 2002 - 2003 Ardis Technolgies <roman@ardistech.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ISCSI_H__
+#define __ISCSI_H__
+
+#include <linux/pagemap.h>
+#include <linux/mm.h>
+#include <linux/net.h>
+#include <net/sock.h>
+
+#include <scst.h>
+
+#include "iscsi_hdr.h"
+#include "iscsi_scst.h"
+
+#include "iscsi_dbg.h"
+
+#define iscsi_sense_crc_error ABORTED_COMMAND, 0x47, 0x05
+#define iscsi_sense_unexpected_unsolicited_data ABORTED_COMMAND, 0x0C, 0x0C
+#define iscsi_sense_incorrect_amount_of_data ABORTED_COMMAND, 0x0C, 0x0D
+
+struct iscsi_sess_params {
+ int initial_r2t;
+ int immediate_data;
+ int max_connections;
+ unsigned int max_recv_data_length;
+ unsigned int max_xmit_data_length;
+ unsigned int max_burst_length;
+ unsigned int first_burst_length;
+ int default_wait_time;
+ int default_retain_time;
+ unsigned int max_outstanding_r2t;
+ int data_pdu_inorder;
+ int data_sequence_inorder;
+ int error_recovery_level;
+ int header_digest;
+ int data_digest;
+ int ofmarker;
+ int ifmarker;
+ int ofmarkint;
+ int ifmarkint;
+};
+
+struct iscsi_tgt_params {
+ int queued_cmnds;
+ unsigned int rsp_timeout;
+ unsigned int nop_in_interval;
+};
+
+struct network_thread_info {
+ struct task_struct *task;
+ unsigned int ready;
+};
+
+struct iscsi_target;
+struct iscsi_cmnd;
+
+struct iscsi_attr {
+ struct list_head attrs_list_entry;
+ struct kobj_attribute attr;
+ struct iscsi_target *target;
+ const char *name;
+};
+
+struct iscsi_target {
+ struct scst_tgt *scst_tgt;
+
+ struct mutex target_mutex;
+
+ struct list_head session_list; /* protected by target_mutex */
+
+ struct list_head target_list_entry;
+ u32 tid;
+
+ /* Protected by iscsi_sysfs_mutex */
+ unsigned int tgt_enabled:1;
+
+ /* Protected by target_mutex */
+ struct list_head attrs_list;
+
+ char name[ISCSI_NAME_LEN];
+};
+
+#define ISCSI_HASH_ORDER 8
+#define cmnd_hashfn(itt) hash_long((itt), ISCSI_HASH_ORDER)
+
+struct iscsi_session {
+ struct iscsi_target *target;
+ struct scst_session *scst_sess;
+
+ struct list_head pending_list; /* protected by sn_lock */
+
+ /* Unprotected, since accessed only from a single read thread */
+ u32 next_ttt;
+
+ /* Read only, if there are connection(s) */
+ struct iscsi_tgt_params tgt_params;
+ atomic_t active_cmds;
+
+ spinlock_t sn_lock;
+ u32 exp_cmd_sn; /* protected by sn_lock */
+
+ /* All 3 protected by sn_lock */
+ int tm_active;
+ u32 tm_sn;
+ struct iscsi_cmnd *tm_rsp;
+
+ /* Read only, if there are connection(s) */
+ struct iscsi_sess_params sess_params;
+
+ /*
+ * In some corner cases commands can be deleted from the hash
+ * not from the corresponding read thread. So, let's simplify
+ * errors recovery and have this lock.
+ */
+ spinlock_t cmnd_data_wait_hash_lock;
+ struct list_head cmnd_data_wait_hash[1 << ISCSI_HASH_ORDER];
+
+ struct list_head conn_list; /* protected by target_mutex */
+
+ struct list_head session_list_entry;
+
+ /* All protected by target_mutex, where necessary */
+ struct iscsi_session *sess_reinst_successor;
+ unsigned int sess_reinstating:1;
+ unsigned int sess_shutting_down:1;
+
+ /* All don't need any protection */
+ char *initiator_name;
+ u64 sid;
+};
+
+#define ISCSI_CONN_IOV_MAX (PAGE_SIZE/sizeof(struct iovec))
+
+#define ISCSI_CONN_RD_STATE_IDLE 0
+#define ISCSI_CONN_RD_STATE_IN_LIST 1
+#define ISCSI_CONN_RD_STATE_PROCESSING 2
+
+#define ISCSI_CONN_WR_STATE_IDLE 0
+#define ISCSI_CONN_WR_STATE_IN_LIST 1
+#define ISCSI_CONN_WR_STATE_SPACE_WAIT 2
+#define ISCSI_CONN_WR_STATE_PROCESSING 3
+
+struct iscsi_conn {
+ struct iscsi_session *session; /* owning session */
+
+ /* Both protected by session->sn_lock */
+ u32 stat_sn;
+ u32 exp_stat_sn;
+
+#define ISCSI_CONN_REINSTATING 1
+#define ISCSI_CONN_SHUTTINGDOWN 2
+ unsigned long conn_aflags;
+
+ spinlock_t cmd_list_lock; /* BH lock */
+
+ /* Protected by cmd_list_lock */
+ struct list_head cmd_list; /* in/outcoming pdus */
+
+ atomic_t conn_ref_cnt;
+
+ spinlock_t write_list_lock;
+ /* List of data pdus to be sent, protected by write_list_lock */
+ struct list_head write_list;
+ /* List of data pdus being sent, protected by write_list_lock */
+ struct list_head write_timeout_list;
+
+ /* Protected by write_list_lock */
+ struct timer_list rsp_timer;
+ unsigned int rsp_timeout; /* in jiffies */
+
+ /* All 2 protected by iscsi_wr_lock */
+ unsigned short wr_state;
+ unsigned short wr_space_ready:1;
+
+ struct list_head wr_list_entry;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ struct task_struct *wr_task;
+#endif
+
+ /*
+ * All are unprotected, since accessed only from a single write
+ * thread.
+ */
+ struct iscsi_cmnd *write_cmnd;
+ struct iovec *write_iop;
+ int write_iop_used;
+ struct iovec write_iov[2];
+ u32 write_size;
+ u32 write_offset;
+ int write_state;
+
+ /* Both don't need any protection */
+ struct file *file;
+ struct socket *sock;
+
+ void (*old_state_change)(struct sock *);
+ void (*old_data_ready)(struct sock *, int);
+ void (*old_write_space)(struct sock *);
+
+ /* Both read only. Stay here for better CPU cache locality. */
+ int hdigest_type;
+ int ddigest_type;
+
+ /* All 6 protected by iscsi_rd_lock */
+ unsigned short rd_state;
+ unsigned short rd_data_ready:1;
+ /* Let's save some cache footprint by putting them here */
+ unsigned short closing:1;
+ unsigned short active_close:1;
+ unsigned short deleting:1;
+ unsigned short conn_tm_active:1;
+
+ struct list_head rd_list_entry;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ struct task_struct *rd_task;
+#endif
+
+ unsigned long last_rcv_time;
+
+ /*
+ * All are unprotected, since accessed only from a single read
+ * thread.
+ */
+ struct iscsi_cmnd *read_cmnd;
+ struct msghdr read_msg;
+ u32 read_size;
+ int read_state;
+ struct iovec *read_iov;
+ struct task_struct *rx_task;
+ uint32_t rpadding;
+
+ struct iscsi_target *target;
+
+ struct list_head conn_list_entry; /* list entry in session conn_list */
+
+ /* All protected by target_mutex, where necessary */
+ struct iscsi_conn *conn_reinst_successor;
+ struct list_head reinst_pending_cmd_list;
+
+ wait_queue_head_t read_state_waitQ;
+ struct completion ready_to_free;
+
+ /* Doesn't need any protection */
+ u16 cid;
+
+ struct delayed_work nop_in_delayed_work;
+ unsigned int nop_in_interval; /* in jiffies */
+ struct list_head nop_req_list;
+ spinlock_t nop_req_list_lock;
+ u32 nop_in_ttt;
+
+ /* Doesn't need any protection */
+ struct kobject iscsi_conn_kobj;
+};
+
+struct iscsi_pdu {
+ struct iscsi_hdr bhs;
+ void *ahs;
+ unsigned int ahssize;
+ unsigned int datasize;
+};
+
+typedef void (iscsi_show_info_t)(struct seq_file *seq,
+ struct iscsi_target *target);
+
+/** Commands' states **/
+
+/* New command and SCST processes it */
+#define ISCSI_CMD_STATE_NEW 0
+
+/* SCST processes cmd after scst_rx_cmd() */
+#define ISCSI_CMD_STATE_RX_CMD 1
+
+/* The command returned from preprocessing_done() */
+#define ISCSI_CMD_STATE_AFTER_PREPROC 2
+
+/* The command is waiting for session or connection reinstatement finished */
+#define ISCSI_CMD_STATE_REINST_PENDING 3
+
+/* scst_restart_cmd() called and SCST processing it */
+#define ISCSI_CMD_STATE_RESTARTED 4
+
+/* SCST done processing */
+#define ISCSI_CMD_STATE_PROCESSED 5
+
+/* AEN processing */
+#define ISCSI_CMD_STATE_AEN 6
+
+/* Out of SCST core preliminary completed */
+#define ISCSI_CMD_STATE_OUT_OF_SCST_PRELIM_COMPL 7
+
+/*
+ * Most of the fields don't need any protection, since accessed from only a
+ * single thread, except where noted.
+ *
+ * ToDo: Eventually divide request and response structures in 2 separate
+ * structures and stop this IET-derived garbage.
+ */
+struct iscsi_cmnd {
+ struct iscsi_conn *conn;
+
+ /*
+ * Some flags used under conn->write_list_lock, but all modified only
+ * from single read thread or when there are no references to cmd.
+ */
+ unsigned int hashed:1;
+ unsigned int should_close_conn:1;
+ unsigned int should_close_all_conn:1;
+ unsigned int pending:1;
+ unsigned int own_sg:1;
+ unsigned int on_write_list:1;
+ unsigned int write_processing_started:1;
+ unsigned int force_cleanup_done:1;
+ unsigned int dec_active_cmnds:1;
+ unsigned int ddigest_checked:1;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ unsigned int on_rx_digest_list:1;
+ unsigned int release_called:1;
+#endif
+
+ /*
+ * We suppose that preliminary commands completion is tested by
+ * comparing prelim_compl_flags with 0. Otherwise, because of the
+ * gap between setting different flags a race is possible,
+ * like sending command in SCST core as PRELIM_COMPLETED, while it
+ * wasn't aborted in it yet and have as the result a wrong success
+ * status sent to the initiator.
+ */
+#define ISCSI_CMD_ABORTED 0
+#define ISCSI_CMD_PRELIM_COMPLETED 1
+ unsigned long prelim_compl_flags;
+
+ struct list_head hash_list_entry;
+
+ /*
+ * Unions are for readability and grepability and to save some
+ * cache footprint.
+ */
+
+ union {
+ /*
+ * Used only to abort not yet sent responses. Usage in
+ * cmnd_done() is only a side effect to have a lockless
+ * accesss to this list from always only a single thread
+ * at any time. So, all responses live in the parent
+ * until it has the last reference put.
+ */
+ struct list_head rsp_cmd_list;
+ struct list_head rsp_cmd_list_entry;
+ };
+
+ union {
+ struct list_head pending_list_entry;
+ struct list_head reinst_pending_cmd_list_entry;
+ };
+
+ union {
+ struct list_head write_list_entry;
+ struct list_head write_timeout_list_entry;
+ };
+
+ /* Both protected by conn->write_list_lock */
+ unsigned int on_write_timeout_list:1;
+ unsigned long write_start;
+
+ /*
+ * All unprotected, since could be accessed from only a single
+ * thread at time
+ */
+ struct iscsi_cmnd *parent_req;
+ struct iscsi_cmnd *cmd_req;
+
+ /*
+ * All unprotected, since could be accessed from only a single
+ * thread at time
+ */
+ union {
+ /* Request only fields */
+ struct {
+ struct list_head rx_ddigest_cmd_list;
+ struct list_head rx_ddigest_cmd_list_entry;
+
+ int scst_state;
+ union {
+ struct scst_cmd *scst_cmd;
+ struct scst_aen *scst_aen;
+ };
+ unsigned int read_size;
+
+ struct iscsi_cmnd *main_rsp;
+ };
+
+ /* Response only fields */
+ struct {
+ struct scatterlist rsp_sg[2];
+ struct iscsi_sense_data sense_hdr;
+ };
+ };
+
+ atomic_t ref_cnt;
+
+ struct iscsi_pdu pdu;
+
+ struct scatterlist *sg;
+ int sg_cnt;
+ unsigned int bufflen;
+ u32 r2t_sn;
+ unsigned int r2t_len_to_receive;
+ unsigned int r2t_len_to_send;
+ unsigned int outstanding_r2t;
+ u32 target_task_tag;
+ u32 hdigest;
+ u32 ddigest;
+
+ struct list_head cmd_list_entry;
+ struct list_head nop_req_list_entry;
+};
+
+/* Max time to wait for our response satisfied for aborted commands */
+#define ISCSI_TM_DATA_WAIT_TIMEOUT (10 * HZ)
+
+/*
+ * Needed addition to all timeouts to complete a burst of commands at once.
+ * Otherwise, a part of the burst can be timeouted only in double timeout time.
+ */
+#define ISCSI_ADD_SCHED_TIME HZ
+
+#define ISCSI_CTR_OPEN_STATE_CLOSED 0
+#define ISCSI_CTR_OPEN_STATE_OPEN 1
+#define ISCSI_CTR_OPEN_STATE_CLOSING 2
+
+extern struct mutex target_mgmt_mutex;
+
+extern int ctr_open_state;
+extern const struct file_operations ctr_fops;
+
+extern spinlock_t iscsi_rd_lock;
+extern struct list_head iscsi_rd_list;
+extern wait_queue_head_t iscsi_rd_waitQ;
+
+extern spinlock_t iscsi_wr_lock;
+extern struct list_head iscsi_wr_list;
+extern wait_queue_head_t iscsi_wr_waitQ;
+
+/* iscsi.c */
+extern struct iscsi_cmnd *cmnd_alloc(struct iscsi_conn *,
+ struct iscsi_cmnd *parent);
+extern int cmnd_rx_start(struct iscsi_cmnd *);
+extern int cmnd_rx_continue(struct iscsi_cmnd *req);
+extern void cmnd_rx_end(struct iscsi_cmnd *);
+extern void cmnd_tx_start(struct iscsi_cmnd *);
+extern void cmnd_tx_end(struct iscsi_cmnd *);
+extern void req_cmnd_release_force(struct iscsi_cmnd *req);
+extern void rsp_cmnd_release(struct iscsi_cmnd *);
+extern void cmnd_done(struct iscsi_cmnd *cmnd);
+extern void conn_abort(struct iscsi_conn *conn);
+extern void iscsi_restart_cmnd(struct iscsi_cmnd *cmnd);
+extern void iscsi_fail_data_waiting_cmnd(struct iscsi_cmnd *cmnd);
+extern void iscsi_send_nop_in(struct iscsi_conn *conn);
+
+/* conn.c */
+extern struct iscsi_conn *conn_lookup(struct iscsi_session *, u16);
+extern void conn_reinst_finished(struct iscsi_conn *);
+extern int __add_conn(struct iscsi_session *, struct iscsi_kern_conn_info *);
+extern int __del_conn(struct iscsi_session *, struct iscsi_kern_conn_info *);
+extern void iscsi_make_conn_rd_active(struct iscsi_conn *conn);
+#define ISCSI_CONN_ACTIVE_CLOSE 1
+#define ISCSI_CONN_DELETING 2
+extern void __mark_conn_closed(struct iscsi_conn *, int);
+extern void mark_conn_closed(struct iscsi_conn *);
+extern void iscsi_make_conn_wr_active(struct iscsi_conn *);
+extern void iscsi_check_tm_data_wait_timeouts(struct iscsi_conn *conn,
+ bool force);
+
+/* nthread.c */
+extern int iscsi_send(struct iscsi_conn *conn);
+extern int istrd(void *arg);
+extern int istwr(void *arg);
+extern void iscsi_task_mgmt_affected_cmds_done(struct scst_mgmt_cmd *scst_mcmd);
+extern void req_add_to_write_timeout_list(struct iscsi_cmnd *req);
+
+/* target.c */
+extern const struct attribute *iscsi_tgt_attrs[];
+extern int iscsi_enable_target(struct scst_tgt *scst_tgt, bool enable);
+extern bool iscsi_is_target_enabled(struct scst_tgt *scst_tgt);
+extern ssize_t iscsi_sysfs_send_event(uint32_t tid,
+ enum iscsi_kern_event_code code,
+ const char *param1, const char *param2, void **data);
+extern struct iscsi_target *target_lookup_by_id(u32);
+extern int __add_target(struct iscsi_kern_target_info *);
+extern int __del_target(u32 id);
+extern ssize_t iscsi_sysfs_add_target(const char *target_name, char *params);
+extern ssize_t iscsi_sysfs_del_target(const char *target_name);
+extern ssize_t iscsi_sysfs_mgmt_cmd(char *cmd);
+extern void target_del_session(struct iscsi_target *target,
+ struct iscsi_session *session, int flags);
+extern void target_del_all_sess(struct iscsi_target *target, int flags);
+extern void target_del_all(void);
+
+/* config.c */
+extern const struct attribute *iscsi_attrs[];
+extern int iscsi_add_attr(struct iscsi_target *target,
+ const struct iscsi_kern_attr *user_info);
+extern void __iscsi_del_attr(struct iscsi_target *target,
+ struct iscsi_attr *tgt_attr);
+
+/* session.c */
+extern const struct attribute *iscsi_sess_attrs[];
+extern const struct file_operations session_seq_fops;
+extern struct iscsi_session *session_lookup(struct iscsi_target *, u64);
+extern void sess_reinst_finished(struct iscsi_session *);
+extern int __add_session(struct iscsi_target *,
+ struct iscsi_kern_session_info *);
+extern int __del_session(struct iscsi_target *, u64);
+extern int session_free(struct iscsi_session *session, bool del);
+
+/* params.c */
+extern const char *iscsi_get_digest_name(int val, char *res);
+extern const char *iscsi_get_bool_value(int val);
+extern int iscsi_params_set(struct iscsi_target *,
+ struct iscsi_kern_params_info *, int);
+
+/* event.c */
+extern int event_send(u32, u64, u32, u32, enum iscsi_kern_event_code,
+ const char *param1, const char *param2);
+extern int event_init(void);
+extern void event_exit(void);
+
+#define get_pgcnt(size, offset) \
+ ((((size) + ((offset) & ~PAGE_MASK)) + PAGE_SIZE - 1) >> PAGE_SHIFT)
+
+static inline void iscsi_cmnd_get_length(struct iscsi_pdu *pdu)
+{
+#if defined(__BIG_ENDIAN)
+ pdu->ahssize = pdu->bhs.length.ahslength * 4;
+ pdu->datasize = pdu->bhs.length.datalength;
+#elif defined(__LITTLE_ENDIAN)
+ pdu->ahssize = (pdu->bhs.length & 0xff) * 4;
+ pdu->datasize = be32_to_cpu(pdu->bhs.length & ~0xff);
+#else
+#error
+#endif
+}
+
+static inline void iscsi_cmnd_set_length(struct iscsi_pdu *pdu)
+{
+#if defined(__BIG_ENDIAN)
+ pdu->bhs.length.ahslength = pdu->ahssize / 4;
+ pdu->bhs.length.datalength = pdu->datasize;
+#elif defined(__LITTLE_ENDIAN)
+ pdu->bhs.length = cpu_to_be32(pdu->datasize) | (pdu->ahssize / 4);
+#else
+#error
+#endif
+}
+
+extern struct scst_tgt_template iscsi_template;
+
+/*
+ * Skip this command if result is not 0. Must be called under
+ * corresponding lock.
+ */
+static inline bool cmnd_get_check(struct iscsi_cmnd *cmnd)
+{
+ int r = atomic_inc_return(&cmnd->ref_cnt);
+ int res;
+ if (unlikely(r == 1)) {
+ TRACE_DBG("cmnd %p is being destroyed", cmnd);
+ atomic_dec(&cmnd->ref_cnt);
+ res = 1;
+ /* Necessary code is serialized by locks in cmnd_done() */
+ } else {
+ TRACE_DBG("cmnd %p, new ref_cnt %d", cmnd,
+ atomic_read(&cmnd->ref_cnt));
+ res = 0;
+ }
+ return res;
+}
+
+static inline void cmnd_get(struct iscsi_cmnd *cmnd)
+{
+ atomic_inc(&cmnd->ref_cnt);
+ TRACE_DBG("cmnd %p, new cmnd->ref_cnt %d", cmnd,
+ atomic_read(&cmnd->ref_cnt));
+}
+
+static inline void cmnd_get_ordered(struct iscsi_cmnd *cmnd)
+{
+ cmnd_get(cmnd);
+ /* See comments for each cmnd_get_ordered() use */
+ smp_mb__after_atomic_inc();
+}
+
+static inline void cmnd_put(struct iscsi_cmnd *cmnd)
+{
+ TRACE_DBG("cmnd %p, new ref_cnt %d", cmnd,
+ atomic_read(&cmnd->ref_cnt)-1);
+
+ EXTRACHECKS_BUG_ON(atomic_read(&cmnd->ref_cnt) == 0);
+
+ if (atomic_dec_and_test(&cmnd->ref_cnt))
+ cmnd_done(cmnd);
+}
+
+/* conn->write_list_lock supposed to be locked and BHs off */
+static inline void cmd_add_on_write_list(struct iscsi_conn *conn,
+ struct iscsi_cmnd *cmnd)
+{
+ TRACE_DBG("cmnd %p", cmnd);
+ /* See comment in iscsi_restart_cmnd() */
+ EXTRACHECKS_BUG_ON(cmnd->parent_req->hashed &&
+ (cmnd_opcode(cmnd) != ISCSI_OP_R2T));
+ list_add_tail(&cmnd->write_list_entry, &conn->write_list);
+ cmnd->on_write_list = 1;
+}
+
+/* conn->write_list_lock supposed to be locked and BHs off */
+static inline void cmd_del_from_write_list(struct iscsi_cmnd *cmnd)
+{
+ TRACE_DBG("%p", cmnd);
+ list_del(&cmnd->write_list_entry);
+ cmnd->on_write_list = 0;
+}
+
+static inline void cmd_add_on_rx_ddigest_list(struct iscsi_cmnd *req,
+ struct iscsi_cmnd *cmnd)
+{
+ TRACE_DBG("Adding RX ddigest cmd %p to digest list "
+ "of req %p", cmnd, req);
+ list_add_tail(&cmnd->rx_ddigest_cmd_list_entry,
+ &req->rx_ddigest_cmd_list);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ cmnd->on_rx_digest_list = 1;
+#endif
+}
+
+static inline void cmd_del_from_rx_ddigest_list(struct iscsi_cmnd *cmnd)
+{
+ TRACE_DBG("Deleting RX digest cmd %p from digest list", cmnd);
+ list_del(&cmnd->rx_ddigest_cmd_list_entry);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ cmnd->on_rx_digest_list = 0;
+#endif
+}
+
+static inline int test_write_ready(struct iscsi_conn *conn)
+{
+ /*
+ * No need for write_list protection, in the worst case we will be
+ * restarted again.
+ */
+ return !list_empty(&conn->write_list) || conn->write_cmnd;
+}
+
+static inline void conn_get(struct iscsi_conn *conn)
+{
+ atomic_inc(&conn->conn_ref_cnt);
+ TRACE_DBG("conn %p, new conn_ref_cnt %d", conn,
+ atomic_read(&conn->conn_ref_cnt));
+}
+
+static inline void conn_get_ordered(struct iscsi_conn *conn)
+{
+ conn_get(conn);
+ /* See comments for each conn_get_ordered() use */
+ smp_mb__after_atomic_inc();
+}
+
+static inline void conn_put(struct iscsi_conn *conn)
+{
+ TRACE_DBG("conn %p, new conn_ref_cnt %d", conn,
+ atomic_read(&conn->conn_ref_cnt)-1);
+ BUG_ON(atomic_read(&conn->conn_ref_cnt) == 0);
+
+ /*
+ * Make it always ordered to protect from undesired side effects like
+ * accessing just destroyed by close_conn() conn caused by reordering
+ * of this atomic_dec().
+ */
+ smp_mb__before_atomic_dec();
+ atomic_dec(&conn->conn_ref_cnt);
+}
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+extern void iscsi_extracheck_is_rd_thread(struct iscsi_conn *conn);
+extern void iscsi_extracheck_is_wr_thread(struct iscsi_conn *conn);
+#else
+static inline void iscsi_extracheck_is_rd_thread(struct iscsi_conn *conn) {}
+static inline void iscsi_extracheck_is_wr_thread(struct iscsi_conn *conn) {}
+#endif
+
+#endif /* __ISCSI_H__ */
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi_hdr.h linux-2.6.33/drivers/scst/iscsi-scst/iscsi_hdr.h
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi_hdr.h
+++ linux-2.6.33/drivers/scst/iscsi-scst/iscsi_hdr.h
@@ -0,0 +1,519 @@
+/*
+ * Copyright (C) 2002 - 2003 Ardis Technolgies <roman@ardistech.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ISCSI_HDR_H__
+#define __ISCSI_HDR_H__
+
+#include <linux/types.h>
+#include <asm/byteorder.h>
+
+#define ISCSI_VERSION 0
+
+#ifndef __packed
+#define __packed __attribute__ ((packed))
+#endif
+
+struct iscsi_hdr {
+ u8 opcode; /* 0 */
+ u8 flags;
+ u8 spec1[2];
+#if defined(__BIG_ENDIAN_BITFIELD)
+ struct { /* 4 */
+ unsigned ahslength:8;
+ unsigned datalength:24;
+ } length;
+#elif defined(__LITTLE_ENDIAN_BITFIELD)
+ u32 length; /* 4 */
+#endif
+ u64 lun; /* 8 */
+ u32 itt; /* 16 */
+ u32 ttt; /* 20 */
+ u32 sn; /* 24 */
+ u32 exp_sn; /* 28 */
+ u32 max_sn; /* 32 */
+ u32 spec3[3]; /* 36 */
+} __packed; /* 48 */
+
+/* Opcode encoding bits */
+#define ISCSI_OP_RETRY 0x80
+#define ISCSI_OP_IMMEDIATE 0x40
+#define ISCSI_OPCODE_MASK 0x3F
+
+/* Client to Server Message Opcode values */
+#define ISCSI_OP_NOP_OUT 0x00
+#define ISCSI_OP_SCSI_CMD 0x01
+#define ISCSI_OP_SCSI_TASK_MGT_MSG 0x02
+#define ISCSI_OP_LOGIN_CMD 0x03
+#define ISCSI_OP_TEXT_CMD 0x04
+#define ISCSI_OP_SCSI_DATA_OUT 0x05
+#define ISCSI_OP_LOGOUT_CMD 0x06
+#define ISCSI_OP_SNACK_CMD 0x10
+
+/* Server to Client Message Opcode values */
+#define ISCSI_OP_NOP_IN 0x20
+#define ISCSI_OP_SCSI_RSP 0x21
+#define ISCSI_OP_SCSI_TASK_MGT_RSP 0x22
+#define ISCSI_OP_LOGIN_RSP 0x23
+#define ISCSI_OP_TEXT_RSP 0x24
+#define ISCSI_OP_SCSI_DATA_IN 0x25
+#define ISCSI_OP_LOGOUT_RSP 0x26
+#define ISCSI_OP_R2T 0x31
+#define ISCSI_OP_ASYNC_MSG 0x32
+#define ISCSI_OP_REJECT 0x3f
+
+struct iscsi_ahs_hdr {
+ u16 ahslength;
+ u8 ahstype;
+} __packed;
+
+#define ISCSI_AHSTYPE_CDB 1
+#define ISCSI_AHSTYPE_RLENGTH 2
+
+union iscsi_sid {
+ struct {
+ u8 isid[6]; /* Initiator Session ID */
+ u16 tsih; /* Target Session ID */
+ } id;
+ u64 id64;
+} __packed;
+
+struct iscsi_scsi_cmd_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u64 lun;
+ u32 itt;
+ u32 data_length;
+ u32 cmd_sn;
+ u32 exp_stat_sn;
+ u8 scb[16];
+} __packed;
+
+#define ISCSI_CMD_FINAL 0x80
+#define ISCSI_CMD_READ 0x40
+#define ISCSI_CMD_WRITE 0x20
+#define ISCSI_CMD_ATTR_MASK 0x07
+#define ISCSI_CMD_UNTAGGED 0x00
+#define ISCSI_CMD_SIMPLE 0x01
+#define ISCSI_CMD_ORDERED 0x02
+#define ISCSI_CMD_HEAD_OF_QUEUE 0x03
+#define ISCSI_CMD_ACA 0x04
+
+struct iscsi_cdb_ahdr {
+ u16 ahslength;
+ u8 ahstype;
+ u8 reserved;
+ u8 cdb[0];
+} __packed;
+
+struct iscsi_rlength_ahdr {
+ u16 ahslength;
+ u8 ahstype;
+ u8 reserved;
+ u32 read_length;
+} __packed;
+
+struct iscsi_scsi_rsp_hdr {
+ u8 opcode;
+ u8 flags;
+ u8 response;
+ u8 cmd_status;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd1[2];
+ u32 itt;
+ u32 snack;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 exp_data_sn;
+ u32 bi_residual_count;
+ u32 residual_count;
+} __packed;
+
+#define ISCSI_FLG_RESIDUAL_UNDERFLOW 0x02
+#define ISCSI_FLG_RESIDUAL_OVERFLOW 0x04
+#define ISCSI_FLG_BIRESIDUAL_UNDERFLOW 0x08
+#define ISCSI_FLG_BIRESIDUAL_OVERFLOW 0x10
+
+#define ISCSI_RESPONSE_COMMAND_COMPLETED 0x00
+#define ISCSI_RESPONSE_TARGET_FAILURE 0x01
+
+struct iscsi_sense_data {
+ u16 length;
+ u8 data[0];
+} __packed;
+
+struct iscsi_task_mgt_hdr {
+ u8 opcode;
+ u8 function;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u64 lun;
+ u32 itt;
+ u32 rtt;
+ u32 cmd_sn;
+ u32 exp_stat_sn;
+ u32 ref_cmd_sn;
+ u32 exp_data_sn;
+ u32 rsvd2[2];
+} __packed;
+
+#define ISCSI_FUNCTION_MASK 0x7f
+
+#define ISCSI_FUNCTION_ABORT_TASK 1
+#define ISCSI_FUNCTION_ABORT_TASK_SET 2
+#define ISCSI_FUNCTION_CLEAR_ACA 3
+#define ISCSI_FUNCTION_CLEAR_TASK_SET 4
+#define ISCSI_FUNCTION_LOGICAL_UNIT_RESET 5
+#define ISCSI_FUNCTION_TARGET_WARM_RESET 6
+#define ISCSI_FUNCTION_TARGET_COLD_RESET 7
+#define ISCSI_FUNCTION_TASK_REASSIGN 8
+
+struct iscsi_task_rsp_hdr {
+ u8 opcode;
+ u8 flags;
+ u8 response;
+ u8 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 itt;
+ u32 rsvd3;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 rsvd4[3];
+} __packed;
+
+#define ISCSI_RESPONSE_FUNCTION_COMPLETE 0
+#define ISCSI_RESPONSE_UNKNOWN_TASK 1
+#define ISCSI_RESPONSE_UNKNOWN_LUN 2
+#define ISCSI_RESPONSE_TASK_ALLEGIANT 3
+#define ISCSI_RESPONSE_ALLEGIANCE_REASSIGNMENT_UNSUPPORTED 4
+#define ISCSI_RESPONSE_FUNCTION_UNSUPPORTED 5
+#define ISCSI_RESPONSE_NO_AUTHORIZATION 6
+#define ISCSI_RESPONSE_FUNCTION_REJECTED 255
+
+struct iscsi_data_out_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u64 lun;
+ u32 itt;
+ u32 ttt;
+ u32 rsvd2;
+ u32 exp_stat_sn;
+ u32 rsvd3;
+ u32 data_sn;
+ u32 buffer_offset;
+ u32 rsvd4;
+} __packed;
+
+struct iscsi_data_in_hdr {
+ u8 opcode;
+ u8 flags;
+ u8 rsvd1;
+ u8 cmd_status;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 itt;
+ u32 ttt;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 data_sn;
+ u32 buffer_offset;
+ u32 residual_count;
+} __packed;
+
+#define ISCSI_FLG_STATUS 0x01
+
+struct iscsi_r2t_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u64 lun;
+ u32 itt;
+ u32 ttt;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 r2t_sn;
+ u32 buffer_offset;
+ u32 data_length;
+} __packed;
+
+struct iscsi_async_msg_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u64 lun;
+ u32 ffffffff;
+ u32 rsvd2;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u8 async_event;
+ u8 async_vcode;
+ u16 param1;
+ u16 param2;
+ u16 param3;
+ u32 rsvd3;
+} __packed;
+
+#define ISCSI_ASYNC_SCSI 0
+#define ISCSI_ASYNC_LOGOUT 1
+#define ISCSI_ASYNC_DROP_CONNECTION 2
+#define ISCSI_ASYNC_DROP_SESSION 3
+#define ISCSI_ASYNC_PARAM_REQUEST 4
+#define ISCSI_ASYNC_VENDOR 255
+
+struct iscsi_text_req_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 itt;
+ u32 ttt;
+ u32 cmd_sn;
+ u32 exp_stat_sn;
+ u32 rsvd3[4];
+} __packed;
+
+struct iscsi_text_rsp_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 itt;
+ u32 ttt;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 rsvd3[3];
+} __packed;
+
+struct iscsi_login_req_hdr {
+ u8 opcode;
+ u8 flags;
+ u8 max_version; /* Max. version supported */
+ u8 min_version; /* Min. version supported */
+ u8 ahslength;
+ u8 datalength[3];
+ union iscsi_sid sid;
+ u32 itt; /* Initiator Task Tag */
+ u16 cid; /* Connection ID */
+ u16 rsvd1;
+ u32 cmd_sn;
+ u32 exp_stat_sn;
+ u32 rsvd2[4];
+} __packed;
+
+struct iscsi_login_rsp_hdr {
+ u8 opcode;
+ u8 flags;
+ u8 max_version; /* Max. version supported */
+ u8 active_version; /* Active version */
+ u8 ahslength;
+ u8 datalength[3];
+ union iscsi_sid sid;
+ u32 itt; /* Initiator Task Tag */
+ u32 rsvd1;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u8 status_class; /* see Login RSP Status classes below */
+ u8 status_detail; /* see Login RSP Status details below */
+ u8 rsvd2[10];
+} __packed;
+
+#define ISCSI_FLG_FINAL 0x80
+#define ISCSI_FLG_TRANSIT 0x80
+#define ISCSI_FLG_CSG_SECURITY 0x00
+#define ISCSI_FLG_CSG_LOGIN 0x04
+#define ISCSI_FLG_CSG_FULL_FEATURE 0x0c
+#define ISCSI_FLG_CSG_MASK 0x0c
+#define ISCSI_FLG_NSG_SECURITY 0x00
+#define ISCSI_FLG_NSG_LOGIN 0x01
+#define ISCSI_FLG_NSG_FULL_FEATURE 0x03
+#define ISCSI_FLG_NSG_MASK 0x03
+
+/* Login Status response classes */
+#define ISCSI_STATUS_SUCCESS 0x00
+#define ISCSI_STATUS_REDIRECT 0x01
+#define ISCSI_STATUS_INITIATOR_ERR 0x02
+#define ISCSI_STATUS_TARGET_ERR 0x03
+
+/* Login Status response detail codes */
+/* Class-0 (Success) */
+#define ISCSI_STATUS_ACCEPT 0x00
+
+/* Class-1 (Redirection) */
+#define ISCSI_STATUS_TGT_MOVED_TEMP 0x01
+#define ISCSI_STATUS_TGT_MOVED_PERM 0x02
+
+/* Class-2 (Initiator Error) */
+#define ISCSI_STATUS_INIT_ERR 0x00
+#define ISCSI_STATUS_AUTH_FAILED 0x01
+#define ISCSI_STATUS_TGT_FORBIDDEN 0x02
+#define ISCSI_STATUS_TGT_NOT_FOUND 0x03
+#define ISCSI_STATUS_TGT_REMOVED 0x04
+#define ISCSI_STATUS_NO_VERSION 0x05
+#define ISCSI_STATUS_TOO_MANY_CONN 0x06
+#define ISCSI_STATUS_MISSING_FIELDS 0x07
+#define ISCSI_STATUS_CONN_ADD_FAILED 0x08
+#define ISCSI_STATUS_INV_SESSION_TYPE 0x09
+#define ISCSI_STATUS_SESSION_NOT_FOUND 0x0a
+#define ISCSI_STATUS_INV_REQ_TYPE 0x0b
+
+/* Class-3 (Target Error) */
+#define ISCSI_STATUS_TARGET_ERROR 0x00
+#define ISCSI_STATUS_SVC_UNAVAILABLE 0x01
+#define ISCSI_STATUS_NO_RESOURCES 0x02
+
+struct iscsi_logout_req_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 itt;
+ u16 cid;
+ u16 rsvd3;
+ u32 cmd_sn;
+ u32 exp_stat_sn;
+ u32 rsvd4[4];
+} __packed;
+
+struct iscsi_logout_rsp_hdr {
+ u8 opcode;
+ u8 flags;
+ u8 response;
+ u8 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 itt;
+ u32 rsvd3;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 rsvd4;
+ u16 time2wait;
+ u16 time2retain;
+ u32 rsvd5;
+} __packed;
+
+struct iscsi_snack_req_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 itt;
+ u32 ttt;
+ u32 rsvd3;
+ u32 exp_stat_sn;
+ u32 rsvd4[2];
+ u32 beg_run;
+ u32 run_length;
+} __packed;
+
+struct iscsi_reject_hdr {
+ u8 opcode;
+ u8 flags;
+ u8 reason;
+ u8 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u32 rsvd2[2];
+ u32 ffffffff;
+ u32 rsvd3;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 data_sn;
+ u32 rsvd4[2];
+} __packed;
+
+#define ISCSI_REASON_RESERVED 0x01
+#define ISCSI_REASON_DATA_DIGEST_ERROR 0x02
+#define ISCSI_REASON_DATA_SNACK_REJECT 0x03
+#define ISCSI_REASON_PROTOCOL_ERROR 0x04
+#define ISCSI_REASON_UNSUPPORTED_COMMAND 0x05
+#define ISCSI_REASON_IMMEDIATE_COMMAND_REJECT 0x06
+#define ISCSI_REASON_TASK_IN_PROGRESS 0x07
+#define ISCSI_REASON_INVALID_DATA_ACK 0x08
+#define ISCSI_REASON_INVALID_PDU_FIELD 0x09
+#define ISCSI_REASON_OUT_OF_RESOURCES 0x0a
+#define ISCSI_REASON_NEGOTIATION_RESET 0x0b
+#define ISCSI_REASON_WAITING_LOGOUT 0x0c
+
+struct iscsi_nop_out_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u64 lun;
+ u32 itt;
+ u32 ttt;
+ u32 cmd_sn;
+ u32 exp_stat_sn;
+ u32 rsvd2[4];
+} __packed;
+
+struct iscsi_nop_in_hdr {
+ u8 opcode;
+ u8 flags;
+ u16 rsvd1;
+ u8 ahslength;
+ u8 datalength[3];
+ u64 lun;
+ u32 itt;
+ u32 ttt;
+ u32 stat_sn;
+ u32 exp_cmd_sn;
+ u32 max_cmd_sn;
+ u32 rsvd2[3];
+} __packed;
+
+#define ISCSI_RESERVED_TAG (0xffffffffU)
+
+#define cmnd_hdr(cmnd) ((struct iscsi_scsi_cmd_hdr *) (&((cmnd)->pdu.bhs)))
+#define cmnd_itt(cmnd) cpu_to_be32((cmnd)->pdu.bhs.itt)
+#define cmnd_ttt(cmnd) cpu_to_be32((cmnd)->pdu.bhs.ttt)
+#define cmnd_opcode(cmnd) ((cmnd)->pdu.bhs.opcode & ISCSI_OPCODE_MASK)
+#define cmnd_scsicode(cmnd) (cmnd_hdr((cmnd))->scb[0])
+
+#endif /* __ISCSI_HDR_H__ */
diff -uprN orig/linux-2.6.33/include/scst/iscsi_scst_itf_ver.h linux-2.6.33/include/scst/iscsi_scst_itf_ver.h
--- orig/linux-2.6.33/include/scst/iscsi_scst_itf_ver.h
+++ linux-2.6.33/include/scst/iscsi_scst_itf_ver.h
@@ -0,0 +1,2 @@
+#define ISCSI_SCST_INTERFACE_VERSION "0b9736e7abc6440bb842a800ec0552ae5388b0ef"
+
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 3/4/4/5] iSCSI-SCST's implementation files
[not found] ` <4BC45EF7.7010304@vlnb.net>
@ 2010-04-13 13:10 ` Vladislav Bolkhovitin
2010-04-13 13:10 ` [PATCH][RFC 4/4/4/5] iSCSI-SCST's README file Vladislav Bolkhovitin
1 sibling, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:10 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Bart Van Assche
This patch contains iSCSI-SCST's implementation files.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
config.c | 933 ++++++++++++++++
conn.c | 785 +++++++++++++
digest.c | 226 +++
event.c | 163 ++
iscsi.c | 3583 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
nthread.c | 1524 ++++++++++++++++++++++++++
param.c | 306 +++++
session.c | 482 ++++++++
target.c | 500 ++++++++
9 files changed, 8502 insertions(+)
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/config.c linux-2.6.33/drivers/scst/iscsi-scst/config.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/config.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/config.c
@@ -0,0 +1,933 @@
+/*
+ * Copyright (C) 2004 - 2005 FUJITA Tomonori <tomof@acm.org>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "iscsi.h"
+
+/* Protected by target_mgmt_mutex */
+int ctr_open_state;
+
+/* Protected by target_mgmt_mutex */
+static LIST_HEAD(iscsi_attrs_list);
+
+static ssize_t iscsi_version_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+
+ sprintf(buf, "%s\n", ISCSI_VERSION_STRING);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ strcat(buf, "EXTRACHECKS\n");
+#endif
+
+#ifdef CONFIG_SCST_TRACING
+ strcat(buf, "TRACING\n");
+#endif
+
+#ifdef CONFIG_SCST_DEBUG
+ strcat(buf, "DEBUG\n");
+#endif
+
+#ifdef CONFIG_SCST_ISCSI_DEBUG_DIGEST_FAILURES
+ strcat(buf, "DEBUG_DIGEST_FAILURES\n");
+#endif
+ return strlen(buf);
+}
+
+static struct kobj_attribute iscsi_version_attr =
+ __ATTR(version, S_IRUGO, iscsi_version_show, NULL);
+
+static ssize_t iscsi_open_state_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ switch (ctr_open_state) {
+ case ISCSI_CTR_OPEN_STATE_CLOSED:
+ sprintf(buf, "%s\n", "closed");
+ break;
+ case ISCSI_CTR_OPEN_STATE_OPEN:
+ sprintf(buf, "%s\n", "open");
+ break;
+ case ISCSI_CTR_OPEN_STATE_CLOSING:
+ sprintf(buf, "%s\n", "closing");
+ break;
+ default:
+ sprintf(buf, "%s\n", "unknown");
+ break;
+ }
+
+ return strlen(buf);
+}
+
+static struct kobj_attribute iscsi_open_state_attr =
+ __ATTR(open_state, S_IRUGO, iscsi_open_state_show, NULL);
+
+const struct attribute *iscsi_attrs[] = {
+ &iscsi_version_attr.attr,
+ &iscsi_open_state_attr.attr,
+ NULL,
+};
+
+/* target_mgmt_mutex supposed to be locked */
+static int add_conn(void __user *ptr)
+{
+ int err, rc;
+ struct iscsi_session *session;
+ struct iscsi_kern_conn_info info;
+ struct iscsi_target *target;
+
+ rc = copy_from_user(&info, ptr, sizeof(info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out;
+ }
+
+ target = target_lookup_by_id(info.tid);
+ if (target == NULL) {
+ PRINT_ERROR("Target %d not found", info.tid);
+ err = -ENOENT;
+ goto out;
+ }
+
+ mutex_lock(&target->target_mutex);
+
+ session = session_lookup(target, info.sid);
+ if (!session) {
+ PRINT_ERROR("Session %lld not found",
+ (long long unsigned int)info.tid);
+ err = -ENOENT;
+ goto out_unlock;
+ }
+
+ err = __add_conn(session, &info);
+
+out_unlock:
+ mutex_unlock(&target->target_mutex);
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int del_conn(void __user *ptr)
+{
+ int err, rc;
+ struct iscsi_session *session;
+ struct iscsi_kern_conn_info info;
+ struct iscsi_target *target;
+
+ rc = copy_from_user(&info, ptr, sizeof(info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out;
+ }
+
+ target = target_lookup_by_id(info.tid);
+ if (target == NULL) {
+ PRINT_ERROR("Target %d not found", info.tid);
+ err = -ENOENT;
+ goto out;
+ }
+
+ mutex_lock(&target->target_mutex);
+
+ session = session_lookup(target, info.sid);
+ if (!session) {
+ PRINT_ERROR("Session %llx not found",
+ (long long unsigned int)info.sid);
+ err = -ENOENT;
+ goto out_unlock;
+ }
+
+ err = __del_conn(session, &info);
+
+out_unlock:
+ mutex_unlock(&target->target_mutex);
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int add_session(void __user *ptr)
+{
+ int err, rc;
+ struct iscsi_kern_session_info *info;
+ struct iscsi_target *target;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (info == NULL) {
+ PRINT_ERROR("Can't alloc info (size %zd)", sizeof(*info));
+ err = -ENOMEM;
+ goto out;
+ }
+
+ rc = copy_from_user(info, ptr, sizeof(*info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out_free;
+ }
+
+ info->initiator_name[sizeof(info->initiator_name)-1] = '\0';
+
+ target = target_lookup_by_id(info->tid);
+ if (target == NULL) {
+ PRINT_ERROR("Target %d not found", info->tid);
+ err = -ENOENT;
+ goto out_free;
+ }
+
+ err = __add_session(target, info);
+
+out_free:
+ kfree(info);
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int del_session(void __user *ptr)
+{
+ int err, rc;
+ struct iscsi_kern_session_info *info;
+ struct iscsi_target *target;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (info == NULL) {
+ PRINT_ERROR("Can't alloc info (size %zd)", sizeof(*info));
+ err = -ENOMEM;
+ goto out;
+ }
+
+ rc = copy_from_user(info, ptr, sizeof(*info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out_free;
+ }
+
+ info->initiator_name[sizeof(info->initiator_name)-1] = '\0';
+
+ target = target_lookup_by_id(info->tid);
+ if (target == NULL) {
+ PRINT_ERROR("Target %d not found", info->tid);
+ err = -ENOENT;
+ goto out_free;
+ }
+
+ mutex_lock(&target->target_mutex);
+ err = __del_session(target, info->sid);
+ mutex_unlock(&target->target_mutex);
+
+out_free:
+ kfree(info);
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int iscsi_params_config(void __user *ptr, int set)
+{
+ int err, rc;
+ struct iscsi_kern_params_info info;
+ struct iscsi_target *target;
+
+ rc = copy_from_user(&info, ptr, sizeof(info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out;
+ }
+
+ target = target_lookup_by_id(info.tid);
+ if (target == NULL) {
+ PRINT_ERROR("Target %d not found", info.tid);
+ err = -ENOENT;
+ goto out;
+ }
+
+ mutex_lock(&target->target_mutex);
+ err = iscsi_params_set(target, &info, set);
+ mutex_unlock(&target->target_mutex);
+
+ if (err < 0)
+ goto out;
+
+ if (!set) {
+ rc = copy_to_user(ptr, &info, sizeof(info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy to user %d bytes", rc);
+ err = -EFAULT;
+ goto out;
+ }
+ }
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int mgmt_cmd_callback(void __user *ptr)
+{
+ int err = 0, rc;
+ struct iscsi_kern_mgmt_cmd_res_info cinfo;
+ struct scst_sysfs_user_info *info;
+
+ rc = copy_from_user(&cinfo, ptr, sizeof(cinfo));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out;
+ }
+
+ cinfo.value[sizeof(cinfo.value)-1] = '\0';
+
+ info = scst_sysfs_user_get_info(cinfo.cookie);
+ TRACE_DBG("cookie %u, info %p, result %d", cinfo.cookie, info,
+ cinfo.result);
+ if (info == NULL) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ info->info_status = 0;
+
+ if (cinfo.result != 0) {
+ info->info_status = cinfo.result;
+ goto out_complete;
+ }
+
+ switch (cinfo.req_cmd) {
+ case E_ENABLE_TARGET:
+ case E_DISABLE_TARGET:
+ {
+ struct iscsi_target *target;
+
+ target = target_lookup_by_id(cinfo.tid);
+ if (target == NULL) {
+ PRINT_ERROR("Target %d not found", cinfo.tid);
+ err = -ENOENT;
+ goto out_status;
+ }
+
+ target->tgt_enabled = (cinfo.req_cmd == E_ENABLE_TARGET) ? 1 : 0;
+ break;
+ }
+
+ case E_GET_ATTR_VALUE:
+ info->data = kstrdup(cinfo.value, GFP_KERNEL);
+ if (info->data == NULL) {
+ PRINT_ERROR("Can't dublicate value %s", cinfo.value);
+ info->info_status = -ENOMEM;
+ goto out_complete;
+ }
+ break;
+ }
+
+out_complete:
+ complete(&info->info_completion);
+
+out:
+ return err;
+
+out_status:
+ info->info_status = err;
+ goto out_complete;
+}
+
+static ssize_t iscsi_attr_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct iscsi_attr *tgt_attr;
+ void *value;
+
+ tgt_attr = container_of(attr, struct iscsi_attr, attr);
+
+ pos = iscsi_sysfs_send_event(
+ (tgt_attr->target != NULL) ? tgt_attr->target->tid : 0,
+ E_GET_ATTR_VALUE, tgt_attr->name, NULL, &value);
+
+ if (pos != 0)
+ goto out;
+
+ pos = scnprintf(buf, SCST_SYSFS_BLOCK_SIZE, "%s\n", (char *)value);
+
+ kfree(value);
+
+out:
+ return pos;
+}
+
+static ssize_t iscsi_attr_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ char *buffer;
+ struct iscsi_attr *tgt_attr;
+
+ buffer = kzalloc(count+1, GFP_KERNEL);
+ if (buffer == NULL) {
+ res = -ENOMEM;
+ goto out;
+ }
+
+ memcpy(buffer, buf, count);
+ buffer[count] = '\0';
+
+ tgt_attr = container_of(attr, struct iscsi_attr, attr);
+
+ res = iscsi_sysfs_send_event(
+ (tgt_attr->target != NULL) ? tgt_attr->target->tid : 0,
+ E_SET_ATTR_VALUE, tgt_attr->name, buffer, NULL);
+
+ kfree(buffer);
+
+ if (res == 0)
+ res = count;
+
+out:
+ return res;
+}
+
+/*
+ * target_mgmt_mutex supposed to be locked. If target != 0, target_mutex
+ * supposed to be locked as well.
+ */
+int iscsi_add_attr(struct iscsi_target *target,
+ const struct iscsi_kern_attr *attr_info)
+{
+ int res = 0;
+ struct iscsi_attr *tgt_attr;
+ struct list_head *attrs_list;
+ const char *name;
+
+ if (target != NULL) {
+ attrs_list = &target->attrs_list;
+ name = target->name;
+ } else {
+ attrs_list = &iscsi_attrs_list;
+ name = "global";
+ }
+
+ list_for_each_entry(tgt_attr, attrs_list, attrs_list_entry) {
+ if (strncmp(tgt_attr->name, attr_info->name,
+ sizeof(tgt_attr->name) == 0)) {
+ PRINT_ERROR("Attribute %s for %s already exist",
+ attr_info->name, name);
+ res = -EEXIST;
+ goto out;
+ }
+ }
+
+ TRACE_DBG("Adding %s's attr %s with mode %x", name,
+ attr_info->name, attr_info->mode);
+
+ tgt_attr = kzalloc(sizeof(*tgt_attr), GFP_KERNEL);
+ if (tgt_attr == NULL) {
+ PRINT_ERROR("Unable to allocate user (size %zd)",
+ sizeof(*tgt_attr));
+ res = -ENOMEM;
+ goto out;
+ }
+
+ tgt_attr->target = target;
+
+ tgt_attr->name = kstrdup(attr_info->name, GFP_KERNEL);
+ if (tgt_attr->name == NULL) {
+ PRINT_ERROR("Unable to allocate attr %s name/value (target %s)",
+ attr_info->name, name);
+ res = -ENOMEM;
+ goto out_free;
+ }
+
+ list_add(&tgt_attr->attrs_list_entry, attrs_list);
+
+ tgt_attr->attr.attr.name = tgt_attr->name;
+ tgt_attr->attr.attr.owner = THIS_MODULE;
+ tgt_attr->attr.attr.mode = attr_info->mode & (S_IRUGO | S_IWUGO);
+ tgt_attr->attr.show = iscsi_attr_show;
+ tgt_attr->attr.store = iscsi_attr_store;
+
+ res = sysfs_create_file(
+ (target != NULL) ? scst_sysfs_get_tgt_kobj(target->scst_tgt) :
+ scst_sysfs_get_tgtt_kobj(&iscsi_template),
+ &tgt_attr->attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Unable to create file '%s' for target '%s'",
+ tgt_attr->attr.attr.name, name);
+ goto out_del;
+ }
+
+out:
+ return res;
+
+out_del:
+ list_del(&tgt_attr->attrs_list_entry);
+
+out_free:
+ kfree(tgt_attr->name);
+ kfree(tgt_attr);
+ goto out;
+}
+
+void __iscsi_del_attr(struct iscsi_target *target,
+ struct iscsi_attr *tgt_attr)
+{
+
+ TRACE_DBG("Deleting %s's attr %s",
+ (target != NULL) ? target->name : "global", tgt_attr->name);
+
+ list_del(&tgt_attr->attrs_list_entry);
+
+ sysfs_remove_file((target != NULL) ?
+ scst_sysfs_get_tgt_kobj(target->scst_tgt) :
+ scst_sysfs_get_tgtt_kobj(&iscsi_template),
+ &tgt_attr->attr.attr);
+
+ kfree(tgt_attr->name);
+ kfree(tgt_attr);
+ return;
+}
+
+/*
+ * target_mgmt_mutex supposed to be locked. If target != 0, target_mutex
+ * supposed to be locked as well.
+ */
+static int iscsi_del_attr(struct iscsi_target *target,
+ const char *attr_name)
+{
+ int res = 0;
+ struct iscsi_attr *tgt_attr, *a;
+ struct list_head *attrs_list;
+
+ if (target != NULL)
+ attrs_list = &target->attrs_list;
+ else
+ attrs_list = &iscsi_attrs_list;
+
+ tgt_attr = NULL;
+ list_for_each_entry(a, attrs_list, attrs_list_entry) {
+ if (strncmp(a->name, attr_name, sizeof(a->name)) == 0) {
+ tgt_attr = a;
+ break;
+ }
+ }
+
+ if (tgt_attr == NULL) {
+ PRINT_ERROR("attr %s not found (target %s)", attr_name,
+ (target != NULL) ? target->name : "global");
+ res = -ENOENT;
+ goto out;
+ }
+
+ __iscsi_del_attr(target, tgt_attr);
+
+out:
+ return res;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int iscsi_attr_cmd(void __user *ptr, unsigned int cmd)
+{
+ int rc, err = 0;
+ struct iscsi_kern_attr_info info;
+ struct iscsi_target *target;
+ struct scst_sysfs_user_info *i = NULL;
+
+ rc = copy_from_user(&info, ptr, sizeof(info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out;
+ }
+
+ info.attr.name[sizeof(info.attr.name)-1] = '\0';
+
+ if (info.cookie != 0) {
+ i = scst_sysfs_user_get_info(info.cookie);
+ TRACE_DBG("cookie %u, uinfo %p", info.cookie, i);
+ if (i == NULL) {
+ err = -EINVAL;
+ goto out;
+ }
+ }
+
+ target = target_lookup_by_id(info.tid);
+
+ if (target != NULL)
+ mutex_lock(&target->target_mutex);
+
+ switch (cmd) {
+ case ISCSI_ATTR_ADD:
+ err = iscsi_add_attr(target, &info.attr);
+ break;
+ case ISCSI_ATTR_DEL:
+ err = iscsi_del_attr(target, info.attr.name);
+ break;
+ default:
+ BUG();
+ }
+
+ if (target != NULL)
+ mutex_unlock(&target->target_mutex);
+
+ if (i != NULL) {
+ i->info_status = err;
+ complete(&i->info_completion);
+ }
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int add_target(void __user *ptr)
+{
+ int err, rc;
+ struct iscsi_kern_target_info *info;
+ struct scst_sysfs_user_info *uinfo;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (info == NULL) {
+ PRINT_ERROR("Can't alloc info (size %zd)", sizeof(*info));
+ err = -ENOMEM;
+ goto out;
+ }
+
+ rc = copy_from_user(info, ptr, sizeof(*info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out_free;
+ }
+
+ if (target_lookup_by_id(info->tid) != NULL) {
+ PRINT_ERROR("Target %u already exist!", info->tid);
+ err = -EEXIST;
+ goto out_free;
+ }
+
+ info->name[sizeof(info->name)-1] = '\0';
+
+ if (info->cookie != 0) {
+ uinfo = scst_sysfs_user_get_info(info->cookie);
+ TRACE_DBG("cookie %u, uinfo %p", info->cookie, uinfo);
+ if (uinfo == NULL) {
+ err = -EINVAL;
+ goto out_free;
+ }
+ } else
+ uinfo = NULL;
+
+ err = __add_target(info);
+
+ if (uinfo != NULL) {
+ uinfo->info_status = err;
+ complete(&uinfo->info_completion);
+ }
+
+out_free:
+ kfree(info);
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int del_target(void __user *ptr)
+{
+ int err, rc;
+ struct iscsi_kern_target_info info;
+ struct scst_sysfs_user_info *uinfo;
+
+ rc = copy_from_user(&info, ptr, sizeof(info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy %d user's bytes", rc);
+ err = -EFAULT;
+ goto out;
+ }
+
+ info.name[sizeof(info.name)-1] = '\0';
+
+ if (info.cookie != 0) {
+ uinfo = scst_sysfs_user_get_info(info.cookie);
+ TRACE_DBG("cookie %u, uinfo %p", info.cookie, uinfo);
+ if (uinfo == NULL) {
+ err = -EINVAL;
+ goto out;
+ }
+ } else
+ uinfo = NULL;
+
+ err = __del_target(info.tid);
+
+ if (uinfo != NULL) {
+ uinfo->info_status = err;
+ complete(&uinfo->info_completion);
+ }
+
+out:
+ return err;
+}
+
+static int iscsi_register(void __user *arg)
+{
+ struct iscsi_kern_register_info reg;
+ char ver[sizeof(ISCSI_SCST_INTERFACE_VERSION)+1];
+ int res, rc;
+
+ rc = copy_from_user(®, arg, sizeof(reg));
+ if (rc != 0) {
+ PRINT_ERROR("%s", "Unable to get register info");
+ res = -EFAULT;
+ goto out;
+ }
+
+ rc = copy_from_user(ver, (void __user *)(unsigned long)reg.version,
+ sizeof(ver));
+ if (rc != 0) {
+ PRINT_ERROR("%s", "Unable to get version string");
+ res = -EFAULT;
+ goto out;
+ }
+ ver[sizeof(ver)-1] = '\0';
+
+ if (strcmp(ver, ISCSI_SCST_INTERFACE_VERSION) != 0) {
+ PRINT_ERROR("Incorrect version of user space %s (expected %s)",
+ ver, ISCSI_SCST_INTERFACE_VERSION);
+ res = -EINVAL;
+ goto out;
+ }
+
+ memset(®, 0, sizeof(reg));
+ reg.max_data_seg_len = ISCSI_CONN_IOV_MAX << PAGE_SHIFT;
+ reg.max_queued_cmds = scst_get_max_lun_commands(NULL, NO_SUCH_LUN);
+
+ res = 0;
+
+ rc = copy_to_user(arg, ®, sizeof(reg));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy to user %d bytes", rc);
+ res = -EFAULT;
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+static long ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ long err;
+
+ if (cmd == REGISTER_USERD) {
+ err = iscsi_register((void __user *)arg);
+ goto out;
+ }
+
+ err = mutex_lock_interruptible(&target_mgmt_mutex);
+ if (err < 0)
+ goto out;
+
+ switch (cmd) {
+ case ADD_TARGET:
+ err = add_target((void __user *)arg);
+ break;
+
+ case DEL_TARGET:
+ err = del_target((void __user *)arg);
+ break;
+
+ case ISCSI_ATTR_ADD:
+ case ISCSI_ATTR_DEL:
+ err = iscsi_attr_cmd((void __user *)arg, cmd);
+ break;
+
+ case MGMT_CMD_CALLBACK:
+ err = mgmt_cmd_callback((void __user *)arg);
+ break;
+
+ case ADD_SESSION:
+ err = add_session((void __user *)arg);
+ break;
+
+ case DEL_SESSION:
+ err = del_session((void __user *)arg);
+ break;
+
+ case ISCSI_PARAM_SET:
+ err = iscsi_params_config((void __user *)arg, 1);
+ break;
+
+ case ISCSI_PARAM_GET:
+ err = iscsi_params_config((void __user *)arg, 0);
+ break;
+
+ case ADD_CONN:
+ err = add_conn((void __user *)arg);
+ break;
+
+ case DEL_CONN:
+ err = del_conn((void __user *)arg);
+ break;
+
+ default:
+ PRINT_ERROR("Invalid ioctl cmd %x", cmd);
+ err = -EINVAL;
+ goto out_unlock;
+ }
+
+out_unlock:
+ mutex_unlock(&target_mgmt_mutex);
+
+out:
+ return err;
+}
+
+static int open(struct inode *inode, struct file *file)
+{
+ bool already;
+
+ mutex_lock(&target_mgmt_mutex);
+ already = (ctr_open_state != ISCSI_CTR_OPEN_STATE_CLOSED);
+ if (!already)
+ ctr_open_state = ISCSI_CTR_OPEN_STATE_OPEN;
+ mutex_unlock(&target_mgmt_mutex);
+
+ if (already) {
+ PRINT_WARNING("%s", "Attempt to second open the control "
+ "device!");
+ return -EBUSY;
+ } else
+ return 0;
+}
+
+static int release(struct inode *inode, struct file *filp)
+{
+ struct iscsi_attr *attr, *t;
+
+ TRACE(TRACE_MGMT, "%s", "Releasing allocated resources");
+
+ mutex_lock(&target_mgmt_mutex);
+ ctr_open_state = ISCSI_CTR_OPEN_STATE_CLOSING;
+ mutex_unlock(&target_mgmt_mutex);
+
+ target_del_all();
+
+ mutex_lock(&target_mgmt_mutex);
+
+ list_for_each_entry_safe(attr, t, &iscsi_attrs_list,
+ attrs_list_entry) {
+ __iscsi_del_attr(NULL, attr);
+ }
+
+ ctr_open_state = ISCSI_CTR_OPEN_STATE_CLOSED;
+
+ mutex_unlock(&target_mgmt_mutex);
+
+ return 0;
+}
+
+const struct file_operations ctr_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = ioctl,
+ .compat_ioctl = ioctl,
+ .open = open,
+ .release = release,
+};
+
+#ifdef CONFIG_SCST_DEBUG
+static void iscsi_dump_char(int ch, unsigned char *text, int *pos)
+{
+ int i = *pos;
+
+ if (ch < 0) {
+ while ((i % 16) != 0) {
+ printk(KERN_CONT " ");
+ text[i] = ' ';
+ i++;
+ if ((i % 16) == 0)
+ printk(KERN_CONT " | %.16s |\n", text);
+ else if ((i % 4) == 0)
+ printk(KERN_CONT " |");
+ }
+ i = 0;
+ goto out;
+ }
+
+ text[i] = (ch < 0x20 || (ch >= 0x80 && ch <= 0xa0)) ? ' ' : ch;
+ printk(KERN_CONT " %02x", ch);
+ i++;
+ if ((i % 16) == 0) {
+ printk(KERN_CONT " | %.16s |\n", text);
+ i = 0;
+ } else if ((i % 4) == 0)
+ printk(KERN_CONT " |");
+
+out:
+ *pos = i;
+ return;
+}
+
+void iscsi_dump_pdu(struct iscsi_pdu *pdu)
+{
+ unsigned char text[16];
+ int pos = 0;
+
+ if (trace_flag & TRACE_D_DUMP_PDU) {
+ unsigned char *buf;
+ int i;
+
+ buf = (void *)&pdu->bhs;
+ printk(KERN_DEBUG "BHS: (%p,%zd)\n", buf, sizeof(pdu->bhs));
+ for (i = 0; i < (int)sizeof(pdu->bhs); i++)
+ iscsi_dump_char(*buf++, text, &pos);
+ iscsi_dump_char(-1, text, &pos);
+
+ buf = (void *)pdu->ahs;
+ printk(KERN_DEBUG "AHS: (%p,%d)\n", buf, pdu->ahssize);
+ for (i = 0; i < pdu->ahssize; i++)
+ iscsi_dump_char(*buf++, text, &pos);
+ iscsi_dump_char(-1, text, &pos);
+
+ printk(KERN_DEBUG "Data: (%d)\n", pdu->datasize);
+ }
+}
+
+unsigned long iscsi_get_flow_ctrl_or_mgmt_dbg_log_flag(struct iscsi_cmnd *cmnd)
+{
+ unsigned long flag;
+
+ if (cmnd->cmd_req != NULL)
+ cmnd = cmnd->cmd_req;
+
+ if (cmnd->scst_cmd == NULL)
+ flag = TRACE_MGMT_DEBUG;
+ else {
+ int status = scst_cmd_get_status(cmnd->scst_cmd);
+ if ((status == SAM_STAT_TASK_SET_FULL) ||
+ (status == SAM_STAT_BUSY))
+ flag = TRACE_FLOW_CONTROL;
+ else
+ flag = TRACE_MGMT_DEBUG;
+ }
+ return flag;
+}
+
+#endif /* CONFIG_SCST_DEBUG */
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/conn.c linux-2.6.33/drivers/scst/iscsi-scst/conn.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/conn.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/conn.c
@@ -0,0 +1,785 @@
+/*
+ * Copyright (C) 2002 - 2003 Ardis Technolgies <roman@ardistech.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/file.h>
+#include <linux/ip.h>
+#include <net/tcp.h>
+
+#include "iscsi.h"
+#include "digest.h"
+
+static int print_conn_state(char *p, size_t size, struct iscsi_conn *conn)
+{
+ int pos = 0;
+
+ if (conn->closing) {
+ pos += scnprintf(p, size, "%s", "closing");
+ goto out;
+ }
+
+ switch (conn->rd_state) {
+ case ISCSI_CONN_RD_STATE_PROCESSING:
+ pos += scnprintf(&p[pos], size - pos, "%s", "read_processing ");
+ break;
+ case ISCSI_CONN_RD_STATE_IN_LIST:
+ pos += scnprintf(&p[pos], size - pos, "%s", "in_read_list ");
+ break;
+ }
+
+ switch (conn->wr_state) {
+ case ISCSI_CONN_WR_STATE_PROCESSING:
+ pos += scnprintf(&p[pos], size - pos, "%s", "write_processing ");
+ break;
+ case ISCSI_CONN_WR_STATE_IN_LIST:
+ pos += scnprintf(&p[pos], size - pos, "%s", "in_write_list ");
+ break;
+ case ISCSI_CONN_WR_STATE_SPACE_WAIT:
+ pos += scnprintf(&p[pos], size - pos, "%s", "space_waiting ");
+ break;
+ }
+
+ if (test_bit(ISCSI_CONN_REINSTATING, &conn->conn_aflags))
+ pos += scnprintf(&p[pos], size - pos, "%s", "reinstating ");
+ else if (pos == 0)
+ pos += scnprintf(&p[pos], size - pos, "%s", "established idle ");
+
+out:
+ return pos;
+}
+
+static int conn_free(struct iscsi_conn *conn);
+
+static void iscsi_conn_release(struct kobject *kobj)
+{
+ struct iscsi_conn *conn;
+ struct iscsi_target *target;
+
+ conn = container_of(kobj, struct iscsi_conn, iscsi_conn_kobj);
+ target = conn->target;
+
+ mutex_lock(&target->target_mutex);
+ conn_free(conn);
+ mutex_unlock(&target->target_mutex);
+ return;
+}
+
+static struct kobj_type iscsi_conn_ktype = {
+ .sysfs_ops = &scst_sysfs_ops,
+ .release = iscsi_conn_release,
+};
+
+static ssize_t iscsi_get_initiator_ip(struct iscsi_conn *conn,
+ char *buf, int size)
+{
+ int pos;
+ struct sock *sk;
+
+ sk = conn->sock->sk;
+ switch (sk->sk_family) {
+ case AF_INET:
+ pos = scnprintf(buf, size,
+ "%u.%u.%u.%u", NIPQUAD(inet_sk(sk)->inet_daddr));
+ break;
+ case AF_INET6:
+ pos = scnprintf(buf, size, "[%p6]",
+ &inet6_sk(sk)->daddr);
+ break;
+ default:
+ pos = scnprintf(buf, size, "Unknown family %d",
+ sk->sk_family);
+ break;
+ }
+ return pos;
+}
+
+static ssize_t iscsi_conn_ip_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct iscsi_conn *conn;
+
+ conn = container_of(kobj, struct iscsi_conn, iscsi_conn_kobj);
+
+ pos = iscsi_get_initiator_ip(conn, buf, SCST_SYSFS_BLOCK_SIZE);
+ return pos;
+}
+
+static struct kobj_attribute iscsi_conn_ip_attr =
+ __ATTR(ip, S_IRUGO, iscsi_conn_ip_show, NULL);
+
+static ssize_t iscsi_conn_cid_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct iscsi_conn *conn;
+
+ conn = container_of(kobj, struct iscsi_conn, iscsi_conn_kobj);
+
+ pos = sprintf(buf, "%u", conn->cid);
+ return pos;
+}
+
+static struct kobj_attribute iscsi_conn_cid_attr =
+ __ATTR(cid, S_IRUGO, iscsi_conn_cid_show, NULL);
+
+static ssize_t iscsi_conn_state_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct iscsi_conn *conn;
+
+ conn = container_of(kobj, struct iscsi_conn, iscsi_conn_kobj);
+
+ pos = print_conn_state(buf, SCST_SYSFS_BLOCK_SIZE, conn);
+ return pos;
+}
+
+static struct kobj_attribute iscsi_conn_state_attr =
+ __ATTR(state, S_IRUGO, iscsi_conn_state_show, NULL);
+
+/* target_mutex supposed to be locked */
+struct iscsi_conn *conn_lookup(struct iscsi_session *session, u16 cid)
+{
+ struct iscsi_conn *conn;
+
+ /*
+ * We need to find the latest conn to correctly handle
+ * multi-reinstatements
+ */
+ list_for_each_entry_reverse(conn, &session->conn_list,
+ conn_list_entry) {
+ if (conn->cid == cid)
+ return conn;
+ }
+ return NULL;
+}
+
+void iscsi_make_conn_rd_active(struct iscsi_conn *conn)
+{
+
+ spin_lock_bh(&iscsi_rd_lock);
+
+ TRACE_DBG("conn %p, rd_state %x, rd_data_ready %d", conn,
+ conn->rd_state, conn->rd_data_ready);
+
+ conn->rd_data_ready = 1;
+
+ if (conn->rd_state == ISCSI_CONN_RD_STATE_IDLE) {
+ list_add_tail(&conn->rd_list_entry, &iscsi_rd_list);
+ conn->rd_state = ISCSI_CONN_RD_STATE_IN_LIST;
+ wake_up(&iscsi_rd_waitQ);
+ }
+
+ spin_unlock_bh(&iscsi_rd_lock);
+ return;
+}
+
+void iscsi_make_conn_wr_active(struct iscsi_conn *conn)
+{
+
+ spin_lock_bh(&iscsi_wr_lock);
+
+ TRACE_DBG("conn %p, wr_state %x, wr_space_ready %d", conn,
+ conn->wr_state, conn->wr_space_ready);
+
+ if (conn->wr_state == ISCSI_CONN_WR_STATE_IDLE) {
+ list_add_tail(&conn->wr_list_entry, &iscsi_wr_list);
+ conn->wr_state = ISCSI_CONN_WR_STATE_IN_LIST;
+ wake_up(&iscsi_wr_waitQ);
+ }
+
+ spin_unlock_bh(&iscsi_wr_lock);
+ return;
+}
+
+void __mark_conn_closed(struct iscsi_conn *conn, int flags)
+{
+ spin_lock_bh(&iscsi_rd_lock);
+ conn->closing = 1;
+ if (flags & ISCSI_CONN_ACTIVE_CLOSE)
+ conn->active_close = 1;
+ if (flags & ISCSI_CONN_DELETING)
+ conn->deleting = 1;
+ spin_unlock_bh(&iscsi_rd_lock);
+
+ iscsi_make_conn_rd_active(conn);
+}
+
+void mark_conn_closed(struct iscsi_conn *conn)
+{
+ __mark_conn_closed(conn, ISCSI_CONN_ACTIVE_CLOSE);
+}
+
+static void __iscsi_state_change(struct sock *sk)
+{
+ struct iscsi_conn *conn = sk->sk_user_data;
+
+ if (unlikely(sk->sk_state != TCP_ESTABLISHED)) {
+ if (!conn->closing) {
+ PRINT_ERROR("Connection with initiator %s "
+ "unexpectedly closed!",
+ conn->session->initiator_name);
+ TRACE_MGMT_DBG("conn %p, sk state %d", conn,
+ sk->sk_state);
+ __mark_conn_closed(conn, 0);
+ }
+ } else
+ iscsi_make_conn_rd_active(conn);
+ return;
+}
+
+static void iscsi_state_change(struct sock *sk)
+{
+ struct iscsi_conn *conn = sk->sk_user_data;
+
+ __iscsi_state_change(sk);
+ conn->old_state_change(sk);
+
+ return;
+}
+
+static void iscsi_data_ready(struct sock *sk, int len)
+{
+ struct iscsi_conn *conn = sk->sk_user_data;
+
+ iscsi_make_conn_rd_active(conn);
+
+ conn->old_data_ready(sk, len);
+ return;
+}
+
+static void iscsi_write_space_ready(struct sock *sk)
+{
+ struct iscsi_conn *conn = sk->sk_user_data;
+
+ TRACE_DBG("Write space ready for conn %p", conn);
+
+ spin_lock_bh(&iscsi_wr_lock);
+ conn->wr_space_ready = 1;
+ if ((conn->wr_state == ISCSI_CONN_WR_STATE_SPACE_WAIT)) {
+ list_add_tail(&conn->wr_list_entry, &iscsi_wr_list);
+ conn->wr_state = ISCSI_CONN_WR_STATE_IN_LIST;
+ wake_up(&iscsi_wr_waitQ);
+ }
+ spin_unlock_bh(&iscsi_wr_lock);
+
+ conn->old_write_space(sk);
+ return;
+}
+
+static void conn_rsp_timer_fn(unsigned long arg)
+{
+ struct iscsi_conn *conn = (struct iscsi_conn *)arg;
+ struct iscsi_cmnd *cmnd;
+ unsigned long j = jiffies;
+
+ TRACE_DBG("Timer (conn %p)", conn);
+
+ spin_lock_bh(&conn->write_list_lock);
+
+ if (!list_empty(&conn->write_timeout_list)) {
+ unsigned long timeout_time;
+ cmnd = list_entry(conn->write_timeout_list.next,
+ struct iscsi_cmnd, write_timeout_list_entry);
+
+ timeout_time = j + conn->rsp_timeout + ISCSI_ADD_SCHED_TIME;
+
+ if (unlikely(time_after_eq(j, cmnd->write_start +
+ conn->rsp_timeout))) {
+ if (!conn->closing) {
+ PRINT_ERROR("Timeout sending data/waiting "
+ "for reply to/from initiator "
+ "%s (SID %llx), closing connection",
+ conn->session->initiator_name,
+ (long long unsigned int)
+ conn->session->sid);
+ /*
+ * We must call mark_conn_closed() outside of
+ * write_list_lock or we will have a circular
+ * locking dependency with iscsi_rd_lock.
+ */
+ spin_unlock_bh(&conn->write_list_lock);
+ mark_conn_closed(conn);
+ goto out;
+ }
+ } else if (!timer_pending(&conn->rsp_timer) ||
+ time_after(conn->rsp_timer.expires, timeout_time)) {
+ TRACE_DBG("Restarting timer on %ld (conn %p)",
+ timeout_time, conn);
+ /*
+ * Timer might have been restarted while we were
+ * entering here.
+ */
+ mod_timer(&conn->rsp_timer, timeout_time);
+ }
+ }
+
+ spin_unlock_bh(&conn->write_list_lock);
+
+ if (unlikely(conn->conn_tm_active)) {
+ TRACE_MGMT_DBG("TM active: making conn %p RD active", conn);
+ iscsi_make_conn_rd_active(conn);
+ }
+
+out:
+ return;
+}
+
+static void conn_nop_in_delayed_work_fn(struct delayed_work *work)
+{
+ struct iscsi_conn *conn = container_of(work, struct iscsi_conn,
+ nop_in_delayed_work);
+
+ if (time_after_eq(jiffies, conn->last_rcv_time +
+ conn->nop_in_interval)) {
+ iscsi_send_nop_in(conn);
+ }
+
+ if (conn->nop_in_interval > 0) {
+ TRACE_DBG("Reschedule Nop-In work for conn %p", conn);
+ schedule_delayed_work(&conn->nop_in_delayed_work,
+ conn->nop_in_interval + ISCSI_ADD_SCHED_TIME);
+ }
+ return;
+}
+
+/* Must be called from rd thread only */
+void iscsi_check_tm_data_wait_timeouts(struct iscsi_conn *conn, bool force)
+{
+ struct iscsi_cmnd *cmnd;
+ unsigned long j = jiffies;
+ bool aborted_cmds_pending;
+ unsigned long timeout_time = j + ISCSI_TM_DATA_WAIT_TIMEOUT +
+ ISCSI_ADD_SCHED_TIME;
+
+ TRACE_DBG_FLAG(force ? TRACE_CONN_OC_DBG : TRACE_MGMT_DEBUG,
+ "j %ld (TIMEOUT %d, force %d)", j,
+ ISCSI_TM_DATA_WAIT_TIMEOUT + ISCSI_ADD_SCHED_TIME, force);
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+again:
+ spin_lock_bh(&iscsi_rd_lock);
+ spin_lock(&conn->write_list_lock);
+
+ aborted_cmds_pending = false;
+ list_for_each_entry(cmnd, &conn->write_timeout_list,
+ write_timeout_list_entry) {
+ if (test_bit(ISCSI_CMD_ABORTED, &cmnd->prelim_compl_flags)) {
+ TRACE_DBG_FLAG(force ? TRACE_CONN_OC_DBG : TRACE_MGMT_DEBUG,
+ "Checking aborted cmnd %p (scst_state %d, "
+ "on_write_timeout_list %d, write_start %ld, "
+ "r2t_len_to_receive %d)", cmnd,
+ cmnd->scst_state, cmnd->on_write_timeout_list,
+ cmnd->write_start, cmnd->r2t_len_to_receive);
+ if ((cmnd->r2t_len_to_receive != 0) &&
+ (time_after_eq(j, cmnd->write_start + ISCSI_TM_DATA_WAIT_TIMEOUT) ||
+ force)) {
+ spin_unlock(&conn->write_list_lock);
+ spin_unlock_bh(&iscsi_rd_lock);
+ iscsi_fail_data_waiting_cmnd(cmnd);
+ goto again;
+ }
+ aborted_cmds_pending = true;
+ }
+ }
+
+ if (aborted_cmds_pending) {
+ if (!force &&
+ (!timer_pending(&conn->rsp_timer) ||
+ time_after(conn->rsp_timer.expires, timeout_time))) {
+ TRACE_MGMT_DBG("Mod timer on %ld (conn %p)",
+ timeout_time, conn);
+ mod_timer(&conn->rsp_timer, timeout_time);
+ }
+ } else {
+ TRACE_MGMT_DBG("Clearing conn_tm_active for conn %p", conn);
+ conn->conn_tm_active = 0;
+ }
+
+ spin_unlock(&conn->write_list_lock);
+ spin_unlock_bh(&iscsi_rd_lock);
+ return;
+}
+
+/* target_mutex supposed to be locked */
+void conn_reinst_finished(struct iscsi_conn *conn)
+{
+ struct iscsi_cmnd *cmnd, *t;
+
+ clear_bit(ISCSI_CONN_REINSTATING, &conn->conn_aflags);
+
+ list_for_each_entry_safe(cmnd, t, &conn->reinst_pending_cmd_list,
+ reinst_pending_cmd_list_entry) {
+ TRACE_MGMT_DBG("Restarting reinst pending cmnd %p",
+ cmnd);
+ list_del(&cmnd->reinst_pending_cmd_list_entry);
+ iscsi_restart_cmnd(cmnd);
+ }
+ return;
+}
+
+static void conn_activate(struct iscsi_conn *conn)
+{
+ TRACE_MGMT_DBG("Enabling conn %p", conn);
+
+ /* Catch double bind */
+ BUG_ON(conn->sock->sk->sk_state_change == iscsi_state_change);
+
+ write_lock_bh(&conn->sock->sk->sk_callback_lock);
+
+ conn->old_state_change = conn->sock->sk->sk_state_change;
+ conn->sock->sk->sk_state_change = iscsi_state_change;
+
+ conn->old_data_ready = conn->sock->sk->sk_data_ready;
+ conn->sock->sk->sk_data_ready = iscsi_data_ready;
+
+ conn->old_write_space = conn->sock->sk->sk_write_space;
+ conn->sock->sk->sk_write_space = iscsi_write_space_ready;
+
+ write_unlock_bh(&conn->sock->sk->sk_callback_lock);
+
+ /*
+ * Check, if conn was closed while we were initializing it.
+ * This function will make conn rd_active, if necessary.
+ */
+ __iscsi_state_change(conn->sock->sk);
+
+ return;
+}
+
+/*
+ * Note: the code below passes a kernel space pointer (&opt) to setsockopt()
+ * while the declaration of setsockopt specifies that it expects a user space
+ * pointer. This seems to work fine, and this approach is also used in some
+ * other parts of the Linux kernel (see e.g. fs/ocfs2/cluster/tcp.c).
+ */
+static int conn_setup_sock(struct iscsi_conn *conn)
+{
+ int res = 0;
+ int opt = 1;
+ mm_segment_t oldfs;
+ struct iscsi_session *session = conn->session;
+
+ TRACE_DBG("%llu", (long long unsigned int)session->sid);
+
+ conn->sock = SOCKET_I(conn->file->f_dentry->d_inode);
+
+ if (conn->sock->ops->sendpage == NULL) {
+ PRINT_ERROR("Socket for sid %llu doesn't support sendpage()",
+ (long long unsigned int)session->sid);
+ res = -EINVAL;
+ goto out;
+ }
+
+#if 0
+ conn->sock->sk->sk_allocation = GFP_NOIO;
+#endif
+ conn->sock->sk->sk_user_data = conn;
+
+ oldfs = get_fs();
+ set_fs(get_ds());
+ conn->sock->ops->setsockopt(conn->sock, SOL_TCP, TCP_NODELAY,
+ (void __force __user *)&opt, sizeof(opt));
+ set_fs(oldfs);
+
+out:
+ return res;
+}
+
+/* target_mutex supposed to be locked */
+static int conn_free(struct iscsi_conn *conn)
+{
+ struct iscsi_session *session = conn->session;
+
+ TRACE_MGMT_DBG("Freeing conn %p (sess=%p, %#Lx %u)", conn,
+ session, (long long unsigned int)session->sid, conn->cid);
+
+ del_timer_sync(&conn->rsp_timer);
+
+ BUG_ON(atomic_read(&conn->conn_ref_cnt) != 0);
+ BUG_ON(!list_empty(&conn->cmd_list));
+ BUG_ON(!list_empty(&conn->write_list));
+ BUG_ON(!list_empty(&conn->write_timeout_list));
+ BUG_ON(conn->conn_reinst_successor != NULL);
+ BUG_ON(!test_bit(ISCSI_CONN_SHUTTINGDOWN, &conn->conn_aflags));
+
+ /* Just in case if new conn gets freed before the old one */
+ if (test_bit(ISCSI_CONN_REINSTATING, &conn->conn_aflags)) {
+ struct iscsi_conn *c;
+ TRACE_MGMT_DBG("Freeing being reinstated conn %p", conn);
+ list_for_each_entry(c, &session->conn_list,
+ conn_list_entry) {
+ if (c->conn_reinst_successor == conn) {
+ c->conn_reinst_successor = NULL;
+ break;
+ }
+ }
+ }
+
+ list_del(&conn->conn_list_entry);
+
+ fput(conn->file);
+ conn->file = NULL;
+ conn->sock = NULL;
+
+ free_page((unsigned long)conn->read_iov);
+
+ kfree(conn);
+
+ if (list_empty(&session->conn_list)) {
+ BUG_ON(session->sess_reinst_successor != NULL);
+ session_free(session, true);
+ }
+
+ return 0;
+}
+
+/* target_mutex supposed to be locked */
+static int iscsi_conn_alloc(struct iscsi_session *session,
+ struct iscsi_kern_conn_info *info, struct iscsi_conn **new_conn)
+{
+ struct iscsi_conn *conn;
+ int res = 0;
+ struct iscsi_conn *c;
+ int n = 1;
+ char addr[64];
+
+ conn = kzalloc(sizeof(*conn), GFP_KERNEL);
+ if (!conn) {
+ res = -ENOMEM;
+ goto out_err;
+ }
+
+ TRACE_MGMT_DBG("Creating connection %p for sid %#Lx, cid %u", conn,
+ (long long unsigned int)session->sid, info->cid);
+
+ /* Changing it, change ISCSI_CONN_IOV_MAX as well !! */
+ conn->read_iov = (struct iovec *)get_zeroed_page(GFP_KERNEL);
+ if (conn->read_iov == NULL) {
+ res = -ENOMEM;
+ goto out_err_free_conn;
+ }
+
+ atomic_set(&conn->conn_ref_cnt, 0);
+ conn->session = session;
+ if (session->sess_reinstating)
+ __set_bit(ISCSI_CONN_REINSTATING, &conn->conn_aflags);
+ conn->cid = info->cid;
+ conn->stat_sn = info->stat_sn;
+ conn->exp_stat_sn = info->exp_stat_sn;
+ conn->rd_state = ISCSI_CONN_RD_STATE_IDLE;
+ conn->wr_state = ISCSI_CONN_WR_STATE_IDLE;
+
+ conn->hdigest_type = session->sess_params.header_digest;
+ conn->ddigest_type = session->sess_params.data_digest;
+ res = digest_init(conn);
+ if (res != 0)
+ goto out_err_free1;
+
+ conn->target = session->target;
+ spin_lock_init(&conn->cmd_list_lock);
+ INIT_LIST_HEAD(&conn->cmd_list);
+ spin_lock_init(&conn->write_list_lock);
+ INIT_LIST_HEAD(&conn->write_list);
+ INIT_LIST_HEAD(&conn->write_timeout_list);
+ setup_timer(&conn->rsp_timer, conn_rsp_timer_fn, (unsigned long)conn);
+ init_waitqueue_head(&conn->read_state_waitQ);
+ init_completion(&conn->ready_to_free);
+ INIT_LIST_HEAD(&conn->reinst_pending_cmd_list);
+ INIT_LIST_HEAD(&conn->nop_req_list);
+ spin_lock_init(&conn->nop_req_list_lock);
+
+ conn->nop_in_ttt = 0;
+ INIT_DELAYED_WORK(&conn->nop_in_delayed_work,
+ (void (*)(struct work_struct *))conn_nop_in_delayed_work_fn);
+ conn->last_rcv_time = jiffies;
+ conn->rsp_timeout = session->tgt_params.rsp_timeout * HZ;
+ conn->nop_in_interval = session->tgt_params.nop_in_interval * HZ;
+ if (conn->nop_in_interval > 0) {
+ TRACE_DBG("Schedule Nop-In work for conn %p", conn);
+ schedule_delayed_work(&conn->nop_in_delayed_work,
+ conn->nop_in_interval + ISCSI_ADD_SCHED_TIME);
+ }
+
+ conn->file = fget(info->fd);
+
+ res = conn_setup_sock(conn);
+ if (res != 0)
+ goto out_err_free2;
+
+ iscsi_get_initiator_ip(conn, addr, sizeof(addr));
+
+restart:
+ list_for_each_entry(c, &session->conn_list, conn_list_entry) {
+ if (strcmp(addr, kobject_name(&conn->iscsi_conn_kobj)) == 0) {
+ char c_addr[64];
+
+ iscsi_get_initiator_ip(conn, c_addr, sizeof(c_addr));
+
+ TRACE_DBG("Duplicated conn from the same initiator "
+ "%s found", c_addr);
+
+ snprintf(addr, sizeof(addr), "%s_%d", c_addr, n);
+ n++;
+ goto restart;
+ }
+ }
+
+ res = kobject_init_and_add(&conn->iscsi_conn_kobj, &iscsi_conn_ktype,
+ scst_sysfs_get_sess_kobj(session->scst_sess), addr);
+ if (res != 0) {
+ PRINT_ERROR("Unable create sysfs entries for conn %s",
+ addr);
+ goto out_err_free2;
+ }
+
+ TRACE_DBG("conn %p, iscsi_conn_kobj %p", conn, &conn->iscsi_conn_kobj);
+
+ res = sysfs_create_file(&conn->iscsi_conn_kobj,
+ &iscsi_conn_state_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Unable create sysfs attribute %s for conn %s",
+ iscsi_conn_state_attr.attr.name, addr);
+ goto out_err_free3;
+ }
+
+ res = sysfs_create_file(&conn->iscsi_conn_kobj,
+ &iscsi_conn_cid_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Unable create sysfs attribute %s for conn %s",
+ iscsi_conn_cid_attr.attr.name, addr);
+ goto out_err_free3;
+ }
+
+ res = sysfs_create_file(&conn->iscsi_conn_kobj,
+ &iscsi_conn_ip_attr.attr);
+ if (res != 0) {
+ PRINT_ERROR("Unable create sysfs attribute %s for conn %s",
+ iscsi_conn_ip_attr.attr.name, addr);
+ goto out_err_free3;
+ }
+
+ list_add_tail(&conn->conn_list_entry, &session->conn_list);
+
+ *new_conn = conn;
+
+out:
+ return res;
+
+out_err_free3:
+ kobject_put(&conn->iscsi_conn_kobj);
+ goto out;
+
+out_err_free2:
+ fput(conn->file);
+
+out_err_free1:
+ free_page((unsigned long)conn->read_iov);
+
+out_err_free_conn:
+ kfree(conn);
+
+out_err:
+ goto out;
+}
+
+/* target_mutex supposed to be locked */
+int __add_conn(struct iscsi_session *session, struct iscsi_kern_conn_info *info)
+{
+ struct iscsi_conn *conn, *new_conn = NULL;
+ int err;
+ bool reinstatement = false;
+
+ conn = conn_lookup(session, info->cid);
+ if ((conn != NULL) &&
+ !test_bit(ISCSI_CONN_SHUTTINGDOWN, &conn->conn_aflags)) {
+ /* conn reinstatement */
+ reinstatement = true;
+ } else if (!list_empty(&session->conn_list)) {
+ err = -EEXIST;
+ goto out;
+ }
+
+ err = iscsi_conn_alloc(session, info, &new_conn);
+ if (err != 0)
+ goto out;
+
+ if (reinstatement) {
+ TRACE_MGMT_DBG("Reinstating conn (old %p, new %p)", conn,
+ new_conn);
+ conn->conn_reinst_successor = new_conn;
+ __set_bit(ISCSI_CONN_REINSTATING, &new_conn->conn_aflags);
+ __mark_conn_closed(conn, 0);
+ }
+
+ conn_activate(new_conn);
+
+out:
+ return err;
+}
+
+/* target_mutex supposed to be locked */
+int __del_conn(struct iscsi_session *session, struct iscsi_kern_conn_info *info)
+{
+ struct iscsi_conn *conn;
+ int err = -EEXIST;
+
+ conn = conn_lookup(session, info->cid);
+ if (!conn) {
+ PRINT_ERROR("Connection %d not found", info->cid);
+ return err;
+ }
+
+ PRINT_INFO("Deleting connection with initiator %s (%p)",
+ conn->session->initiator_name, conn);
+
+ __mark_conn_closed(conn, ISCSI_CONN_ACTIVE_CLOSE|ISCSI_CONN_DELETING);
+
+ return 0;
+}
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+
+void iscsi_extracheck_is_rd_thread(struct iscsi_conn *conn)
+{
+ if (unlikely(current != conn->rd_task)) {
+ printk(KERN_EMERG "conn %p rd_task != current %p (pid %d)\n",
+ conn, current, current->pid);
+ while (in_softirq())
+ local_bh_enable();
+ printk(KERN_EMERG "rd_state %x\n", conn->rd_state);
+ printk(KERN_EMERG "rd_task %p\n", conn->rd_task);
+ printk(KERN_EMERG "rd_task->pid %d\n", conn->rd_task->pid);
+ BUG();
+ }
+}
+
+void iscsi_extracheck_is_wr_thread(struct iscsi_conn *conn)
+{
+ if (unlikely(current != conn->wr_task)) {
+ printk(KERN_EMERG "conn %p wr_task != current %p (pid %d)\n",
+ conn, current, current->pid);
+ while (in_softirq())
+ local_bh_enable();
+ printk(KERN_EMERG "wr_state %x\n", conn->wr_state);
+ printk(KERN_EMERG "wr_task %p\n", conn->wr_task);
+ printk(KERN_EMERG "wr_task->pid %d\n", conn->wr_task->pid);
+ BUG();
+ }
+}
+
+#endif /* CONFIG_SCST_EXTRACHECKS */
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/digest.c linux-2.6.33/drivers/scst/iscsi-scst/digest.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/digest.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/digest.c
@@ -0,0 +1,226 @@
+/*
+ * iSCSI digest handling.
+ *
+ * Copyright (C) 2004 - 2006 Xiranet Communications GmbH
+ * <arne.redlich@xiranet.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/types.h>
+#include <linux/scatterlist.h>
+
+#include "iscsi.h"
+#include "digest.h"
+#include <linux/crc32c.h>
+
+void digest_alg_available(int *val)
+{
+#if defined(CONFIG_LIBCRC32C_MODULE) || defined(CONFIG_LIBCRC32C)
+ int crc32c = 1;
+#else
+ int crc32c = 0;
+#endif
+
+ if ((*val & DIGEST_CRC32C) && !crc32c) {
+ PRINT_ERROR("%s", "CRC32C digest algorithm not available "
+ "in kernel");
+ *val |= ~DIGEST_CRC32C;
+ }
+}
+
+/**
+ * initialize support for digest calculation.
+ *
+ * digest_init -
+ * @conn: ptr to connection to make use of digests
+ *
+ * @return: 0 on success, < 0 on error
+ */
+int digest_init(struct iscsi_conn *conn)
+{
+ if (!(conn->hdigest_type & DIGEST_ALL))
+ conn->hdigest_type = DIGEST_NONE;
+
+ if (!(conn->ddigest_type & DIGEST_ALL))
+ conn->ddigest_type = DIGEST_NONE;
+
+ return 0;
+}
+
+static u32 evaluate_crc32_from_sg(struct scatterlist *sg, int nbytes,
+ uint32_t padding)
+{
+ u32 crc = ~0;
+ int pad_bytes = ((nbytes + 3) & -4) - nbytes;
+
+#ifdef CONFIG_SCST_ISCSI_DEBUG_DIGEST_FAILURES
+ if (((scst_random() % 100000) == 752)) {
+ PRINT_INFO("%s", "Simulating digest failure");
+ return 0;
+ }
+#endif
+
+#if defined(CONFIG_LIBCRC32C_MODULE) || defined(CONFIG_LIBCRC32C)
+ while (nbytes > 0) {
+ int d = min(nbytes, (int)(sg->length));
+ crc = crc32c(crc, sg_virt(sg), d);
+ nbytes -= d;
+ sg++;
+ }
+
+ if (pad_bytes)
+ crc = crc32c(crc, (u8 *)&padding, pad_bytes);
+#endif
+
+ return ~cpu_to_le32(crc);
+}
+
+static u32 digest_header(struct iscsi_pdu *pdu)
+{
+ struct scatterlist sg[2];
+ unsigned int nbytes = sizeof(struct iscsi_hdr);
+ int asize = (pdu->ahssize + 3) & -4;
+
+ sg_init_table(sg, 2);
+
+ sg_set_buf(&sg[0], &pdu->bhs, nbytes);
+ if (pdu->ahssize) {
+ sg_set_buf(&sg[1], pdu->ahs, asize);
+ nbytes += asize;
+ }
+ EXTRACHECKS_BUG_ON((nbytes & 3) != 0);
+ return evaluate_crc32_from_sg(sg, nbytes, 0);
+}
+
+static u32 digest_data(struct iscsi_cmnd *cmd, u32 size, u32 offset,
+ uint32_t padding)
+{
+ struct scatterlist *sg = cmd->sg;
+ int idx, count;
+ struct scatterlist saved_sg;
+ u32 crc;
+
+ offset += sg[0].offset;
+ idx = offset >> PAGE_SHIFT;
+ offset &= ~PAGE_MASK;
+
+ count = get_pgcnt(size, offset);
+
+ TRACE_DBG("req %p, idx %d, count %d, sg_cnt %d, size %d, "
+ "offset %d", cmd, idx, count, cmd->sg_cnt, size, offset);
+ BUG_ON(idx + count > cmd->sg_cnt);
+
+ saved_sg = sg[idx];
+ sg[idx].offset = offset;
+ sg[idx].length -= offset - saved_sg.offset;
+
+ crc = evaluate_crc32_from_sg(sg + idx, size, padding);
+
+ sg[idx] = saved_sg;
+ return crc;
+}
+
+int digest_rx_header(struct iscsi_cmnd *cmnd)
+{
+ u32 crc;
+
+ crc = digest_header(&cmnd->pdu);
+ if (unlikely(crc != cmnd->hdigest)) {
+ PRINT_ERROR("%s", "RX header digest failed");
+ return -EIO;
+ } else
+ TRACE_DBG("RX header digest OK for cmd %p", cmnd);
+
+ return 0;
+}
+
+void digest_tx_header(struct iscsi_cmnd *cmnd)
+{
+ cmnd->hdigest = digest_header(&cmnd->pdu);
+ TRACE_DBG("TX header digest for cmd %p: %x", cmnd, cmnd->hdigest);
+}
+
+int digest_rx_data(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_cmnd *req;
+ struct iscsi_data_out_hdr *req_hdr;
+ u32 offset, crc;
+ int res = 0;
+
+ switch (cmnd_opcode(cmnd)) {
+ case ISCSI_OP_SCSI_DATA_OUT:
+ req = cmnd->cmd_req;
+ if (unlikely(req == NULL)) {
+ /* It can be for prelim completed commands */
+ req = cmnd;
+ goto out;
+ }
+ req_hdr = (struct iscsi_data_out_hdr *)&cmnd->pdu.bhs;
+ offset = be32_to_cpu(req_hdr->buffer_offset);
+ break;
+
+ default:
+ req = cmnd;
+ offset = 0;
+ }
+
+ /*
+ * We need to skip the digest check for prelim completed commands,
+ * because we use shared data buffer for them, so, most likely, the
+ * check will fail. Plus, for such commands we sometimes don't have
+ * sg_cnt set correctly (cmnd_prepare_get_rejected_cmd_data() doesn't
+ * do it).
+ */
+ if (unlikely(req->prelim_compl_flags != 0))
+ goto out;
+
+ crc = digest_data(req, cmnd->pdu.datasize, offset,
+ cmnd->conn->rpadding);
+
+ if (unlikely(crc != cmnd->ddigest)) {
+ PRINT_ERROR("%s", "RX data digest failed");
+ TRACE_MGMT_DBG("Calculated crc %x, ddigest %x, offset %d", crc,
+ cmnd->ddigest, offset);
+ iscsi_dump_pdu(&cmnd->pdu);
+ res = -EIO;
+ } else
+ TRACE_DBG("RX data digest OK for cmd %p", cmnd);
+
+out:
+ return res;
+}
+
+void digest_tx_data(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_data_in_hdr *hdr;
+ u32 offset;
+
+ TRACE_DBG("%s:%d req %p, own_sg %d, sg %p, sgcnt %d cmnd %p, "
+ "own_sg %d, sg %p, sgcnt %d", __func__, __LINE__,
+ cmnd->parent_req, cmnd->parent_req->own_sg,
+ cmnd->parent_req->sg, cmnd->parent_req->sg_cnt,
+ cmnd, cmnd->own_sg, cmnd->sg, cmnd->sg_cnt);
+
+ switch (cmnd_opcode(cmnd)) {
+ case ISCSI_OP_SCSI_DATA_IN:
+ hdr = (struct iscsi_data_in_hdr *)&cmnd->pdu.bhs;
+ offset = be32_to_cpu(hdr->buffer_offset);
+ break;
+ default:
+ offset = 0;
+ }
+
+ cmnd->ddigest = digest_data(cmnd, cmnd->pdu.datasize, offset, 0);
+ TRACE_DBG("TX data digest for cmd %p: %x (offset %d, opcode %x)", cmnd,
+ cmnd->ddigest, offset, cmnd_opcode(cmnd));
+}
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/event.c linux-2.6.33/drivers/scst/iscsi-scst/event.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/event.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/event.c
@@ -0,0 +1,163 @@
+/*
+ * Event notification code.
+ *
+ * Copyright (C) 2005 FUJITA Tomonori <tomof@acm.org>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Some functions are based on audit code.
+ */
+
+#include <net/tcp.h>
+#include "iscsi_scst.h"
+#include "iscsi.h"
+
+static struct sock *nl;
+static u32 iscsid_pid;
+
+static int event_recv_msg(struct sk_buff *skb, struct nlmsghdr *nlh)
+{
+ u32 uid, pid, seq;
+ char *data;
+
+ pid = NETLINK_CREDS(skb)->pid;
+ uid = NETLINK_CREDS(skb)->uid;
+ seq = nlh->nlmsg_seq;
+ data = NLMSG_DATA(nlh);
+
+ iscsid_pid = pid;
+
+ return 0;
+}
+
+static void event_recv_skb(struct sk_buff *skb)
+{
+ int err;
+ struct nlmsghdr *nlh;
+ u32 rlen;
+
+ while (skb->len >= NLMSG_SPACE(0)) {
+ nlh = (struct nlmsghdr *)skb->data;
+ if (nlh->nlmsg_len < sizeof(*nlh) || skb->len < nlh->nlmsg_len)
+ goto out;
+ rlen = NLMSG_ALIGN(nlh->nlmsg_len);
+ if (rlen > skb->len)
+ rlen = skb->len;
+ err = event_recv_msg(skb, nlh);
+ if (err)
+ netlink_ack(skb, nlh, -err);
+ else if (nlh->nlmsg_flags & NLM_F_ACK)
+ netlink_ack(skb, nlh, 0);
+ skb_pull(skb, rlen);
+ }
+
+out:
+ return;
+}
+
+/* event_mutex supposed to be held */
+static int __event_send(const void *buf, int buf_len)
+{
+ int res = 0, len;
+ struct sk_buff *skb;
+ struct nlmsghdr *nlh;
+ static u32 seq; /* protected by event_mutex */
+
+ if (ctr_open_state != ISCSI_CTR_OPEN_STATE_OPEN)
+ goto out;
+
+ len = NLMSG_SPACE(buf_len);
+
+ skb = alloc_skb(NLMSG_SPACE(len), GFP_KERNEL);
+ if (skb == NULL) {
+ PRINT_ERROR("alloc_skb() failed (len %d)", len);
+ res = -ENOMEM;
+ goto out;
+ }
+
+ nlh = __nlmsg_put(skb, iscsid_pid, seq++, NLMSG_DONE,
+ len - sizeof(*nlh), 0);
+
+ memcpy(NLMSG_DATA(nlh), buf, buf_len);
+ res = netlink_unicast(nl, skb, iscsid_pid, 0);
+ if (res <= 0) {
+ if (res != -ECONNREFUSED)
+ PRINT_ERROR("netlink_unicast() failed: %d", res);
+ else
+ TRACE(TRACE_MINOR, "netlink_unicast() failed: %s. "
+ "Not functioning user space?",
+ "Connection refused");
+ goto out;
+ }
+
+out:
+ return res;
+}
+
+int event_send(u32 tid, u64 sid, u32 cid, u32 cookie,
+ enum iscsi_kern_event_code code,
+ const char *param1, const char *param2)
+{
+ int err;
+ static DEFINE_MUTEX(event_mutex);
+ struct iscsi_kern_event event;
+ int param1_size, param2_size;
+
+ param1_size = (param1 != NULL) ? strlen(param1) : 0;
+ param2_size = (param2 != NULL) ? strlen(param2) : 0;
+
+ event.tid = tid;
+ event.sid = sid;
+ event.cid = cid;
+ event.code = code;
+ event.cookie = cookie;
+ event.param1_size = param1_size;
+ event.param2_size = param2_size;
+
+ mutex_lock(&event_mutex);
+
+ err = __event_send(&event, sizeof(event));
+ if (err <= 0)
+ goto out_unlock;
+
+ if (param1_size > 0) {
+ err = __event_send(param1, param1_size);
+ if (err <= 0)
+ goto out_unlock;
+ }
+
+ if (param2_size > 0) {
+ err = __event_send(param2, param2_size);
+ if (err <= 0)
+ goto out_unlock;
+ }
+
+out_unlock:
+ mutex_unlock(&event_mutex);
+ return err;
+}
+
+int __init event_init(void)
+{
+ nl = netlink_kernel_create(&init_net, NETLINK_ISCSI_SCST, 1,
+ event_recv_skb, NULL, THIS_MODULE);
+ if (!nl) {
+ PRINT_ERROR("%s", "netlink_kernel_create() failed");
+ return -ENOMEM;
+ } else
+ return 0;
+}
+
+void event_exit(void)
+{
+ netlink_kernel_release(nl);
+}
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi.c linux-2.6.33/drivers/scst/iscsi-scst/iscsi.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/iscsi.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/iscsi.c
@@ -0,0 +1,3583 @@
+/*
+ * Copyright (C) 2002 - 2003 Ardis Technolgies <roman@ardistech.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/hash.h>
+#include <linux/kthread.h>
+#include <linux/scatterlist.h>
+#include <net/tcp.h>
+#include <scsi/scsi.h>
+
+#include "iscsi.h"
+#include "digest.h"
+
+#define ISCSI_INIT_WRITE_WAKE 0x1
+
+static int ctr_major;
+static char ctr_name[] = "iscsi-scst-ctl";
+
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+unsigned long iscsi_trace_flag = ISCSI_DEFAULT_LOG_FLAGS;
+#endif
+
+static struct kmem_cache *iscsi_cmnd_cache;
+
+DEFINE_SPINLOCK(iscsi_rd_lock);
+LIST_HEAD(iscsi_rd_list);
+DECLARE_WAIT_QUEUE_HEAD(iscsi_rd_waitQ);
+
+DEFINE_SPINLOCK(iscsi_wr_lock);
+LIST_HEAD(iscsi_wr_list);
+DECLARE_WAIT_QUEUE_HEAD(iscsi_wr_waitQ);
+
+static struct page *dummy_page;
+static struct scatterlist dummy_sg;
+
+struct iscsi_thread_t {
+ struct task_struct *thr;
+ struct list_head threads_list_entry;
+};
+
+static LIST_HEAD(iscsi_threads_list);
+
+static void cmnd_remove_data_wait_hash(struct iscsi_cmnd *cmnd);
+static void iscsi_send_task_mgmt_resp(struct iscsi_cmnd *req, int status);
+static void iscsi_check_send_delayed_tm_resp(struct iscsi_session *sess);
+static void req_cmnd_release(struct iscsi_cmnd *req);
+static int iscsi_preliminary_complete(struct iscsi_cmnd *req,
+ struct iscsi_cmnd *orig_req, bool get_data);
+static int cmnd_insert_data_wait_hash(struct iscsi_cmnd *cmnd);
+static void __cmnd_abort(struct iscsi_cmnd *cmnd);
+static void iscsi_set_resid(struct iscsi_cmnd *rsp, bool bufflen_set);
+static void iscsi_cmnd_init_write(struct iscsi_cmnd *rsp, int flags);
+
+static void req_del_from_write_timeout_list(struct iscsi_cmnd *req)
+{
+ struct iscsi_conn *conn;
+
+ if (!req->on_write_timeout_list)
+ goto out;
+
+ conn = req->conn;
+
+ TRACE_DBG("Deleting cmd %p from conn %p write_timeout_list",
+ req, conn);
+
+ spin_lock_bh(&conn->write_list_lock);
+
+ /* Recheck, since it can be changed behind us */
+ if (unlikely(!req->on_write_timeout_list))
+ goto out_unlock;
+
+ list_del(&req->write_timeout_list_entry);
+ req->on_write_timeout_list = 0;
+
+out_unlock:
+ spin_unlock_bh(&conn->write_list_lock);
+
+out:
+ return;
+}
+
+static inline u32 cmnd_write_size(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_scsi_cmd_hdr *hdr = cmnd_hdr(cmnd);
+
+ if (hdr->flags & ISCSI_CMD_WRITE)
+ return be32_to_cpu(hdr->data_length);
+ return 0;
+}
+
+static inline int cmnd_read_size(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_scsi_cmd_hdr *hdr = cmnd_hdr(cmnd);
+
+ if (hdr->flags & ISCSI_CMD_READ) {
+ struct iscsi_ahs_hdr *ahdr;
+
+ if (!(hdr->flags & ISCSI_CMD_WRITE))
+ return be32_to_cpu(hdr->data_length);
+
+ ahdr = (struct iscsi_ahs_hdr *)cmnd->pdu.ahs;
+ if (ahdr != NULL) {
+ uint8_t *p = (uint8_t *)ahdr;
+ unsigned int size = 0;
+ do {
+ int s;
+
+ ahdr = (struct iscsi_ahs_hdr *)p;
+
+ if (ahdr->ahstype == ISCSI_AHSTYPE_RLENGTH) {
+ struct iscsi_rlength_ahdr *rh =
+ (struct iscsi_rlength_ahdr *)ahdr;
+ return be32_to_cpu(rh->read_length);
+ }
+
+ s = 3 + be16_to_cpu(ahdr->ahslength);
+ s = (s + 3) & -4;
+ size += s;
+ p += s;
+ } while (size < cmnd->pdu.ahssize);
+ }
+ return -1;
+ }
+ return 0;
+}
+
+void iscsi_restart_cmnd(struct iscsi_cmnd *cmnd)
+{
+ int status;
+
+ EXTRACHECKS_BUG_ON(cmnd->r2t_len_to_receive != 0);
+ EXTRACHECKS_BUG_ON(cmnd->r2t_len_to_send != 0);
+
+ req_del_from_write_timeout_list(cmnd);
+
+ /*
+ * Let's remove cmnd from the hash earlier to keep it smaller.
+ * See also corresponding comment in req_cmnd_release().
+ */
+ if (cmnd->hashed)
+ cmnd_remove_data_wait_hash(cmnd);
+
+ if (unlikely(test_bit(ISCSI_CONN_REINSTATING,
+ &cmnd->conn->conn_aflags))) {
+ struct iscsi_target *target = cmnd->conn->session->target;
+ bool get_out;
+
+ mutex_lock(&target->target_mutex);
+
+ get_out = test_bit(ISCSI_CONN_REINSTATING,
+ &cmnd->conn->conn_aflags);
+ /* Let's don't look dead */
+ if (scst_cmd_get_cdb(cmnd->scst_cmd)[0] == TEST_UNIT_READY)
+ get_out = false;
+
+ if (!get_out)
+ goto unlock_cont;
+
+ TRACE_MGMT_DBG("Pending cmnd %p, because conn %p is "
+ "reinstated", cmnd, cmnd->conn);
+
+ cmnd->scst_state = ISCSI_CMD_STATE_REINST_PENDING;
+ list_add_tail(&cmnd->reinst_pending_cmd_list_entry,
+ &cmnd->conn->reinst_pending_cmd_list);
+
+unlock_cont:
+ mutex_unlock(&target->target_mutex);
+
+ if (get_out)
+ goto out;
+ }
+
+ if (unlikely(cmnd->prelim_compl_flags != 0)) {
+ if (test_bit(ISCSI_CMD_ABORTED, &cmnd->prelim_compl_flags)) {
+ TRACE_MGMT_DBG("cmnd %p (scst_cmd %p) aborted", cmnd,
+ cmnd->scst_cmd);
+ req_cmnd_release_force(cmnd);
+ goto out;
+ }
+
+ if (cmnd->scst_cmd == NULL) {
+ TRACE_MGMT_DBG("Finishing preliminary completed cmd %p "
+ "with NULL scst_cmd", cmnd);
+ req_cmnd_release(cmnd);
+ goto out;
+ }
+
+ status = SCST_PREPROCESS_STATUS_ERROR_SENSE_SET;
+ } else
+ status = SCST_PREPROCESS_STATUS_SUCCESS;
+
+ cmnd->scst_state = ISCSI_CMD_STATE_RESTARTED;
+
+ scst_restart_cmd(cmnd->scst_cmd, status, SCST_CONTEXT_THREAD);
+
+out:
+ return;
+}
+
+void iscsi_fail_data_waiting_cmnd(struct iscsi_cmnd *cmnd)
+{
+
+ TRACE_MGMT_DBG("Failing data waiting cmnd %p", cmnd);
+
+ /*
+ * There is no race with conn_abort(), since all functions
+ * called from single read thread
+ */
+ iscsi_extracheck_is_rd_thread(cmnd->conn);
+ cmnd->r2t_len_to_receive = 0;
+ cmnd->r2t_len_to_send = 0;
+
+ req_cmnd_release_force(cmnd);
+ return;
+}
+
+struct iscsi_cmnd *cmnd_alloc(struct iscsi_conn *conn,
+ struct iscsi_cmnd *parent)
+{
+ struct iscsi_cmnd *cmnd;
+
+ /* ToDo: __GFP_NOFAIL?? */
+ cmnd = kmem_cache_zalloc(iscsi_cmnd_cache, GFP_KERNEL|__GFP_NOFAIL);
+
+ atomic_set(&cmnd->ref_cnt, 1);
+ cmnd->scst_state = ISCSI_CMD_STATE_NEW;
+ cmnd->conn = conn;
+ cmnd->parent_req = parent;
+
+ if (parent == NULL) {
+ conn_get(conn);
+
+ INIT_LIST_HEAD(&cmnd->rsp_cmd_list);
+ INIT_LIST_HEAD(&cmnd->rx_ddigest_cmd_list);
+ cmnd->target_task_tag = cpu_to_be32(ISCSI_RESERVED_TAG);
+
+ spin_lock_bh(&conn->cmd_list_lock);
+ list_add_tail(&cmnd->cmd_list_entry, &conn->cmd_list);
+ spin_unlock_bh(&conn->cmd_list_lock);
+ }
+
+ TRACE_DBG("conn %p, parent %p, cmnd %p", conn, parent, cmnd);
+ return cmnd;
+}
+
+/* Frees a command. Also frees the additional header. */
+static void cmnd_free(struct iscsi_cmnd *cmnd)
+{
+
+ TRACE_DBG("cmnd %p", cmnd);
+
+ if (unlikely(test_bit(ISCSI_CMD_ABORTED, &cmnd->prelim_compl_flags))) {
+ TRACE_MGMT_DBG("Free aborted cmd %p (scst cmd %p, state %d, "
+ "parent_req %p)", cmnd, cmnd->scst_cmd,
+ cmnd->scst_state, cmnd->parent_req);
+ }
+
+ /* Catch users from cmd_list or rsp_cmd_list */
+ EXTRACHECKS_BUG_ON(atomic_read(&cmnd->ref_cnt) != 0);
+
+ kfree(cmnd->pdu.ahs);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(cmnd->on_write_list || cmnd->on_write_timeout_list)) {
+ struct iscsi_scsi_cmd_hdr *req = cmnd_hdr(cmnd);
+
+ PRINT_CRIT_ERROR("cmnd %p still on some list?, %x, %x, %x, "
+ "%x, %x, %x, %x", cmnd, req->opcode, req->scb[0],
+ req->flags, req->itt, be32_to_cpu(req->data_length),
+ req->cmd_sn, be32_to_cpu(cmnd->pdu.datasize));
+
+ if (unlikely(cmnd->parent_req)) {
+ struct iscsi_scsi_cmd_hdr *preq =
+ cmnd_hdr(cmnd->parent_req);
+ PRINT_CRIT_ERROR("%p %x %u", preq, preq->opcode,
+ preq->scb[0]);
+ }
+ BUG();
+ }
+#endif
+
+ kmem_cache_free(iscsi_cmnd_cache, cmnd);
+ return;
+}
+
+/* Might be called under some lock and on SIRQ */
+void cmnd_done(struct iscsi_cmnd *cmnd)
+{
+
+ TRACE_DBG("cmnd %p", cmnd);
+
+ if (unlikely(test_bit(ISCSI_CMD_ABORTED, &cmnd->prelim_compl_flags))) {
+ TRACE_MGMT_DBG("Done aborted cmd %p (scst cmd %p, state %d, "
+ "parent_req %p)", cmnd, cmnd->scst_cmd,
+ cmnd->scst_state, cmnd->parent_req);
+ }
+
+ EXTRACHECKS_BUG_ON(cmnd->on_rx_digest_list);
+ EXTRACHECKS_BUG_ON(cmnd->hashed);
+
+ req_del_from_write_timeout_list(cmnd);
+
+ if (cmnd->parent_req == NULL) {
+ struct iscsi_conn *conn = cmnd->conn;
+ struct iscsi_cmnd *rsp, *t;
+
+ TRACE_DBG("Deleting req %p from conn %p", cmnd, conn);
+
+ spin_lock_bh(&conn->cmd_list_lock);
+ list_del(&cmnd->cmd_list_entry);
+ spin_unlock_bh(&conn->cmd_list_lock);
+
+ conn_put(conn);
+
+ EXTRACHECKS_BUG_ON(!list_empty(&cmnd->rx_ddigest_cmd_list));
+
+ /* Order between above and below code is important! */
+
+ if ((cmnd->scst_cmd != NULL) || (cmnd->scst_aen != NULL)) {
+ switch (cmnd->scst_state) {
+ case ISCSI_CMD_STATE_PROCESSED:
+ TRACE_DBG("cmd %p PROCESSED", cmnd);
+ scst_tgt_cmd_done(cmnd->scst_cmd,
+ SCST_CONTEXT_DIRECT_ATOMIC);
+ break;
+
+ case ISCSI_CMD_STATE_AFTER_PREPROC:
+ {
+ /* It can be for some aborted commands */
+ struct scst_cmd *scst_cmd = cmnd->scst_cmd;
+ TRACE_DBG("cmd %p AFTER_PREPROC", cmnd);
+ cmnd->scst_state = ISCSI_CMD_STATE_RESTARTED;
+ cmnd->scst_cmd = NULL;
+ scst_restart_cmd(scst_cmd,
+ SCST_PREPROCESS_STATUS_ERROR_FATAL,
+ SCST_CONTEXT_THREAD);
+ break;
+ }
+
+ case ISCSI_CMD_STATE_AEN:
+ TRACE_DBG("cmd %p AEN PROCESSED", cmnd);
+ scst_aen_done(cmnd->scst_aen);
+ break;
+
+ case ISCSI_CMD_STATE_OUT_OF_SCST_PRELIM_COMPL:
+ break;
+
+ default:
+ PRINT_CRIT_ERROR("Unexpected cmnd scst state "
+ "%d", cmnd->scst_state);
+ BUG();
+ break;
+ }
+ }
+
+ if (cmnd->own_sg) {
+ TRACE_DBG("own_sg for req %p", cmnd);
+ if (cmnd->sg != &dummy_sg)
+ scst_free(cmnd->sg, cmnd->sg_cnt);
+#ifdef CONFIG_SCST_DEBUG
+ cmnd->own_sg = 0;
+ cmnd->sg = NULL;
+ cmnd->sg_cnt = -1;
+#endif
+ }
+
+ if (cmnd->dec_active_cmnds) {
+ struct iscsi_session *sess = cmnd->conn->session;
+ TRACE_DBG("Decrementing active_cmds (cmd %p, sess %p, "
+ "new value %d)", cmnd, sess,
+ atomic_read(&sess->active_cmds)-1);
+ atomic_dec(&sess->active_cmds);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(atomic_read(&sess->active_cmds) < 0)) {
+ PRINT_CRIT_ERROR("active_cmds < 0 (%d)!!",
+ atomic_read(&sess->active_cmds));
+ BUG();
+ }
+#endif
+ }
+
+ list_for_each_entry_safe(rsp, t, &cmnd->rsp_cmd_list,
+ rsp_cmd_list_entry) {
+ cmnd_free(rsp);
+ }
+
+ cmnd_free(cmnd);
+ } else {
+ if (cmnd->own_sg) {
+ TRACE_DBG("own_sg for rsp %p", cmnd);
+ if ((cmnd->sg != &dummy_sg) && (cmnd->sg != cmnd->rsp_sg))
+ scst_free(cmnd->sg, cmnd->sg_cnt);
+#ifdef CONFIG_SCST_DEBUG
+ cmnd->own_sg = 0;
+ cmnd->sg = NULL;
+ cmnd->sg_cnt = -1;
+#endif
+ }
+
+ EXTRACHECKS_BUG_ON(cmnd->dec_active_cmnds);
+
+ if (cmnd == cmnd->parent_req->main_rsp) {
+ TRACE_DBG("Finishing main rsp %p (req %p)", cmnd,
+ cmnd->parent_req);
+ cmnd->parent_req->main_rsp = NULL;
+ }
+
+ cmnd_put(cmnd->parent_req);
+ /*
+ * rsp will be freed on the last parent's put and can already
+ * be freed!!
+ */
+ }
+ return;
+}
+
+/*
+ * Corresponding conn may also get destroyed after this function, except only
+ * if it's called from the read thread!
+ *
+ * It can't be called in parallel with iscsi_cmnds_init_write()!
+ */
+void req_cmnd_release_force(struct iscsi_cmnd *req)
+{
+ struct iscsi_cmnd *rsp, *t;
+ struct iscsi_conn *conn = req->conn;
+ LIST_HEAD(cmds_list);
+
+ TRACE_MGMT_DBG("req %p", req);
+
+ BUG_ON(req == conn->read_cmnd);
+
+ spin_lock_bh(&conn->write_list_lock);
+ list_for_each_entry_safe(rsp, t, &conn->write_list, write_list_entry) {
+ if (rsp->parent_req != req)
+ continue;
+
+ cmd_del_from_write_list(rsp);
+
+ list_add_tail(&rsp->write_list_entry, &cmds_list);
+ }
+ spin_unlock_bh(&conn->write_list_lock);
+
+ list_for_each_entry_safe(rsp, t, &cmds_list, write_list_entry) {
+ TRACE_MGMT_DBG("Putting write rsp %p", rsp);
+ list_del(&rsp->write_list_entry);
+ cmnd_put(rsp);
+ }
+
+ /* Supposed nobody can add responses in the list anymore */
+ list_for_each_entry_reverse(rsp, &req->rsp_cmd_list,
+ rsp_cmd_list_entry) {
+ bool r;
+
+ if (rsp->force_cleanup_done)
+ continue;
+
+ rsp->force_cleanup_done = 1;
+
+ if (cmnd_get_check(rsp))
+ continue;
+
+ spin_lock_bh(&conn->write_list_lock);
+ r = rsp->on_write_list || rsp->write_processing_started;
+ spin_unlock_bh(&conn->write_list_lock);
+
+ cmnd_put(rsp);
+
+ if (r)
+ continue;
+
+ /*
+ * If both on_write_list and write_processing_started not set,
+ * we can safely put() rsp.
+ */
+ TRACE_MGMT_DBG("Putting rsp %p", rsp);
+ cmnd_put(rsp);
+ }
+
+ if (req->main_rsp != NULL) {
+ TRACE_MGMT_DBG("Putting main rsp %p", req->main_rsp);
+ cmnd_put(req->main_rsp);
+ req->main_rsp = NULL;
+ }
+
+ req_cmnd_release(req);
+ return;
+}
+
+/*
+ * Corresponding conn may also get destroyed after this function, except only
+ * if it's called from the read thread!
+ */
+static void req_cmnd_release(struct iscsi_cmnd *req)
+{
+ struct iscsi_cmnd *c, *t;
+
+ TRACE_DBG("req %p", req);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ BUG_ON(req->release_called);
+ req->release_called = 1;
+#endif
+
+ if (unlikely(test_bit(ISCSI_CMD_ABORTED, &req->prelim_compl_flags))) {
+ TRACE_MGMT_DBG("Release aborted req cmd %p (scst cmd %p, "
+ "state %d)", req, req->scst_cmd, req->scst_state);
+ }
+
+ BUG_ON(req->parent_req != NULL);
+
+ /*
+ * We have to remove hashed req from the hash list before sending
+ * response. Otherwise we can have a race, when for some reason cmd's
+ * release (and, hence, removal from the hash) is delayed after the
+ * transmission and initiator sends cmd with the same ITT, hence
+ * the new command will be erroneously rejected as a duplicate.
+ */
+ if (unlikely(req->hashed)) {
+ /* It sometimes can happen during errors recovery */
+ cmnd_remove_data_wait_hash(req);
+ }
+
+ if (unlikely(req->main_rsp != NULL)) {
+ TRACE_DBG("Sending main rsp %p", req->main_rsp);
+ iscsi_cmnd_init_write(req->main_rsp, ISCSI_INIT_WRITE_WAKE);
+ req->main_rsp = NULL;
+ }
+
+ list_for_each_entry_safe(c, t, &req->rx_ddigest_cmd_list,
+ rx_ddigest_cmd_list_entry) {
+ cmd_del_from_rx_ddigest_list(c);
+ cmnd_put(c);
+ }
+
+ EXTRACHECKS_BUG_ON(req->pending);
+
+ if (req->dec_active_cmnds) {
+ struct iscsi_session *sess = req->conn->session;
+ TRACE_DBG("Decrementing active_cmds (cmd %p, sess %p, "
+ "new value %d)", req, sess,
+ atomic_read(&sess->active_cmds)-1);
+ atomic_dec(&sess->active_cmds);
+ req->dec_active_cmnds = 0;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(atomic_read(&sess->active_cmds) < 0)) {
+ PRINT_CRIT_ERROR("active_cmds < 0 (%d)!!",
+ atomic_read(&sess->active_cmds));
+ BUG();
+ }
+#endif
+ }
+
+ cmnd_put(req);
+ return;
+}
+
+/*
+ * Corresponding conn may also get destroyed after this function, except only
+ * if it's called from the read thread!
+ */
+void rsp_cmnd_release(struct iscsi_cmnd *cmnd)
+{
+ TRACE_DBG("%p", cmnd);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ BUG_ON(cmnd->release_called);
+ cmnd->release_called = 1;
+#endif
+
+ EXTRACHECKS_BUG_ON(cmnd->parent_req == NULL);
+
+ cmnd_put(cmnd);
+ return;
+}
+
+static struct iscsi_cmnd *iscsi_alloc_rsp(struct iscsi_cmnd *parent)
+{
+ struct iscsi_cmnd *rsp;
+
+ rsp = cmnd_alloc(parent->conn, parent);
+
+ TRACE_DBG("Adding rsp %p to parent %p", rsp, parent);
+ list_add_tail(&rsp->rsp_cmd_list_entry, &parent->rsp_cmd_list);
+
+ cmnd_get(parent);
+ return rsp;
+}
+
+static inline struct iscsi_cmnd *iscsi_alloc_main_rsp(struct iscsi_cmnd *parent)
+{
+ struct iscsi_cmnd *rsp;
+
+ EXTRACHECKS_BUG_ON(parent->main_rsp != NULL);
+
+ rsp = iscsi_alloc_rsp(parent);
+ parent->main_rsp = rsp;
+ return rsp;
+}
+
+static void iscsi_cmnds_init_write(struct list_head *send, int flags)
+{
+ struct iscsi_cmnd *rsp = list_entry(send->next, struct iscsi_cmnd,
+ write_list_entry);
+ struct iscsi_conn *conn = rsp->conn;
+ struct list_head *pos, *next;
+
+ BUG_ON(list_empty(send));
+
+ if (!(conn->ddigest_type & DIGEST_NONE)) {
+ list_for_each(pos, send) {
+ rsp = list_entry(pos, struct iscsi_cmnd,
+ write_list_entry);
+
+ if (rsp->pdu.datasize != 0) {
+ TRACE_DBG("Doing data digest (%p:%x)", rsp,
+ cmnd_opcode(rsp));
+ digest_tx_data(rsp);
+ }
+ }
+ }
+
+ spin_lock_bh(&conn->write_list_lock);
+ list_for_each_safe(pos, next, send) {
+ rsp = list_entry(pos, struct iscsi_cmnd, write_list_entry);
+
+ TRACE_DBG("%p:%x", rsp, cmnd_opcode(rsp));
+
+ BUG_ON(conn != rsp->conn);
+
+ list_del(&rsp->write_list_entry);
+ cmd_add_on_write_list(conn, rsp);
+ }
+ spin_unlock_bh(&conn->write_list_lock);
+
+ if (flags & ISCSI_INIT_WRITE_WAKE)
+ iscsi_make_conn_wr_active(conn);
+
+ return;
+}
+
+static void iscsi_cmnd_init_write(struct iscsi_cmnd *rsp, int flags)
+{
+ LIST_HEAD(head);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ if (unlikely(rsp->on_write_list)) {
+ PRINT_CRIT_ERROR("cmd already on write list (%x %x %x "
+ "%u %u %d %d", cmnd_itt(rsp),
+ cmnd_opcode(rsp), cmnd_scsicode(rsp),
+ rsp->hdigest, rsp->ddigest,
+ list_empty(&rsp->rsp_cmd_list), rsp->hashed);
+ BUG();
+ }
+#endif
+ list_add_tail(&rsp->write_list_entry, &head);
+ iscsi_cmnds_init_write(&head, flags);
+ return;
+}
+
+static void send_data_rsp(struct iscsi_cmnd *req, u8 status, int send_status)
+{
+ struct iscsi_cmnd *rsp;
+ struct iscsi_scsi_cmd_hdr *req_hdr = cmnd_hdr(req);
+ struct iscsi_data_in_hdr *rsp_hdr;
+ u32 pdusize, expsize, size, offset, sn;
+ LIST_HEAD(send);
+
+ TRACE_DBG("req %p", req);
+
+ pdusize = req->conn->session->sess_params.max_xmit_data_length;
+ expsize = req->read_size;
+ size = min(expsize, (u32)req->bufflen);
+ offset = 0;
+ sn = 0;
+
+ while (1) {
+ rsp = iscsi_alloc_rsp(req);
+ TRACE_DBG("rsp %p", rsp);
+ rsp->sg = req->sg;
+ rsp->sg_cnt = req->sg_cnt;
+ rsp->bufflen = req->bufflen;
+ rsp_hdr = (struct iscsi_data_in_hdr *)&rsp->pdu.bhs;
+
+ rsp_hdr->opcode = ISCSI_OP_SCSI_DATA_IN;
+ rsp_hdr->itt = req_hdr->itt;
+ rsp_hdr->ttt = cpu_to_be32(ISCSI_RESERVED_TAG);
+ rsp_hdr->buffer_offset = cpu_to_be32(offset);
+ rsp_hdr->data_sn = cpu_to_be32(sn);
+
+ if (size <= pdusize) {
+ TRACE_DBG("offset %d, size %d", offset, size);
+ rsp->pdu.datasize = size;
+ if (send_status) {
+ unsigned int scsisize;
+
+ TRACE_DBG("status %x", status);
+
+ EXTRACHECKS_BUG_ON((cmnd_hdr(req)->flags & ISCSI_CMD_WRITE) != 0);
+
+ rsp_hdr->flags = ISCSI_FLG_FINAL | ISCSI_FLG_STATUS;
+ rsp_hdr->cmd_status = status;
+
+ scsisize = req->bufflen;
+ if (scsisize < expsize) {
+ rsp_hdr->flags |= ISCSI_FLG_RESIDUAL_UNDERFLOW;
+ size = expsize - scsisize;
+ } else if (scsisize > expsize) {
+ rsp_hdr->flags |= ISCSI_FLG_RESIDUAL_OVERFLOW;
+ size = scsisize - expsize;
+ } else
+ size = 0;
+ rsp_hdr->residual_count = cpu_to_be32(size);
+ }
+ list_add_tail(&rsp->write_list_entry, &send);
+ break;
+ }
+
+ TRACE_DBG("pdusize %d, offset %d, size %d", pdusize, offset,
+ size);
+
+ rsp->pdu.datasize = pdusize;
+
+ size -= pdusize;
+ offset += pdusize;
+ sn++;
+
+ list_add_tail(&rsp->write_list_entry, &send);
+ }
+ iscsi_cmnds_init_write(&send, 0);
+ return;
+}
+
+static void iscsi_init_status_rsp(struct iscsi_cmnd *rsp,
+ int status, const u8 *sense_buf, int sense_len, bool bufflen_set)
+{
+ struct iscsi_cmnd *req = rsp->parent_req;
+ struct iscsi_scsi_rsp_hdr *rsp_hdr;
+ struct scatterlist *sg;
+
+ rsp_hdr = (struct iscsi_scsi_rsp_hdr *)&rsp->pdu.bhs;
+ rsp_hdr->opcode = ISCSI_OP_SCSI_RSP;
+ rsp_hdr->flags = ISCSI_FLG_FINAL;
+ rsp_hdr->response = ISCSI_RESPONSE_COMMAND_COMPLETED;
+ rsp_hdr->cmd_status = status;
+ rsp_hdr->itt = cmnd_hdr(req)->itt;
+
+ if (SCST_SENSE_VALID(sense_buf)) {
+ TRACE_DBG("%s", "SENSE VALID");
+
+ sg = rsp->sg = rsp->rsp_sg;
+ rsp->sg_cnt = 2;
+ rsp->own_sg = 1;
+
+ sg_init_table(sg, 2);
+ sg_set_buf(&sg[0], &rsp->sense_hdr, sizeof(rsp->sense_hdr));
+ sg_set_buf(&sg[1], sense_buf, sense_len);
+
+ rsp->sense_hdr.length = cpu_to_be16(sense_len);
+
+ rsp->pdu.datasize = sizeof(rsp->sense_hdr) + sense_len;
+ rsp->bufflen = rsp->pdu.datasize;
+ } else {
+ rsp->pdu.datasize = 0;
+ rsp->bufflen = 0;
+ }
+
+ iscsi_set_resid(rsp, bufflen_set);
+ return;
+}
+
+static inline struct iscsi_cmnd *create_status_rsp(struct iscsi_cmnd *req,
+ int status, const u8 *sense_buf, int sense_len, bool bufflen_set)
+{
+ struct iscsi_cmnd *rsp;
+
+ rsp = iscsi_alloc_rsp(req);
+ TRACE_DBG("rsp %p", rsp);
+
+ iscsi_init_status_rsp(rsp, status, sense_buf, sense_len, bufflen_set);
+ return rsp;
+}
+
+static struct iscsi_cmnd *create_prelim_status_rsp(struct iscsi_cmnd *req,
+ int status, const u8 *sense_buf, int sense_len)
+{
+ struct iscsi_cmnd *rsp;
+
+ rsp = iscsi_alloc_main_rsp(req);
+ TRACE_DBG("main rsp %p", rsp);
+
+ iscsi_init_status_rsp(rsp, status, sense_buf, sense_len, false);
+ return rsp;
+}
+
+static int iscsi_set_prelim_r2t_len_to_receive(struct iscsi_cmnd *req)
+{
+ struct iscsi_hdr *req_hdr = &req->pdu.bhs;
+ int res = 0;
+
+ if (req_hdr->flags & ISCSI_CMD_FINAL)
+ goto out;
+
+ res = cmnd_insert_data_wait_hash(req);
+ if (res != 0) {
+ /*
+ * We have to close connection, because otherwise a data
+ * corruption is possible if we allow to receive data
+ * for this request in another request with dublicated ITT.
+ */
+ mark_conn_closed(req->conn);
+ goto out;
+ }
+
+ /*
+ * We need to wait for one or more PDUs. Let's simplify
+ * other code and pretend we need to receive 1 byte.
+ * In data_out_start() we will correct it.
+ */
+ if (req->outstanding_r2t == 0) {
+ req->outstanding_r2t = 1;
+ req_add_to_write_timeout_list(req);
+ }
+ req->r2t_len_to_receive = 1;
+ req->r2t_len_to_send = 0;
+
+ TRACE_DBG("req %p, op %x, outstanding_r2t %d, r2t_len_to_receive %d, "
+ "r2t_len_to_send %d", req, cmnd_opcode(req),
+ req->outstanding_r2t, req->r2t_len_to_receive,
+ req->r2t_len_to_send);
+
+out:
+ return res;
+}
+
+static int create_preliminary_status_rsp(struct iscsi_cmnd *req,
+ int status, const u8 *sense_buf, int sense_len)
+{
+ int res = 0;
+ struct iscsi_scsi_cmd_hdr *req_hdr = cmnd_hdr(req);
+
+ if (req->prelim_compl_flags != 0) {
+ TRACE_MGMT_DBG("req %p already prelim completed", req);
+ goto out;
+ }
+
+ req->scst_state = ISCSI_CMD_STATE_OUT_OF_SCST_PRELIM_COMPL;
+
+ if ((req_hdr->flags & ISCSI_CMD_READ) &&
+ (req_hdr->flags & ISCSI_CMD_WRITE)) {
+ int sz = cmnd_read_size(req);
+ if (sz > 0)
+ req->read_size = sz;
+ } else if (req_hdr->flags & ISCSI_CMD_READ)
+ req->read_size = be32_to_cpu(req_hdr->data_length);
+
+ create_prelim_status_rsp(req, status, sense_buf, sense_len);
+ res = iscsi_preliminary_complete(req, req, true);
+
+out:
+ return res;
+}
+
+static int set_scst_preliminary_status_rsp(struct iscsi_cmnd *req,
+ bool get_data, int key, int asc, int ascq)
+{
+ int res = 0;
+
+ if (req->scst_cmd == NULL) {
+ /* There must be already error set */
+ goto complete;
+ }
+
+ scst_set_cmd_error(req->scst_cmd, key, asc, ascq);
+
+complete:
+ res = iscsi_preliminary_complete(req, req, get_data);
+ return res;
+}
+
+static int create_reject_rsp(struct iscsi_cmnd *req, int reason, bool get_data)
+{
+ int res = 0;
+ struct iscsi_cmnd *rsp;
+ struct iscsi_reject_hdr *rsp_hdr;
+ struct scatterlist *sg;
+
+ TRACE_MGMT_DBG("Reject: req %p, reason %x", req, reason);
+
+ if (cmnd_opcode(req) == ISCSI_OP_SCSI_CMD) {
+ if (req->scst_cmd == NULL) {
+ /* BUSY status must be already set */
+ struct iscsi_scsi_rsp_hdr *rsp_hdr;
+ rsp_hdr = (struct iscsi_scsi_rsp_hdr *)&req->main_rsp->pdu.bhs;
+ BUG_ON(rsp_hdr->cmd_status == 0);
+ /*
+ * Let's not send REJECT here. The initiator will retry
+ * and, hopefully, next time we will not fail allocating
+ * scst_cmd, so we will then send the REJECT.
+ */
+ goto out;
+ } else
+ set_scst_preliminary_status_rsp(req, get_data,
+ SCST_LOAD_SENSE(scst_sense_invalid_message));
+ }
+
+ rsp = iscsi_alloc_main_rsp(req);
+ rsp_hdr = (struct iscsi_reject_hdr *)&rsp->pdu.bhs;
+
+ rsp_hdr->opcode = ISCSI_OP_REJECT;
+ rsp_hdr->ffffffff = ISCSI_RESERVED_TAG;
+ rsp_hdr->reason = reason;
+
+ sg = rsp->sg = rsp->rsp_sg;
+ rsp->sg_cnt = 1;
+ rsp->own_sg = 1;
+ sg_init_one(sg, &req->pdu.bhs, sizeof(struct iscsi_hdr));
+ rsp->bufflen = rsp->pdu.datasize = sizeof(struct iscsi_hdr);
+
+ res = iscsi_preliminary_complete(req, req, true);
+
+out:
+ return res;
+}
+
+static inline int iscsi_get_allowed_cmds(struct iscsi_session *sess)
+{
+ int res = max(-1, (int)sess->tgt_params.queued_cmnds -
+ atomic_read(&sess->active_cmds)-1);
+ TRACE_DBG("allowed cmds %d (sess %p, active_cmds %d)", res,
+ sess, atomic_read(&sess->active_cmds));
+ return res;
+}
+
+static u32 cmnd_set_sn(struct iscsi_cmnd *cmnd, int set_stat_sn)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+ struct iscsi_session *sess = conn->session;
+ u32 res;
+
+ spin_lock(&sess->sn_lock);
+
+ if (set_stat_sn)
+ cmnd->pdu.bhs.sn = cpu_to_be32(conn->stat_sn++);
+ cmnd->pdu.bhs.exp_sn = cpu_to_be32(sess->exp_cmd_sn);
+ cmnd->pdu.bhs.max_sn = cpu_to_be32(sess->exp_cmd_sn +
+ iscsi_get_allowed_cmds(sess));
+
+ res = cpu_to_be32(conn->stat_sn);
+
+ spin_unlock(&sess->sn_lock);
+ return res;
+}
+
+/* Called under sn_lock */
+static void __update_stat_sn(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+ u32 exp_stat_sn;
+
+ cmnd->pdu.bhs.exp_sn = exp_stat_sn = be32_to_cpu(cmnd->pdu.bhs.exp_sn);
+ TRACE_DBG("%x,%x", cmnd_opcode(cmnd), exp_stat_sn);
+ if ((int)(exp_stat_sn - conn->exp_stat_sn) > 0 &&
+ (int)(exp_stat_sn - conn->stat_sn) <= 0) {
+ /* free pdu resources */
+ cmnd->conn->exp_stat_sn = exp_stat_sn;
+ }
+ return;
+}
+
+static inline void update_stat_sn(struct iscsi_cmnd *cmnd)
+{
+ spin_lock(&cmnd->conn->session->sn_lock);
+ __update_stat_sn(cmnd);
+ spin_unlock(&cmnd->conn->session->sn_lock);
+ return;
+}
+
+/* Called under sn_lock */
+static int check_cmd_sn(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_session *session = cmnd->conn->session;
+ u32 cmd_sn;
+
+ cmnd->pdu.bhs.sn = cmd_sn = be32_to_cpu(cmnd->pdu.bhs.sn);
+ TRACE_DBG("%d(%d)", cmd_sn, session->exp_cmd_sn);
+ if (likely((s32)(cmd_sn - session->exp_cmd_sn) >= 0))
+ return 0;
+ PRINT_ERROR("sequence error (%x,%x)", cmd_sn, session->exp_cmd_sn);
+ return -ISCSI_REASON_PROTOCOL_ERROR;
+}
+
+static struct iscsi_cmnd *cmnd_find_itt_get(struct iscsi_conn *conn, u32 itt)
+{
+ struct iscsi_cmnd *cmnd, *found_cmnd = NULL;
+
+ spin_lock_bh(&conn->cmd_list_lock);
+ list_for_each_entry(cmnd, &conn->cmd_list, cmd_list_entry) {
+ if ((cmnd->pdu.bhs.itt == itt) && !cmnd_get_check(cmnd)) {
+ found_cmnd = cmnd;
+ break;
+ }
+ }
+ spin_unlock_bh(&conn->cmd_list_lock);
+
+ return found_cmnd;
+}
+
+/**
+ ** We use the ITT hash only to find original request PDU for subsequent
+ ** Data-Out PDUs.
+ **/
+
+/* Must be called under cmnd_data_wait_hash_lock */
+static struct iscsi_cmnd *__cmnd_find_data_wait_hash(struct iscsi_conn *conn,
+ u32 itt)
+{
+ struct list_head *head;
+ struct iscsi_cmnd *cmnd;
+
+ head = &conn->session->cmnd_data_wait_hash[cmnd_hashfn(itt)];
+
+ list_for_each_entry(cmnd, head, hash_list_entry) {
+ if (cmnd->pdu.bhs.itt == itt)
+ return cmnd;
+ }
+ return NULL;
+}
+
+static struct iscsi_cmnd *cmnd_find_data_wait_hash(struct iscsi_conn *conn,
+ u32 itt)
+{
+ struct iscsi_cmnd *res;
+ struct iscsi_session *session = conn->session;
+
+ spin_lock(&session->cmnd_data_wait_hash_lock);
+ res = __cmnd_find_data_wait_hash(conn, itt);
+ spin_unlock(&session->cmnd_data_wait_hash_lock);
+
+ return res;
+}
+
+static inline u32 get_next_ttt(struct iscsi_conn *conn)
+{
+ u32 ttt;
+ struct iscsi_session *session = conn->session;
+
+ /* Not compatible with MC/S! */
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+ if (unlikely(session->next_ttt == ISCSI_RESERVED_TAG))
+ session->next_ttt++;
+ ttt = session->next_ttt++;
+
+ return ttt;
+}
+
+static int cmnd_insert_data_wait_hash(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_session *session = cmnd->conn->session;
+ struct iscsi_cmnd *tmp;
+ struct list_head *head;
+ int err = 0;
+ u32 itt = cmnd->pdu.bhs.itt;
+
+ if (unlikely(cmnd->hashed)) {
+ /* It can be for preliminary completed commands */
+ goto out;
+ }
+
+ /*
+ * We don't need TTT, because ITT/buffer_offset pair is sufficient
+ * to find out the original request and buffer for Data-Out PDUs, but
+ * crazy iSCSI spec requires us to send this superfluous field in
+ * R2T PDUs and some initiators may rely on it.
+ */
+ cmnd->target_task_tag = get_next_ttt(cmnd->conn);
+
+ TRACE_DBG("%p:%x", cmnd, itt);
+ if (unlikely(itt == ISCSI_RESERVED_TAG)) {
+ PRINT_ERROR("%s", "ITT is RESERVED_TAG");
+ PRINT_BUFFER("Incorrect BHS", &cmnd->pdu.bhs,
+ sizeof(cmnd->pdu.bhs));
+ err = -ISCSI_REASON_PROTOCOL_ERROR;
+ goto out;
+ }
+
+ spin_lock(&session->cmnd_data_wait_hash_lock);
+
+ head = &session->cmnd_data_wait_hash[cmnd_hashfn(itt)];
+
+ tmp = __cmnd_find_data_wait_hash(cmnd->conn, itt);
+ if (likely(!tmp)) {
+ TRACE_DBG("Adding cmnd %p to the hash (ITT %x)", cmnd,
+ cmnd_itt(cmnd));
+ list_add_tail(&cmnd->hash_list_entry, head);
+ cmnd->hashed = 1;
+ } else {
+ PRINT_ERROR("Task %x in progress, cmnd %p", itt, cmnd);
+ err = -ISCSI_REASON_TASK_IN_PROGRESS;
+ }
+
+ spin_unlock(&session->cmnd_data_wait_hash_lock);
+
+out:
+ return err;
+}
+
+static void cmnd_remove_data_wait_hash(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_session *session = cmnd->conn->session;
+ struct iscsi_cmnd *tmp;
+
+ spin_lock(&session->cmnd_data_wait_hash_lock);
+
+ tmp = __cmnd_find_data_wait_hash(cmnd->conn, cmnd->pdu.bhs.itt);
+
+ if (likely(tmp && tmp == cmnd)) {
+ TRACE_DBG("Deleting cmnd %p from the hash (ITT %x)", cmnd,
+ cmnd_itt(cmnd));
+ list_del(&cmnd->hash_list_entry);
+ cmnd->hashed = 0;
+ } else
+ PRINT_ERROR("%p:%x not found", cmnd, cmnd_itt(cmnd));
+
+ spin_unlock(&session->cmnd_data_wait_hash_lock);
+
+ return;
+}
+
+static void cmnd_prepare_get_rejected_immed_data(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+ struct scatterlist *sg = cmnd->sg;
+ char __user *addr;
+ u32 size;
+ unsigned int i;
+
+ TRACE_DBG_FLAG(iscsi_get_flow_ctrl_or_mgmt_dbg_log_flag(cmnd),
+ "Skipping (cmnd %p, ITT %x, op %x, cmd op %x, "
+ "datasize %u, scst_cmd %p, scst state %d)", cmnd,
+ cmnd_itt(cmnd), cmnd_opcode(cmnd), cmnd_hdr(cmnd)->scb[0],
+ cmnd->pdu.datasize, cmnd->scst_cmd, cmnd->scst_state);
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+ size = cmnd->pdu.datasize;
+ if (!size)
+ goto out;
+
+ /* We already checked pdu.datasize in check_segment_length() */
+
+ if (sg == NULL) {
+ /*
+ * There are no problems with the safety from concurrent
+ * accesses to dummy_page in dummy_sg, since data only
+ * will be read and then discarded.
+ */
+ sg = cmnd->sg = &dummy_sg;
+ cmnd->bufflen = PAGE_SIZE;
+ cmnd->own_sg = 1;
+ }
+
+ addr = (char __force __user *)(page_address(sg_page(&sg[0])));
+ BUG_ON(addr == NULL);
+ conn->read_size = size;
+ for (i = 0; size > PAGE_SIZE; i++, size -= cmnd->bufflen) {
+ /* We already checked pdu.datasize in check_segment_length() */
+ BUG_ON(i >= ISCSI_CONN_IOV_MAX);
+ conn->read_iov[i].iov_base = addr;
+ conn->read_iov[i].iov_len = cmnd->bufflen;
+ }
+ conn->read_iov[i].iov_base = addr;
+ conn->read_iov[i].iov_len = size;
+ conn->read_msg.msg_iov = conn->read_iov;
+ conn->read_msg.msg_iovlen = ++i;
+
+out:
+ return;
+}
+
+static void iscsi_set_resid(struct iscsi_cmnd *rsp, bool bufflen_set)
+{
+ struct iscsi_cmnd *req = rsp->parent_req;
+ struct iscsi_scsi_cmd_hdr *req_hdr = cmnd_hdr(req);
+ struct iscsi_scsi_rsp_hdr *rsp_hdr;
+ int resid, resp_len, in_resp_len;
+
+ if ((req_hdr->flags & ISCSI_CMD_READ) &&
+ (req_hdr->flags & ISCSI_CMD_WRITE)) {
+ rsp_hdr = (struct iscsi_scsi_rsp_hdr *)&rsp->pdu.bhs;
+
+ if (bufflen_set) {
+ resp_len = req->bufflen;
+ if (req->scst_cmd != NULL)
+ in_resp_len = scst_cmd_get_in_bufflen(req->scst_cmd);
+ else
+ in_resp_len = 0;
+ } else {
+ resp_len = 0;
+ in_resp_len = 0;
+ }
+
+ resid = be32_to_cpu(req_hdr->data_length) - in_resp_len;
+ if (resid > 0) {
+ rsp_hdr->flags |= ISCSI_FLG_RESIDUAL_UNDERFLOW;
+ rsp_hdr->residual_count = cpu_to_be32(resid);
+ } else if (resid < 0) {
+ resid = -resid;
+ rsp_hdr->flags |= ISCSI_FLG_RESIDUAL_OVERFLOW;
+ rsp_hdr->residual_count = cpu_to_be32(resid);
+ }
+
+ resid = req->read_size - resp_len;
+ if (resid > 0) {
+ rsp_hdr->flags |= ISCSI_FLG_BIRESIDUAL_UNDERFLOW;
+ rsp_hdr->bi_residual_count = cpu_to_be32(resid);
+ } else if (resid < 0) {
+ resid = -resid;
+ rsp_hdr->flags |= ISCSI_FLG_BIRESIDUAL_OVERFLOW;
+ rsp_hdr->bi_residual_count = cpu_to_be32(resid);
+ }
+ } else {
+ if (bufflen_set)
+ resp_len = req->bufflen;
+ else
+ resp_len = 0;
+
+ resid = req->read_size - resp_len;
+ if (resid > 0) {
+ rsp_hdr = (struct iscsi_scsi_rsp_hdr *)&rsp->pdu.bhs;
+ rsp_hdr->flags |= ISCSI_FLG_RESIDUAL_UNDERFLOW;
+ rsp_hdr->residual_count = cpu_to_be32(resid);
+ } else if (resid < 0) {
+ rsp_hdr = (struct iscsi_scsi_rsp_hdr *)&rsp->pdu.bhs;
+ resid = -resid;
+ rsp_hdr->flags |= ISCSI_FLG_RESIDUAL_OVERFLOW;
+ rsp_hdr->residual_count = cpu_to_be32(resid);
+ }
+ }
+ return;
+}
+
+static int iscsi_preliminary_complete(struct iscsi_cmnd *req,
+ struct iscsi_cmnd *orig_req, bool get_data)
+{
+ int res = 0;
+ bool set_r2t_len;
+
+#ifdef CONFIG_SCST_DEBUG
+ {
+ struct iscsi_hdr *req_hdr = &req->pdu.bhs;
+ TRACE_DBG_FLAG(iscsi_get_flow_ctrl_or_mgmt_dbg_log_flag(orig_req),
+ "Prelim completed req %p, orig_req %p (FINAL %x, "
+ "outstanding_r2t %d)", req, orig_req,
+ (req_hdr->flags & ISCSI_CMD_FINAL),
+ orig_req->outstanding_r2t);
+ }
+#endif
+
+ iscsi_extracheck_is_rd_thread(req->conn);
+ BUG_ON(req->parent_req != NULL);
+
+ if (test_bit(ISCSI_CMD_PRELIM_COMPLETED, &req->prelim_compl_flags)) {
+ TRACE_MGMT_DBG("req %p already prelim completed", req);
+ /* To not try to get data twice */
+ get_data = false;
+ }
+
+ set_r2t_len = !req->hashed &&
+ (cmnd_opcode(req) == ISCSI_OP_SCSI_CMD) &&
+ !test_bit(ISCSI_CMD_PRELIM_COMPLETED,
+ &orig_req->prelim_compl_flags);
+ set_bit(ISCSI_CMD_PRELIM_COMPLETED, &orig_req->prelim_compl_flags);
+
+ TRACE_DBG("get_data %d, set_r2t_len %d", get_data, set_r2t_len);
+
+ if (get_data)
+ cmnd_prepare_get_rejected_immed_data(req);
+
+ if (set_r2t_len)
+ res = iscsi_set_prelim_r2t_len_to_receive(orig_req);
+ return res;
+}
+
+static int cmnd_prepare_recv_pdu(struct iscsi_conn *conn,
+ struct iscsi_cmnd *cmd, u32 offset, u32 size)
+{
+ struct scatterlist *sg = cmd->sg;
+ unsigned int bufflen = cmd->bufflen;
+ unsigned int idx, i;
+ char __user *addr;
+ int res = 0;
+
+ TRACE_DBG("%p %u,%u", cmd->sg, offset, size);
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+ if (unlikely((offset >= bufflen) ||
+ (offset + size > bufflen))) {
+ PRINT_ERROR("Wrong ltn (%u %u %u)", offset, size, bufflen);
+ mark_conn_closed(conn);
+ res = -EIO;
+ goto out;
+ }
+
+ offset += sg[0].offset;
+ idx = offset >> PAGE_SHIFT;
+ offset &= ~PAGE_MASK;
+
+ conn->read_msg.msg_iov = conn->read_iov;
+ conn->read_size = size;
+
+ i = 0;
+ while (1) {
+ addr = (char __force __user *)(page_address(sg_page(&sg[idx])));
+ BUG_ON(addr == NULL);
+ conn->read_iov[i].iov_base = addr + offset;
+ if (offset + size <= PAGE_SIZE) {
+ TRACE_DBG("idx=%d, offset=%u, size=%d, addr=%p",
+ idx, offset, size, addr);
+ conn->read_iov[i].iov_len = size;
+ conn->read_msg.msg_iovlen = ++i;
+ break;
+ }
+ conn->read_iov[i].iov_len = PAGE_SIZE - offset;
+ TRACE_DBG("idx=%d, offset=%u, size=%d, iov_len=%zd, addr=%p",
+ idx, offset, size, conn->read_iov[i].iov_len, addr);
+ size -= conn->read_iov[i].iov_len;
+ if (unlikely(++i >= ISCSI_CONN_IOV_MAX)) {
+ PRINT_ERROR("Initiator %s violated negotiated "
+ "parameters by sending too much data (size "
+ "left %d)", conn->session->initiator_name,
+ size);
+ mark_conn_closed(conn);
+ res = -EINVAL;
+ break;
+ }
+ idx++;
+ offset = sg[idx].offset;
+ }
+ TRACE_DBG("msg_iov=%p, msg_iovlen=%zd",
+ conn->read_msg.msg_iov, conn->read_msg.msg_iovlen);
+
+out:
+ return res;
+}
+
+static void send_r2t(struct iscsi_cmnd *req)
+{
+ struct iscsi_session *sess = req->conn->session;
+ struct iscsi_cmnd *rsp;
+ struct iscsi_r2t_hdr *rsp_hdr;
+ u32 offset, burst;
+ LIST_HEAD(send);
+
+ EXTRACHECKS_BUG_ON(req->r2t_len_to_send == 0);
+
+ /*
+ * There is no race with data_out_start() and conn_abort(), since
+ * all functions called from single read thread
+ */
+ iscsi_extracheck_is_rd_thread(req->conn);
+
+ /*
+ * We don't need to check for PRELIM_COMPLETED here, because for such
+ * commands we set r2t_len_to_send = 0, hence made sure we won't
+ * called here.
+ */
+
+ EXTRACHECKS_BUG_ON(req->outstanding_r2t >
+ sess->sess_params.max_outstanding_r2t);
+
+ if (req->outstanding_r2t == sess->sess_params.max_outstanding_r2t)
+ goto out;
+
+ burst = sess->sess_params.max_burst_length;
+ offset = be32_to_cpu(cmnd_hdr(req)->data_length) -
+ req->r2t_len_to_send;
+
+ do {
+ rsp = iscsi_alloc_rsp(req);
+ rsp->pdu.bhs.ttt = req->target_task_tag;
+ rsp_hdr = (struct iscsi_r2t_hdr *)&rsp->pdu.bhs;
+ rsp_hdr->opcode = ISCSI_OP_R2T;
+ rsp_hdr->flags = ISCSI_FLG_FINAL;
+ rsp_hdr->lun = cmnd_hdr(req)->lun;
+ rsp_hdr->itt = cmnd_hdr(req)->itt;
+ rsp_hdr->r2t_sn = cpu_to_be32(req->r2t_sn++);
+ rsp_hdr->buffer_offset = cpu_to_be32(offset);
+ if (req->r2t_len_to_send > burst) {
+ rsp_hdr->data_length = cpu_to_be32(burst);
+ req->r2t_len_to_send -= burst;
+ offset += burst;
+ } else {
+ rsp_hdr->data_length = cpu_to_be32(req->r2t_len_to_send);
+ req->r2t_len_to_send = 0;
+ }
+
+ TRACE_WRITE("req %p, data_length %u, buffer_offset %u, "
+ "r2t_sn %u, outstanding_r2t %u", req,
+ be32_to_cpu(rsp_hdr->data_length),
+ be32_to_cpu(rsp_hdr->buffer_offset),
+ be32_to_cpu(rsp_hdr->r2t_sn), req->outstanding_r2t);
+
+ list_add_tail(&rsp->write_list_entry, &send);
+ req->outstanding_r2t++;
+
+ } while ((req->outstanding_r2t < sess->sess_params.max_outstanding_r2t) &&
+ (req->r2t_len_to_send != 0));
+
+ iscsi_cmnds_init_write(&send, ISCSI_INIT_WRITE_WAKE);
+
+out:
+ return;
+}
+
+static int iscsi_pre_exec(struct scst_cmd *scst_cmd)
+{
+ int res = SCST_PREPROCESS_STATUS_SUCCESS;
+ struct iscsi_cmnd *req = (struct iscsi_cmnd *)
+ scst_cmd_get_tgt_priv(scst_cmd);
+ struct iscsi_cmnd *c, *t;
+
+ EXTRACHECKS_BUG_ON(scst_cmd_atomic(scst_cmd));
+
+ /* If data digest isn't used this list will be empty */
+ list_for_each_entry_safe(c, t, &req->rx_ddigest_cmd_list,
+ rx_ddigest_cmd_list_entry) {
+ TRACE_DBG("Checking digest of RX ddigest cmd %p", c);
+ if (digest_rx_data(c) != 0) {
+ scst_set_cmd_error(scst_cmd,
+ SCST_LOAD_SENSE(iscsi_sense_crc_error));
+ res = SCST_PREPROCESS_STATUS_ERROR_SENSE_SET;
+ /*
+ * The rest of rx_ddigest_cmd_list will be freed
+ * in req_cmnd_release()
+ */
+ goto out;
+ }
+ cmd_del_from_rx_ddigest_list(c);
+ cmnd_put(c);
+ }
+
+out:
+ return res;
+}
+
+static int nop_out_start(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+ struct iscsi_hdr *req_hdr = &cmnd->pdu.bhs;
+ u32 size, tmp;
+ int i, err = 0;
+
+ TRACE_DBG("%p", cmnd);
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+ if (!(req_hdr->flags & ISCSI_FLG_FINAL)) {
+ PRINT_ERROR("%s", "Initiator sent Nop-Out with not a single "
+ "PDU");
+ err = -ISCSI_REASON_PROTOCOL_ERROR;
+ goto out;
+ }
+
+ if (cmnd_itt(cmnd) == cpu_to_be32(ISCSI_RESERVED_TAG)) {
+ if (unlikely(!(cmnd->pdu.bhs.opcode & ISCSI_OP_IMMEDIATE)))
+ PRINT_ERROR("%s", "Initiator sent RESERVED tag for "
+ "non-immediate Nop-Out command");
+ }
+
+ spin_lock(&conn->session->sn_lock);
+ __update_stat_sn(cmnd);
+ err = check_cmd_sn(cmnd);
+ spin_unlock(&conn->session->sn_lock);
+ if (unlikely(err))
+ goto out;
+
+ size = cmnd->pdu.datasize;
+
+ if (size) {
+ conn->read_msg.msg_iov = conn->read_iov;
+ if (cmnd->pdu.bhs.itt != cpu_to_be32(ISCSI_RESERVED_TAG)) {
+ struct scatterlist *sg;
+
+ cmnd->sg = sg = scst_alloc(size, GFP_KERNEL,
+ &cmnd->sg_cnt);
+ if (sg == NULL) {
+ TRACE(TRACE_OUT_OF_MEM, "Allocating buffer for"
+ " %d Nop-Out payload failed", size);
+ err = -ISCSI_REASON_OUT_OF_RESOURCES;
+ goto out;
+ }
+
+ /* We already checked it in check_segment_length() */
+ BUG_ON(cmnd->sg_cnt > (signed)ISCSI_CONN_IOV_MAX);
+
+ cmnd->own_sg = 1;
+ cmnd->bufflen = size;
+
+ for (i = 0; i < cmnd->sg_cnt; i++) {
+ conn->read_iov[i].iov_base =
+ (void __force __user *)(page_address(sg_page(&sg[i])));
+ tmp = min_t(u32, size, PAGE_SIZE);
+ conn->read_iov[i].iov_len = tmp;
+ conn->read_size += tmp;
+ size -= tmp;
+ }
+ BUG_ON(size != 0);
+ } else {
+ /*
+ * There are no problems with the safety from concurrent
+ * accesses to dummy_page, since for ISCSI_RESERVED_TAG
+ * the data only read and then discarded.
+ */
+ for (i = 0; i < (signed)ISCSI_CONN_IOV_MAX; i++) {
+ conn->read_iov[i].iov_base =
+ (void __force __user *)(page_address(dummy_page));
+ tmp = min_t(u32, size, PAGE_SIZE);
+ conn->read_iov[i].iov_len = tmp;
+ conn->read_size += tmp;
+ size -= tmp;
+ }
+
+ /* We already checked size in check_segment_length() */
+ BUG_ON(size != 0);
+ }
+
+ conn->read_msg.msg_iovlen = i;
+ TRACE_DBG("msg_iov=%p, msg_iovlen=%zd", conn->read_msg.msg_iov,
+ conn->read_msg.msg_iovlen);
+ }
+
+out:
+ return err;
+}
+
+int cmnd_rx_continue(struct iscsi_cmnd *req)
+{
+ struct iscsi_conn *conn = req->conn;
+ struct iscsi_session *session = conn->session;
+ struct iscsi_scsi_cmd_hdr *req_hdr = cmnd_hdr(req);
+ struct scst_cmd *scst_cmd = req->scst_cmd;
+ scst_data_direction dir;
+ bool unsolicited_data_expected = false;
+ int res = 0;
+
+ TRACE_DBG("scsi command: %x", req_hdr->scb[0]);
+
+ EXTRACHECKS_BUG_ON(req->scst_state != ISCSI_CMD_STATE_AFTER_PREPROC);
+
+ dir = scst_cmd_get_data_direction(scst_cmd);
+
+ /*
+ * Check for preliminary completion here to save R2Ts. For TASK QUEUE
+ * FULL statuses that might be a big performance win.
+ */
+ if (unlikely(scst_cmd_prelim_completed(scst_cmd) ||
+ unlikely(req->prelim_compl_flags != 0))) {
+ /*
+ * If necessary, ISCSI_CMD_ABORTED will be set by
+ * iscsi_xmit_response().
+ */
+ res = iscsi_preliminary_complete(req, req, true);
+ goto trace;
+ }
+
+ /* For prelim completed commands sg&K can be already set! */
+
+ if (dir != SCST_DATA_BIDI) {
+ req->sg = scst_cmd_get_sg(scst_cmd);
+ req->sg_cnt = scst_cmd_get_sg_cnt(scst_cmd);
+ req->bufflen = scst_cmd_get_bufflen(scst_cmd);
+ } else {
+ req->sg = scst_cmd_get_in_sg(scst_cmd);
+ req->sg_cnt = scst_cmd_get_in_sg_cnt(scst_cmd);
+ req->bufflen = scst_cmd_get_in_bufflen(scst_cmd);
+ }
+
+ if (dir & SCST_DATA_WRITE) {
+ unsolicited_data_expected = !(req_hdr->flags & ISCSI_CMD_FINAL);
+
+ if (unlikely(session->sess_params.initial_r2t &&
+ unsolicited_data_expected)) {
+ PRINT_ERROR("Initiator %s violated negotiated "
+ "parameters: initial R2T is required (ITT %x, "
+ "op %x)", session->initiator_name,
+ cmnd_itt(req), req_hdr->scb[0]);
+ goto out_close;
+ }
+
+ if (unlikely(!session->sess_params.immediate_data &&
+ req->pdu.datasize)) {
+ PRINT_ERROR("Initiator %s violated negotiated "
+ "parameters: forbidden immediate data sent "
+ "(ITT %x, op %x)", session->initiator_name,
+ cmnd_itt(req), req_hdr->scb[0]);
+ goto out_close;
+ }
+
+ if (unlikely(session->sess_params.first_burst_length < req->pdu.datasize)) {
+ PRINT_ERROR("Initiator %s violated negotiated "
+ "parameters: immediate data len (%d) > "
+ "first_burst_length (%d) (ITT %x, op %x)",
+ session->initiator_name,
+ req->pdu.datasize,
+ session->sess_params.first_burst_length,
+ cmnd_itt(req), req_hdr->scb[0]);
+ goto out_close;
+ }
+
+ req->r2t_len_to_receive = be32_to_cpu(req_hdr->data_length) -
+ req->pdu.datasize;
+
+ if (unlikely(req->r2t_len_to_receive > req->bufflen)) {
+ PRINT_ERROR("req->r2t_len_to_receive %d > req->bufflen "
+ "%d", req->r2t_len_to_receive, req->bufflen);
+ goto out_close;
+ }
+
+ res = cmnd_insert_data_wait_hash(req);
+ if (unlikely(res != 0)) {
+ /*
+ * We have to close connection, because otherwise a data
+ * corruption is possible if we allow to receive data
+ * for this request in another request with dublicated
+ * ITT.
+ */
+ goto out_close;
+ }
+
+ if (unsolicited_data_expected) {
+ req->outstanding_r2t = 1;
+ req->r2t_len_to_send = req->r2t_len_to_receive -
+ min_t(unsigned int,
+ session->sess_params.first_burst_length -
+ req->pdu.datasize,
+ req->r2t_len_to_receive);
+ } else
+ req->r2t_len_to_send = req->r2t_len_to_receive;
+
+ req_add_to_write_timeout_list(req);
+
+ if (req->pdu.datasize) {
+ res = cmnd_prepare_recv_pdu(conn, req, 0, req->pdu.datasize);
+ /* For performance better to send R2Ts ASAP */
+ if (likely(res == 0) && (req->r2t_len_to_send != 0))
+ send_r2t(req);
+ }
+ } else {
+ if (unlikely(!(req_hdr->flags & ISCSI_CMD_FINAL) ||
+ req->pdu.datasize)) {
+ PRINT_ERROR("Unexpected unsolicited data (ITT %x "
+ "CDB %x", cmnd_itt(req), req_hdr->scb[0]);
+ set_scst_preliminary_status_rsp(req, true,
+ SCST_LOAD_SENSE(iscsi_sense_unexpected_unsolicited_data));
+ }
+ }
+
+trace:
+ TRACE_DBG("req=%p, dir=%d, unsolicited_data_expected=%d, "
+ "r2t_len_to_receive=%d, r2t_len_to_send=%d, bufflen=%d, "
+ "own_sg %d", req, dir, unsolicited_data_expected,
+ req->r2t_len_to_receive, req->r2t_len_to_send, req->bufflen,
+ req->own_sg);
+
+out:
+ return res;
+
+out_close:
+ mark_conn_closed(conn);
+ res = -EINVAL;
+ goto out;
+}
+
+static int scsi_cmnd_start(struct iscsi_cmnd *req)
+{
+ struct iscsi_conn *conn = req->conn;
+ struct iscsi_session *session = conn->session;
+ struct iscsi_scsi_cmd_hdr *req_hdr = cmnd_hdr(req);
+ struct scst_cmd *scst_cmd;
+ scst_data_direction dir;
+ struct iscsi_ahs_hdr *ahdr;
+ int res = 0;
+
+ TRACE_DBG("scsi command: %x", req_hdr->scb[0]);
+
+ TRACE_DBG("Incrementing active_cmds (cmd %p, sess %p, "
+ "new value %d)", req, session,
+ atomic_read(&session->active_cmds)+1);
+ atomic_inc(&session->active_cmds);
+ req->dec_active_cmnds = 1;
+
+ scst_cmd = scst_rx_cmd(session->scst_sess,
+ (uint8_t *)&req_hdr->lun, sizeof(req_hdr->lun),
+ req_hdr->scb, sizeof(req_hdr->scb), SCST_NON_ATOMIC);
+ if (scst_cmd == NULL) {
+ res = create_preliminary_status_rsp(req, SAM_STAT_BUSY,
+ NULL, 0);
+ goto out;
+ }
+
+ req->scst_cmd = scst_cmd;
+ scst_cmd_set_tag(scst_cmd, req_hdr->itt);
+ scst_cmd_set_tgt_priv(scst_cmd, req);
+
+ if ((req_hdr->flags & ISCSI_CMD_READ) &&
+ (req_hdr->flags & ISCSI_CMD_WRITE)) {
+ int sz = cmnd_read_size(req);
+ if (unlikely(sz < 0)) {
+ PRINT_ERROR("%s", "BIDI data transfer, but initiator "
+ "not supplied Bidirectional Read Expected Data "
+ "Transfer Length AHS");
+ set_scst_preliminary_status_rsp(req, true,
+ SCST_LOAD_SENSE(scst_sense_parameter_value_invalid));
+ } else {
+ req->read_size = sz;
+ dir = SCST_DATA_BIDI;
+ scst_cmd_set_expected(scst_cmd, dir, sz);
+ scst_cmd_set_expected_in_transfer_len(scst_cmd,
+ be32_to_cpu(req_hdr->data_length));
+ scst_cmd_set_tgt_need_alloc_data_buf(scst_cmd);
+ }
+ } else if (req_hdr->flags & ISCSI_CMD_READ) {
+ req->read_size = be32_to_cpu(req_hdr->data_length);
+ dir = SCST_DATA_READ;
+ scst_cmd_set_expected(scst_cmd, dir, req->read_size);
+ scst_cmd_set_tgt_need_alloc_data_buf(scst_cmd);
+ } else if (req_hdr->flags & ISCSI_CMD_WRITE) {
+ dir = SCST_DATA_WRITE;
+ scst_cmd_set_expected(scst_cmd, dir,
+ be32_to_cpu(req_hdr->data_length));
+ } else {
+ dir = SCST_DATA_NONE;
+ scst_cmd_set_expected(scst_cmd, dir, 0);
+ }
+
+ switch (req_hdr->flags & ISCSI_CMD_ATTR_MASK) {
+ case ISCSI_CMD_SIMPLE:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_SIMPLE);
+ break;
+ case ISCSI_CMD_HEAD_OF_QUEUE:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_HEAD_OF_QUEUE);
+ break;
+ case ISCSI_CMD_ORDERED:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_ORDERED);
+ break;
+ case ISCSI_CMD_ACA:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_ACA);
+ break;
+ case ISCSI_CMD_UNTAGGED:
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_UNTAGGED);
+ break;
+ default:
+ PRINT_ERROR("Unknown task code %x, use ORDERED instead",
+ req_hdr->flags & ISCSI_CMD_ATTR_MASK);
+ scst_cmd_set_queue_type(scst_cmd, SCST_CMD_QUEUE_ORDERED);
+ break;
+ }
+
+ /* cmd_sn is already in CPU format converted in check_cmd_sn() */
+ scst_cmd_set_tgt_sn(scst_cmd, req_hdr->cmd_sn);
+
+ ahdr = (struct iscsi_ahs_hdr *)req->pdu.ahs;
+ if (ahdr != NULL) {
+ uint8_t *p = (uint8_t *)ahdr;
+ unsigned int size = 0;
+ do {
+ int s;
+
+ ahdr = (struct iscsi_ahs_hdr *)p;
+
+ if (ahdr->ahstype == ISCSI_AHSTYPE_CDB) {
+ struct iscsi_cdb_ahdr *eca =
+ (struct iscsi_cdb_ahdr *)ahdr;
+ scst_cmd_set_ext_cdb(scst_cmd, eca->cdb,
+ be16_to_cpu(ahdr->ahslength) - 1);
+ break;
+ }
+ s = 3 + be16_to_cpu(ahdr->ahslength);
+ s = (s + 3) & -4;
+ size += s;
+ p += s;
+ } while (size < req->pdu.ahssize);
+ }
+
+ TRACE_DBG("START Command (itt %x, queue_type %d)",
+ req_hdr->itt, scst_cmd_get_queue_type(scst_cmd));
+ req->scst_state = ISCSI_CMD_STATE_RX_CMD;
+ conn->rx_task = current;
+ scst_cmd_init_stage1_done(scst_cmd, SCST_CONTEXT_DIRECT, 0);
+
+ if (req->scst_state != ISCSI_CMD_STATE_RX_CMD)
+ res = cmnd_rx_continue(req);
+ else {
+ TRACE_DBG("Delaying req %p post processing (scst_state %d)",
+ req, req->scst_state);
+ res = 1;
+ }
+
+out:
+ return res;
+}
+
+static int data_out_start(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+ struct iscsi_data_out_hdr *req_hdr =
+ (struct iscsi_data_out_hdr *)&cmnd->pdu.bhs;
+ struct iscsi_cmnd *orig_req;
+#if 0
+ struct iscsi_hdr *orig_req_hdr;
+#endif
+ u32 offset = be32_to_cpu(req_hdr->buffer_offset);
+ int res = 0;
+
+ /*
+ * There is no race with send_r2t() and conn_abort(), since
+ * all functions called from single read thread
+ */
+ iscsi_extracheck_is_rd_thread(cmnd->conn);
+
+ update_stat_sn(cmnd);
+
+ orig_req = cmnd_find_data_wait_hash(conn, req_hdr->itt);
+ cmnd->cmd_req = orig_req;
+ if (unlikely(orig_req == NULL)) {
+ /*
+ * It shouldn't happen, since we don't abort any request until
+ * we received all related PDUs from the initiator or timeout
+ * them. Let's quietly drop such PDUs.
+ */
+ TRACE_MGMT_DBG("Unable to find scsi task ITT %x",
+ cmnd_itt(cmnd));
+ res = iscsi_preliminary_complete(cmnd, cmnd, true);
+ goto out;
+ }
+
+ if (unlikely(orig_req->r2t_len_to_receive < cmnd->pdu.datasize)) {
+ if (orig_req->prelim_compl_flags != 0) {
+ /* We can have fake r2t_len_to_receive */
+ goto go;
+ }
+ PRINT_ERROR("Data size (%d) > R2T length to receive (%d)",
+ cmnd->pdu.datasize, orig_req->r2t_len_to_receive);
+ set_scst_preliminary_status_rsp(orig_req, false,
+ SCST_LOAD_SENSE(iscsi_sense_incorrect_amount_of_data));
+ goto go;
+ }
+
+ /* Crazy iSCSI spec requires us to make this unneeded check */
+#if 0 /* ...but some initiators (Windows) don't care to correctly set it */
+ orig_req_hdr = &orig_req->pdu.bhs;
+ if (unlikely(orig_req_hdr->lun != req_hdr->lun)) {
+ PRINT_ERROR("Wrong LUN (%lld) in Data-Out PDU (expected %lld), "
+ "orig_req %p, cmnd %p", (unsigned long long)req_hdr->lun,
+ (unsigned long long)orig_req_hdr->lun, orig_req, cmnd);
+ create_reject_rsp(orig_req, ISCSI_REASON_PROTOCOL_ERROR, false);
+ goto go;
+ }
+#endif
+
+go:
+ if (req_hdr->flags & ISCSI_FLG_FINAL)
+ orig_req->outstanding_r2t--;
+
+ if (unlikely(orig_req->prelim_compl_flags != 0)) {
+ res = iscsi_preliminary_complete(cmnd, orig_req, true);
+ goto out;
+ }
+
+ TRACE_WRITE("cmnd %p, orig_req %p, offset %u, datasize %u", cmnd,
+ orig_req, offset, cmnd->pdu.datasize);
+
+ res = cmnd_prepare_recv_pdu(conn, orig_req, offset, cmnd->pdu.datasize);
+
+out:
+ return res;
+}
+
+static void data_out_end(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_data_out_hdr *req_hdr =
+ (struct iscsi_data_out_hdr *)&cmnd->pdu.bhs;
+ struct iscsi_cmnd *req;
+
+ EXTRACHECKS_BUG_ON(cmnd == NULL);
+ req = cmnd->cmd_req;
+ if (unlikely(req == NULL))
+ goto out;
+
+ TRACE_DBG("cmnd %p, req %p", cmnd, req);
+
+ iscsi_extracheck_is_rd_thread(cmnd->conn);
+
+ if (!(cmnd->conn->ddigest_type & DIGEST_NONE) &&
+ !cmnd->ddigest_checked) {
+ cmd_add_on_rx_ddigest_list(req, cmnd);
+ cmnd_get(cmnd);
+ }
+
+ /*
+ * Now we received the data and can adjust r2t_len_to_receive of the
+ * orig req. We couldn't do it earlier, because it will break data
+ * receiving errors recovery (calls of iscsi_fail_data_waiting_cmnd()).
+ */
+ req->r2t_len_to_receive -= cmnd->pdu.datasize;
+
+ if (unlikely(req->prelim_compl_flags != 0)) {
+ /*
+ * We might need to wait for one or more PDUs. Let's simplify
+ * other code.
+ */
+ req->r2t_len_to_receive = req->outstanding_r2t;
+ req->r2t_len_to_send = 0;
+ }
+
+ TRACE_DBG("req %p, FINAL %x, outstanding_r2t %d, r2t_len_to_receive %d,"
+ " r2t_len_to_send %d", req, req_hdr->flags & ISCSI_FLG_FINAL,
+ req->outstanding_r2t, req->r2t_len_to_receive,
+ req->r2t_len_to_send);
+
+ if (!(req_hdr->flags & ISCSI_FLG_FINAL))
+ goto out;
+
+ if (req->r2t_len_to_receive == 0) {
+ if (!req->pending)
+ iscsi_restart_cmnd(req);
+ } else if (req->r2t_len_to_send != 0)
+ send_r2t(req);
+
+out:
+ return;
+}
+
+/* Might be called under target_mutex and cmd_list_lock */
+static void __cmnd_abort(struct iscsi_cmnd *cmnd)
+{
+ unsigned long timeout_time = jiffies + ISCSI_TM_DATA_WAIT_TIMEOUT +
+ ISCSI_ADD_SCHED_TIME;
+ struct iscsi_conn *conn = cmnd->conn;
+
+ TRACE_MGMT_DBG("Aborting cmd %p, scst_cmd %p (scst state %x, "
+ "ref_cnt %d, on_write_timeout_list %d, write_start %ld, ITT %x, "
+ "sn %u, op %x, r2t_len_to_receive %d, r2t_len_to_send %d, "
+ "CDB op %x, size to write %u, outstanding_r2t %d, "
+ "sess->exp_cmd_sn %u, conn %p, rd_task %p)",
+ cmnd, cmnd->scst_cmd, cmnd->scst_state,
+ atomic_read(&cmnd->ref_cnt), cmnd->on_write_timeout_list,
+ cmnd->write_start, cmnd_itt(cmnd), cmnd->pdu.bhs.sn,
+ cmnd_opcode(cmnd), cmnd->r2t_len_to_receive,
+ cmnd->r2t_len_to_send, cmnd_scsicode(cmnd),
+ cmnd_write_size(cmnd), cmnd->outstanding_r2t,
+ cmnd->conn->session->exp_cmd_sn, cmnd->conn,
+ cmnd->conn->rd_task);
+
+ /*
+ * Lock to sync with iscsi_check_tm_data_wait_timeouts(), including
+ * CMD_ABORTED bit set.
+ */
+ spin_lock_bh(&iscsi_rd_lock);
+
+ /*
+ * We suppose that preliminary commands completion is tested by
+ * comparing prelim_compl_flags with 0. Otherwise a race is possible,
+ * like sending command in SCST core as PRELIM_COMPLETED, while it
+ * wasn't aborted in it yet and have as the result a wrong success
+ * status sent to the initiator.
+ */
+ set_bit(ISCSI_CMD_ABORTED, &cmnd->prelim_compl_flags);
+
+ TRACE_MGMT_DBG("Setting conn_tm_active for conn %p", conn);
+ conn->conn_tm_active = 1;
+
+ spin_unlock_bh(&iscsi_rd_lock);
+
+ /*
+ * We need the lock to sync with req_add_to_write_timeout_list() and
+ * close races for rsp_timer.expires.
+ */
+ spin_lock_bh(&conn->write_list_lock);
+ if (!timer_pending(&conn->rsp_timer) ||
+ time_after(conn->rsp_timer.expires, timeout_time)) {
+ TRACE_MGMT_DBG("Mod timer on %ld (conn %p)", timeout_time,
+ conn);
+ mod_timer(&conn->rsp_timer, timeout_time);
+ } else
+ TRACE_MGMT_DBG("Timer for conn %p is going to fire on %ld "
+ "(timeout time %ld)", conn, conn->rsp_timer.expires,
+ timeout_time);
+ spin_unlock_bh(&conn->write_list_lock);
+
+ return;
+}
+
+/* Must be called from the read or conn close thread */
+static int cmnd_abort(struct iscsi_cmnd *req, int *status)
+{
+ struct iscsi_task_mgt_hdr *req_hdr =
+ (struct iscsi_task_mgt_hdr *)&req->pdu.bhs;
+ struct iscsi_cmnd *cmnd;
+ int res = -1;
+
+ req_hdr->ref_cmd_sn = be32_to_cpu(req_hdr->ref_cmd_sn);
+
+ if (!before(req_hdr->ref_cmd_sn, req_hdr->cmd_sn)) {
+ TRACE(TRACE_MGMT, "ABORT TASK: RefCmdSN(%u) > CmdSN(%u)",
+ req_hdr->ref_cmd_sn, req_hdr->cmd_sn);
+ *status = ISCSI_RESPONSE_UNKNOWN_TASK;
+ goto out;
+ }
+
+ cmnd = cmnd_find_itt_get(req->conn, req_hdr->rtt);
+ if (cmnd) {
+ struct iscsi_conn *conn = cmnd->conn;
+ struct iscsi_scsi_cmd_hdr *hdr = cmnd_hdr(cmnd);
+
+ if (req_hdr->lun != hdr->lun) {
+ PRINT_ERROR("ABORT TASK: LUN mismatch: req LUN "
+ "%llx, cmd LUN %llx, rtt %u",
+ (long long unsigned int)req_hdr->lun,
+ (long long unsigned int)hdr->lun,
+ req_hdr->rtt);
+ *status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ goto out_put;
+ }
+
+ if (cmnd->pdu.bhs.opcode & ISCSI_OP_IMMEDIATE) {
+ if (req_hdr->ref_cmd_sn != req_hdr->cmd_sn) {
+ PRINT_ERROR("ABORT TASK: RefCmdSN(%u) != TM "
+ "cmd CmdSN(%u) for immediate command "
+ "%p", req_hdr->ref_cmd_sn,
+ req_hdr->cmd_sn, cmnd);
+ *status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ goto out_put;
+ }
+ } else {
+ if (req_hdr->ref_cmd_sn != hdr->cmd_sn) {
+ PRINT_ERROR("ABORT TASK: RefCmdSN(%u) != "
+ "CmdSN(%u) for command %p",
+ req_hdr->ref_cmd_sn, req_hdr->cmd_sn,
+ cmnd);
+ *status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ goto out_put;
+ }
+ }
+
+ if (before(req_hdr->cmd_sn, hdr->cmd_sn) ||
+ (req_hdr->cmd_sn == hdr->cmd_sn)) {
+ PRINT_ERROR("ABORT TASK: SN mismatch: req SN %x, "
+ "cmd SN %x, rtt %u", req_hdr->cmd_sn,
+ hdr->cmd_sn, req_hdr->rtt);
+ *status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ goto out_put;
+ }
+
+ spin_lock_bh(&conn->cmd_list_lock);
+ __cmnd_abort(cmnd);
+ spin_unlock_bh(&conn->cmd_list_lock);
+
+ cmnd_put(cmnd);
+ res = 0;
+ } else {
+ TRACE_MGMT_DBG("cmd RTT %x not found", req_hdr->rtt);
+ /*
+ * iSCSI RFC:
+ *
+ * b) If the Referenced Task Tag does not identify an existing task,
+ * but if the CmdSN indicated by the RefCmdSN field in the Task
+ * Management function request is within the valid CmdSN window
+ * and less than the CmdSN of the Task Management function
+ * request itself, then targets must consider the CmdSN received
+ * and return the "Function complete" response.
+ *
+ * c) If the Referenced Task Tag does not identify an existing task
+ * and if the CmdSN indicated by the RefCmdSN field in the Task
+ * Management function request is outside the valid CmdSN window,
+ * then targets must return the "Task does not exist" response.
+ *
+ * 128 seems to be a good "window".
+ */
+ if (between(req_hdr->ref_cmd_sn, req_hdr->cmd_sn - 128,
+ req_hdr->cmd_sn)) {
+ *status = ISCSI_RESPONSE_FUNCTION_COMPLETE;
+ res = 0;
+ } else
+ *status = ISCSI_RESPONSE_UNKNOWN_TASK;
+ }
+
+out:
+ return res;
+
+out_put:
+ cmnd_put(cmnd);
+ goto out;
+}
+
+/* Must be called from the read or conn close thread */
+static int target_abort(struct iscsi_cmnd *req, int all)
+{
+ struct iscsi_target *target = req->conn->session->target;
+ struct iscsi_task_mgt_hdr *req_hdr =
+ (struct iscsi_task_mgt_hdr *)&req->pdu.bhs;
+ struct iscsi_session *session;
+ struct iscsi_conn *conn;
+ struct iscsi_cmnd *cmnd;
+
+ mutex_lock(&target->target_mutex);
+
+ list_for_each_entry(session, &target->session_list,
+ session_list_entry) {
+ list_for_each_entry(conn, &session->conn_list,
+ conn_list_entry) {
+ spin_lock_bh(&conn->cmd_list_lock);
+ list_for_each_entry(cmnd, &conn->cmd_list,
+ cmd_list_entry) {
+ if (cmnd == req)
+ continue;
+ if (all)
+ __cmnd_abort(cmnd);
+ else if (req_hdr->lun == cmnd_hdr(cmnd)->lun)
+ __cmnd_abort(cmnd);
+ }
+ spin_unlock_bh(&conn->cmd_list_lock);
+ }
+ }
+
+ mutex_unlock(&target->target_mutex);
+ return 0;
+}
+
+/* Must be called from the read or conn close thread */
+static void task_set_abort(struct iscsi_cmnd *req)
+{
+ struct iscsi_session *session = req->conn->session;
+ struct iscsi_task_mgt_hdr *req_hdr =
+ (struct iscsi_task_mgt_hdr *)&req->pdu.bhs;
+ struct iscsi_target *target = session->target;
+ struct iscsi_conn *conn;
+ struct iscsi_cmnd *cmnd;
+
+ mutex_lock(&target->target_mutex);
+
+ list_for_each_entry(conn, &session->conn_list, conn_list_entry) {
+ spin_lock_bh(&conn->cmd_list_lock);
+ list_for_each_entry(cmnd, &conn->cmd_list, cmd_list_entry) {
+ struct iscsi_scsi_cmd_hdr *hdr = cmnd_hdr(cmnd);
+ if (cmnd == req)
+ continue;
+ if (req_hdr->lun != hdr->lun)
+ continue;
+ if (before(req_hdr->cmd_sn, hdr->cmd_sn) ||
+ req_hdr->cmd_sn == hdr->cmd_sn)
+ continue;
+ __cmnd_abort(cmnd);
+ }
+ spin_unlock_bh(&conn->cmd_list_lock);
+ }
+
+ mutex_unlock(&target->target_mutex);
+ return;
+}
+
+/* Must be called from the read or conn close thread */
+void conn_abort(struct iscsi_conn *conn)
+{
+ struct iscsi_cmnd *cmnd, *r, *t;
+
+ TRACE_MGMT_DBG("Aborting conn %p", conn);
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+ cancel_delayed_work_sync(&conn->nop_in_delayed_work);
+
+ /* No locks, we are the only user */
+ list_for_each_entry_safe(r, t, &conn->nop_req_list,
+ nop_req_list_entry) {
+ list_del(&r->nop_req_list_entry);
+ cmnd_put(r);
+ }
+
+ spin_lock_bh(&conn->cmd_list_lock);
+again:
+ list_for_each_entry(cmnd, &conn->cmd_list, cmd_list_entry) {
+ __cmnd_abort(cmnd);
+ if (cmnd->r2t_len_to_receive != 0) {
+ if (!cmnd_get_check(cmnd)) {
+ spin_unlock_bh(&conn->cmd_list_lock);
+
+ /* ToDo: this is racy for MC/S */
+ iscsi_fail_data_waiting_cmnd(cmnd);
+
+ cmnd_put(cmnd);
+
+ /*
+ * We are in the read thread, so we may not
+ * worry that after cmnd release conn gets
+ * released as well.
+ */
+ spin_lock_bh(&conn->cmd_list_lock);
+ goto again;
+ }
+ }
+ }
+ spin_unlock_bh(&conn->cmd_list_lock);
+
+ return;
+}
+
+static void execute_task_management(struct iscsi_cmnd *req)
+{
+ struct iscsi_conn *conn = req->conn;
+ struct iscsi_session *sess = conn->session;
+ struct iscsi_task_mgt_hdr *req_hdr =
+ (struct iscsi_task_mgt_hdr *)&req->pdu.bhs;
+ int rc, status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ int function = req_hdr->function & ISCSI_FUNCTION_MASK;
+ struct scst_rx_mgmt_params params;
+
+ TRACE(TRACE_MGMT, "TM fn %d", function);
+
+ TRACE_MGMT_DBG("TM req %p, ITT %x, RTT %x, sn %u, con %p", req,
+ cmnd_itt(req), req_hdr->rtt, req_hdr->cmd_sn, conn);
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+ spin_lock(&sess->sn_lock);
+ sess->tm_active++;
+ sess->tm_sn = req_hdr->cmd_sn;
+ if (sess->tm_rsp != NULL) {
+ struct iscsi_cmnd *tm_rsp = sess->tm_rsp;
+
+ TRACE_MGMT_DBG("Dropping delayed TM rsp %p", tm_rsp);
+
+ sess->tm_rsp = NULL;
+ sess->tm_active--;
+
+ spin_unlock(&sess->sn_lock);
+
+ BUG_ON(sess->tm_active < 0);
+
+ rsp_cmnd_release(tm_rsp);
+ } else
+ spin_unlock(&sess->sn_lock);
+
+ memset(¶ms, 0, sizeof(params));
+ params.atomic = SCST_NON_ATOMIC;
+ params.tgt_priv = req;
+
+ if ((function != ISCSI_FUNCTION_ABORT_TASK) &&
+ (req_hdr->rtt != ISCSI_RESERVED_TAG)) {
+ PRINT_ERROR("Invalid RTT %x (TM fn %d)", req_hdr->rtt,
+ function);
+ rc = -1;
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ goto reject;
+ }
+
+ /* cmd_sn is already in CPU format converted in check_cmd_sn() */
+
+ switch (function) {
+ case ISCSI_FUNCTION_ABORT_TASK:
+ rc = cmnd_abort(req, &status);
+ if (rc == 0) {
+ params.fn = SCST_ABORT_TASK;
+ params.tag = req_hdr->rtt;
+ params.tag_set = 1;
+ params.lun = (uint8_t *)&req_hdr->lun;
+ params.lun_len = sizeof(req_hdr->lun);
+ params.lun_set = 1;
+ params.cmd_sn = req_hdr->cmd_sn;
+ params.cmd_sn_set = 1;
+ rc = scst_rx_mgmt_fn(conn->session->scst_sess,
+ ¶ms);
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ }
+ break;
+ case ISCSI_FUNCTION_ABORT_TASK_SET:
+ task_set_abort(req);
+ params.fn = SCST_ABORT_TASK_SET;
+ params.lun = (uint8_t *)&req_hdr->lun;
+ params.lun_len = sizeof(req_hdr->lun);
+ params.lun_set = 1;
+ params.cmd_sn = req_hdr->cmd_sn;
+ params.cmd_sn_set = 1;
+ rc = scst_rx_mgmt_fn(conn->session->scst_sess,
+ ¶ms);
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ break;
+ case ISCSI_FUNCTION_CLEAR_TASK_SET:
+ task_set_abort(req);
+ params.fn = SCST_CLEAR_TASK_SET;
+ params.lun = (uint8_t *)&req_hdr->lun;
+ params.lun_len = sizeof(req_hdr->lun);
+ params.lun_set = 1;
+ params.cmd_sn = req_hdr->cmd_sn;
+ params.cmd_sn_set = 1;
+ rc = scst_rx_mgmt_fn(conn->session->scst_sess,
+ ¶ms);
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ break;
+ case ISCSI_FUNCTION_CLEAR_ACA:
+ params.fn = SCST_CLEAR_ACA;
+ params.lun = (uint8_t *)&req_hdr->lun;
+ params.lun_len = sizeof(req_hdr->lun);
+ params.lun_set = 1;
+ params.cmd_sn = req_hdr->cmd_sn;
+ params.cmd_sn_set = 1;
+ rc = scst_rx_mgmt_fn(conn->session->scst_sess,
+ ¶ms);
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ break;
+ case ISCSI_FUNCTION_TARGET_COLD_RESET:
+ case ISCSI_FUNCTION_TARGET_WARM_RESET:
+ target_abort(req, 1);
+ params.fn = SCST_TARGET_RESET;
+ params.cmd_sn = req_hdr->cmd_sn;
+ params.cmd_sn_set = 1;
+ rc = scst_rx_mgmt_fn(conn->session->scst_sess,
+ ¶ms);
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ break;
+ case ISCSI_FUNCTION_LOGICAL_UNIT_RESET:
+ target_abort(req, 0);
+ params.fn = SCST_LUN_RESET;
+ params.lun = (uint8_t *)&req_hdr->lun;
+ params.lun_len = sizeof(req_hdr->lun);
+ params.lun_set = 1;
+ params.cmd_sn = req_hdr->cmd_sn;
+ params.cmd_sn_set = 1;
+ rc = scst_rx_mgmt_fn(conn->session->scst_sess,
+ ¶ms);
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ break;
+ case ISCSI_FUNCTION_TASK_REASSIGN:
+ rc = -1;
+ status = ISCSI_RESPONSE_ALLEGIANCE_REASSIGNMENT_UNSUPPORTED;
+ break;
+ default:
+ PRINT_ERROR("Unknown TM function %d", function);
+ rc = -1;
+ status = ISCSI_RESPONSE_FUNCTION_REJECTED;
+ break;
+ }
+
+reject:
+ if (rc != 0)
+ iscsi_send_task_mgmt_resp(req, status);
+
+ return;
+}
+
+static void nop_out_exec(struct iscsi_cmnd *req)
+{
+ struct iscsi_cmnd *rsp;
+ struct iscsi_nop_in_hdr *rsp_hdr;
+
+ TRACE_DBG("%p", req);
+
+ if (cmnd_itt(req) != cpu_to_be32(ISCSI_RESERVED_TAG)) {
+ rsp = iscsi_alloc_main_rsp(req);
+
+ rsp_hdr = (struct iscsi_nop_in_hdr *)&rsp->pdu.bhs;
+ rsp_hdr->opcode = ISCSI_OP_NOP_IN;
+ rsp_hdr->flags = ISCSI_FLG_FINAL;
+ rsp_hdr->itt = req->pdu.bhs.itt;
+ rsp_hdr->ttt = cpu_to_be32(ISCSI_RESERVED_TAG);
+
+ if (req->pdu.datasize)
+ BUG_ON(req->sg == NULL);
+ else
+ BUG_ON(req->sg != NULL);
+
+ if (req->sg) {
+ rsp->sg = req->sg;
+ rsp->sg_cnt = req->sg_cnt;
+ rsp->bufflen = req->bufflen;
+ }
+
+ /* We already checked it in check_segment_length() */
+ BUG_ON(get_pgcnt(req->pdu.datasize, 0) > ISCSI_CONN_IOV_MAX);
+
+ rsp->pdu.datasize = req->pdu.datasize;
+ } else {
+ bool found = false;
+ struct iscsi_cmnd *r;
+ struct iscsi_conn *conn = req->conn;
+
+ TRACE_DBG("Receive Nop-In response (ttt 0x%08x)",
+ be32_to_cpu(cmnd_ttt(req)));
+
+ spin_lock_bh(&conn->nop_req_list_lock);
+ list_for_each_entry(r, &conn->nop_req_list,
+ nop_req_list_entry) {
+ if (cmnd_ttt(req) == cmnd_ttt(r)) {
+ list_del(&r->nop_req_list_entry);
+ found = true;
+ break;
+ }
+ }
+ spin_unlock_bh(&conn->nop_req_list_lock);
+
+ if (found)
+ cmnd_put(r);
+ else
+ TRACE_MGMT_DBG("%s", "Got Nop-out response without "
+ "corresponding Nop-In request");
+ }
+
+ req_cmnd_release(req);
+ return;
+}
+
+static void logout_exec(struct iscsi_cmnd *req)
+{
+ struct iscsi_logout_req_hdr *req_hdr;
+ struct iscsi_cmnd *rsp;
+ struct iscsi_logout_rsp_hdr *rsp_hdr;
+
+ PRINT_INFO("Logout received from initiator %s",
+ req->conn->session->initiator_name);
+ TRACE_DBG("%p", req);
+
+ req_hdr = (struct iscsi_logout_req_hdr *)&req->pdu.bhs;
+ rsp = iscsi_alloc_main_rsp(req);
+ rsp_hdr = (struct iscsi_logout_rsp_hdr *)&rsp->pdu.bhs;
+ rsp_hdr->opcode = ISCSI_OP_LOGOUT_RSP;
+ rsp_hdr->flags = ISCSI_FLG_FINAL;
+ rsp_hdr->itt = req_hdr->itt;
+ rsp->should_close_conn = 1;
+
+ req_cmnd_release(req);
+
+ return;
+}
+
+static void iscsi_cmnd_exec(struct iscsi_cmnd *cmnd)
+{
+
+ TRACE_DBG("cmnd %p, op %x, SN %u", cmnd, cmnd_opcode(cmnd),
+ cmnd->pdu.bhs.sn);
+
+ iscsi_extracheck_is_rd_thread(cmnd->conn);
+
+ if (cmnd_opcode(cmnd) == ISCSI_OP_SCSI_CMD) {
+ if (cmnd->r2t_len_to_receive == 0)
+ iscsi_restart_cmnd(cmnd);
+ else if (cmnd->r2t_len_to_send != 0)
+ send_r2t(cmnd);
+ goto out;
+ }
+
+ if (cmnd->prelim_compl_flags != 0) {
+ TRACE_MGMT_DBG("Terminating prelim completed non-SCSI cmnd %p "
+ "(op %x)", cmnd, cmnd_opcode(cmnd));
+ req_cmnd_release(cmnd);
+ goto out;
+ }
+
+ switch (cmnd_opcode(cmnd)) {
+ case ISCSI_OP_NOP_OUT:
+ nop_out_exec(cmnd);
+ break;
+ case ISCSI_OP_SCSI_TASK_MGT_MSG:
+ execute_task_management(cmnd);
+ break;
+ case ISCSI_OP_LOGOUT_CMD:
+ logout_exec(cmnd);
+ break;
+ default:
+ PRINT_CRIT_ERROR("Unexpected cmnd op %x", cmnd_opcode(cmnd));
+ BUG();
+ break;
+ }
+
+out:
+ return;
+}
+
+static void set_cork(struct socket *sock, int on)
+{
+ int opt = on;
+ mm_segment_t oldfs;
+
+ oldfs = get_fs();
+ set_fs(get_ds());
+ sock->ops->setsockopt(sock, SOL_TCP, TCP_CORK,
+ (void __force __user *)&opt, sizeof(opt));
+ set_fs(oldfs);
+ return;
+}
+
+void cmnd_tx_start(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+
+ TRACE_DBG("conn %p, cmnd %p, opcode %x", conn, cmnd, cmnd_opcode(cmnd));
+ iscsi_cmnd_set_length(&cmnd->pdu);
+
+ iscsi_extracheck_is_wr_thread(conn);
+
+ set_cork(conn->sock, 1);
+
+ conn->write_iop = conn->write_iov;
+ conn->write_iop->iov_base = (void __force __user *)(&cmnd->pdu.bhs);
+ conn->write_iop->iov_len = sizeof(cmnd->pdu.bhs);
+ conn->write_iop_used = 1;
+ conn->write_size = sizeof(cmnd->pdu.bhs) + cmnd->pdu.datasize;
+ conn->write_offset = 0;
+
+ switch (cmnd_opcode(cmnd)) {
+ case ISCSI_OP_NOP_IN:
+ if (cmnd_itt(cmnd) == cpu_to_be32(ISCSI_RESERVED_TAG))
+ cmnd->pdu.bhs.sn = cmnd_set_sn(cmnd, 0);
+ else
+ cmnd_set_sn(cmnd, 1);
+ break;
+ case ISCSI_OP_SCSI_RSP:
+ cmnd_set_sn(cmnd, 1);
+ break;
+ case ISCSI_OP_SCSI_TASK_MGT_RSP:
+ cmnd_set_sn(cmnd, 1);
+ break;
+ case ISCSI_OP_TEXT_RSP:
+ cmnd_set_sn(cmnd, 1);
+ break;
+ case ISCSI_OP_SCSI_DATA_IN:
+ {
+ struct iscsi_data_in_hdr *rsp =
+ (struct iscsi_data_in_hdr *)&cmnd->pdu.bhs;
+ u32 offset = cpu_to_be32(rsp->buffer_offset);
+
+ TRACE_DBG("cmnd %p, offset %u, datasize %u, bufflen %u", cmnd,
+ offset, cmnd->pdu.datasize, cmnd->bufflen);
+
+ BUG_ON(offset > cmnd->bufflen);
+ BUG_ON(offset + cmnd->pdu.datasize > cmnd->bufflen);
+
+ conn->write_offset = offset;
+
+ cmnd_set_sn(cmnd, (rsp->flags & ISCSI_FLG_FINAL) ? 1 : 0);
+ break;
+ }
+ case ISCSI_OP_LOGOUT_RSP:
+ cmnd_set_sn(cmnd, 1);
+ break;
+ case ISCSI_OP_R2T:
+ cmnd->pdu.bhs.sn = cmnd_set_sn(cmnd, 0);
+ break;
+ case ISCSI_OP_ASYNC_MSG:
+ cmnd_set_sn(cmnd, 1);
+ break;
+ case ISCSI_OP_REJECT:
+ cmnd_set_sn(cmnd, 1);
+ break;
+ default:
+ PRINT_ERROR("Unexpected cmnd op %x", cmnd_opcode(cmnd));
+ break;
+ }
+
+ iscsi_dump_pdu(&cmnd->pdu);
+ return;
+}
+
+void cmnd_tx_end(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+
+ TRACE_DBG("%p:%x (should_close_conn %d, should_close_all_conn %d)",
+ cmnd, cmnd_opcode(cmnd), cmnd->should_close_conn,
+ cmnd->should_close_all_conn);
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ switch (cmnd_opcode(cmnd)) {
+ case ISCSI_OP_NOP_IN:
+ case ISCSI_OP_SCSI_RSP:
+ case ISCSI_OP_SCSI_TASK_MGT_RSP:
+ case ISCSI_OP_TEXT_RSP:
+ case ISCSI_OP_R2T:
+ case ISCSI_OP_ASYNC_MSG:
+ case ISCSI_OP_REJECT:
+ case ISCSI_OP_SCSI_DATA_IN:
+ case ISCSI_OP_LOGOUT_RSP:
+ break;
+ default:
+ PRINT_CRIT_ERROR("unexpected cmnd op %x", cmnd_opcode(cmnd));
+ BUG();
+ break;
+ }
+#endif
+
+ if (unlikely(cmnd->should_close_conn)) {
+ if (cmnd->should_close_all_conn) {
+ PRINT_INFO("Closing all connections for target %x at "
+ "initiator's %s request",
+ cmnd->conn->session->target->tid,
+ conn->session->initiator_name);
+ target_del_all_sess(cmnd->conn->session->target, 0);
+ } else {
+ PRINT_INFO("Closing connection at initiator's %s "
+ "request", conn->session->initiator_name);
+ mark_conn_closed(conn);
+ }
+ }
+
+ set_cork(cmnd->conn->sock, 0);
+ return;
+}
+
+/*
+ * Push the command for execution. This functions reorders the commands.
+ * Called from the read thread.
+ *
+ * Basically, since we don't support MC/S and TCP guarantees data delivery
+ * order, all that SN's stuff isn't needed at all (commands delivery order is
+ * a natural commands execution order), but insane iSCSI spec requires
+ * us to check it and we have to, because some crazy initiators can rely
+ * on the SN's based order and reorder requests during sending. For all other
+ * normal initiators all that code is a NOP.
+ */
+static void iscsi_push_cmnd(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_session *session = cmnd->conn->session;
+ struct list_head *entry;
+ u32 cmd_sn;
+
+ TRACE_DBG("cmnd %p, iSCSI opcode %x, sn %u, exp sn %u", cmnd,
+ cmnd_opcode(cmnd), cmnd->pdu.bhs.sn, session->exp_cmd_sn);
+
+ iscsi_extracheck_is_rd_thread(cmnd->conn);
+
+ BUG_ON(cmnd->parent_req != NULL);
+
+ if (cmnd->pdu.bhs.opcode & ISCSI_OP_IMMEDIATE) {
+ TRACE_DBG("Immediate cmd %p (cmd_sn %u)", cmnd,
+ cmnd->pdu.bhs.sn);
+ iscsi_cmnd_exec(cmnd);
+ goto out;
+ }
+
+ spin_lock(&session->sn_lock);
+
+ cmd_sn = cmnd->pdu.bhs.sn;
+ if (cmd_sn == session->exp_cmd_sn) {
+ while (1) {
+ session->exp_cmd_sn = ++cmd_sn;
+
+ if (unlikely(session->tm_active > 0)) {
+ if (before(cmd_sn, session->tm_sn)) {
+ struct iscsi_conn *conn = cmnd->conn;
+
+ spin_unlock(&session->sn_lock);
+
+ spin_lock_bh(&conn->cmd_list_lock);
+ __cmnd_abort(cmnd);
+ spin_unlock_bh(&conn->cmd_list_lock);
+
+ spin_lock(&session->sn_lock);
+ }
+ iscsi_check_send_delayed_tm_resp(session);
+ }
+
+ spin_unlock(&session->sn_lock);
+
+ iscsi_cmnd_exec(cmnd);
+
+ spin_lock(&session->sn_lock);
+
+ if (list_empty(&session->pending_list))
+ break;
+ cmnd = list_entry(session->pending_list.next,
+ struct iscsi_cmnd,
+ pending_list_entry);
+ if (cmnd->pdu.bhs.sn != cmd_sn)
+ break;
+
+ list_del(&cmnd->pending_list_entry);
+ cmnd->pending = 0;
+
+ TRACE_MGMT_DBG("Processing pending cmd %p (cmd_sn %u)",
+ cmnd, cmd_sn);
+ }
+ } else {
+ int drop = 0;
+
+ TRACE_DBG("Pending cmd %p (cmd_sn %u, exp_cmd_sn %u)",
+ cmnd, cmd_sn, session->exp_cmd_sn);
+
+ /*
+ * iSCSI RFC 3720: "The target MUST silently ignore any
+ * non-immediate command outside of [from ExpCmdSN to MaxCmdSN
+ * inclusive] range". But we won't honor the MaxCmdSN
+ * requirement, because, since we adjust MaxCmdSN from the
+ * separate write thread, rarely it is possible that initiator
+ * can legally send command with CmdSN>MaxSN. But it won't
+ * hurt anything, in the worst case it will lead to
+ * additional QUEUE FULL status.
+ */
+
+ if (unlikely(before(cmd_sn, session->exp_cmd_sn))) {
+ PRINT_ERROR("Unexpected cmd_sn (%u,%u)", cmd_sn,
+ session->exp_cmd_sn);
+ drop = 1;
+ }
+
+#if 0
+ if (unlikely(after(cmd_sn, session->exp_cmd_sn +
+ iscsi_get_allowed_cmds(session)))) {
+ TRACE_MGMT_DBG("Too large cmd_sn %u (exp_cmd_sn %u, "
+ "max_sn %u)", cmd_sn, session->exp_cmd_sn,
+ iscsi_get_allowed_cmds(session));
+ }
+#endif
+
+ spin_unlock(&session->sn_lock);
+
+ if (unlikely(drop)) {
+ req_cmnd_release_force(cmnd);
+ goto out;
+ }
+
+ if (unlikely(test_bit(ISCSI_CMD_ABORTED,
+ &cmnd->prelim_compl_flags))) {
+ struct iscsi_cmnd *tm_clone;
+
+ TRACE_MGMT_DBG("Pending aborted cmnd %p, creating TM "
+ "clone (scst cmd %p, state %d)", cmnd,
+ cmnd->scst_cmd, cmnd->scst_state);
+
+ tm_clone = cmnd_alloc(cmnd->conn, NULL);
+ if (tm_clone != NULL) {
+ set_bit(ISCSI_CMD_ABORTED,
+ &tm_clone->prelim_compl_flags);
+ tm_clone->pdu = cmnd->pdu;
+
+ TRACE_MGMT_DBG("TM clone %p created",
+ tm_clone);
+
+ iscsi_cmnd_exec(cmnd);
+ cmnd = tm_clone;
+ } else
+ PRINT_ERROR("%s", "Unable to create TM clone");
+ }
+
+ TRACE_MGMT_DBG("Pending cmnd %p (op %x, sn %u, exp sn %u)",
+ cmnd, cmnd_opcode(cmnd), cmd_sn, session->exp_cmd_sn);
+
+ spin_lock(&session->sn_lock);
+ list_for_each(entry, &session->pending_list) {
+ struct iscsi_cmnd *tmp =
+ list_entry(entry, struct iscsi_cmnd,
+ pending_list_entry);
+ if (before(cmd_sn, tmp->pdu.bhs.sn))
+ break;
+ }
+ list_add_tail(&cmnd->pending_list_entry, entry);
+ cmnd->pending = 1;
+ }
+
+ spin_unlock(&session->sn_lock);
+out:
+ return;
+}
+
+static int check_segment_length(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+ struct iscsi_session *session = conn->session;
+
+ if (unlikely(cmnd->pdu.datasize > session->sess_params.max_recv_data_length)) {
+ PRINT_ERROR("Initiator %s violated negotiated parameters: "
+ "data too long (ITT %x, datasize %u, "
+ "max_recv_data_length %u", session->initiator_name,
+ cmnd_itt(cmnd), cmnd->pdu.datasize,
+ session->sess_params.max_recv_data_length);
+ mark_conn_closed(conn);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int cmnd_rx_start(struct iscsi_cmnd *cmnd)
+{
+ int res, rc;
+
+ iscsi_dump_pdu(&cmnd->pdu);
+
+ res = check_segment_length(cmnd);
+ if (res != 0)
+ goto out;
+
+ switch (cmnd_opcode(cmnd)) {
+ case ISCSI_OP_SCSI_CMD:
+ res = scsi_cmnd_start(cmnd);
+ if (unlikely(res < 0))
+ goto out;
+ spin_lock(&cmnd->conn->session->sn_lock);
+ __update_stat_sn(cmnd);
+ rc = check_cmd_sn(cmnd);
+ spin_unlock(&cmnd->conn->session->sn_lock);
+ break;
+ case ISCSI_OP_SCSI_DATA_OUT:
+ res = data_out_start(cmnd);
+ goto out;
+ case ISCSI_OP_NOP_OUT:
+ rc = nop_out_start(cmnd);
+ break;
+ case ISCSI_OP_SCSI_TASK_MGT_MSG:
+ case ISCSI_OP_LOGOUT_CMD:
+ spin_lock(&cmnd->conn->session->sn_lock);
+ __update_stat_sn(cmnd);
+ rc = check_cmd_sn(cmnd);
+ spin_unlock(&cmnd->conn->session->sn_lock);
+ break;
+ case ISCSI_OP_TEXT_CMD:
+ case ISCSI_OP_SNACK_CMD:
+ default:
+ rc = -ISCSI_REASON_UNSUPPORTED_COMMAND;
+ break;
+ }
+
+ if (unlikely(rc < 0)) {
+ PRINT_ERROR("Error %d (iSCSI opcode %x, ITT %x)", rc,
+ cmnd_opcode(cmnd), cmnd_itt(cmnd));
+ res = create_reject_rsp(cmnd, -rc, true);
+ }
+
+out:
+ return res;
+}
+
+void cmnd_rx_end(struct iscsi_cmnd *cmnd)
+{
+
+ TRACE_DBG("cmnd %p, opcode %x", cmnd, cmnd_opcode(cmnd));
+
+ cmnd->conn->last_rcv_time = jiffies;
+ TRACE_DBG("Updated last_rcv_time %ld", cmnd->conn->last_rcv_time);
+
+ switch (cmnd_opcode(cmnd)) {
+ case ISCSI_OP_SCSI_CMD:
+ case ISCSI_OP_NOP_OUT:
+ case ISCSI_OP_SCSI_TASK_MGT_MSG:
+ case ISCSI_OP_LOGOUT_CMD:
+ iscsi_push_cmnd(cmnd);
+ goto out;
+ case ISCSI_OP_SCSI_DATA_OUT:
+ data_out_end(cmnd);
+ break;
+ default:
+ PRINT_ERROR("Unexpected cmnd op %x", cmnd_opcode(cmnd));
+ break;
+ }
+
+ req_cmnd_release(cmnd);
+
+out:
+ return;
+}
+
+static int iscsi_alloc_data_buf(struct scst_cmd *cmd)
+{
+ /*
+ * sock->ops->sendpage() is async zero copy operation,
+ * so we must be sure not to free and reuse
+ * the command's buffer before the sending was completed
+ * by the network layers. It is possible only if we
+ * don't use SGV cache.
+ */
+ EXTRACHECKS_BUG_ON(!(scst_cmd_get_data_direction(cmd) & SCST_DATA_READ));
+ scst_cmd_set_no_sgv(cmd);
+ return 1;
+}
+
+static void iscsi_preprocessing_done(struct scst_cmd *scst_cmd)
+{
+ struct iscsi_cmnd *req = (struct iscsi_cmnd *)
+ scst_cmd_get_tgt_priv(scst_cmd);
+
+ TRACE_DBG("req %p", req);
+
+ if (req->conn->rx_task == current)
+ req->scst_state = ISCSI_CMD_STATE_AFTER_PREPROC;
+ else {
+ /*
+ * We wait for the state change without any protection, so
+ * without cmnd_get() it is possible that req will die
+ * "immediately" after the state assignment and
+ * iscsi_make_conn_rd_active() will operate on dead data.
+ * We use the ordered version of cmnd_get(), because "get"
+ * must be done before the state assignment.
+ *
+ * We protected from the race on calling cmnd_rx_continue(),
+ * because there can be only one read thread processing
+ * connection.
+ */
+ cmnd_get_ordered(req);
+ req->scst_state = ISCSI_CMD_STATE_AFTER_PREPROC;
+ iscsi_make_conn_rd_active(req->conn);
+ if (unlikely(req->conn->closing)) {
+ TRACE_DBG("Waking up closing conn %p", req->conn);
+ wake_up(&req->conn->read_state_waitQ);
+ }
+ cmnd_put(req);
+ }
+
+ return;
+}
+
+/*
+ * No locks.
+ *
+ * IMPORTANT! Connection conn must be protected by additional conn_get()
+ * upon entrance in this function, because otherwise it could be destroyed
+ * inside as a result of iscsi_send(), which releases sent commands.
+ */
+static void iscsi_try_local_processing(struct iscsi_conn *conn)
+{
+ int local;
+
+ spin_lock_bh(&iscsi_wr_lock);
+ switch (conn->wr_state) {
+ case ISCSI_CONN_WR_STATE_IN_LIST:
+ list_del(&conn->wr_list_entry);
+ /* go through */
+ case ISCSI_CONN_WR_STATE_IDLE:
+#ifdef CONFIG_SCST_EXTRACHECKS
+ conn->wr_task = current;
+#endif
+ conn->wr_state = ISCSI_CONN_WR_STATE_PROCESSING;
+ conn->wr_space_ready = 0;
+ local = 1;
+ break;
+ default:
+ local = 0;
+ break;
+ }
+ spin_unlock_bh(&iscsi_wr_lock);
+
+ if (local) {
+ int rc = 1;
+
+ if (test_write_ready(conn))
+ rc = iscsi_send(conn);
+
+ spin_lock_bh(&iscsi_wr_lock);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ conn->wr_task = NULL;
+#endif
+ if ((rc <= 0) || test_write_ready(conn)) {
+ list_add_tail(&conn->wr_list_entry, &iscsi_wr_list);
+ conn->wr_state = ISCSI_CONN_WR_STATE_IN_LIST;
+ wake_up(&iscsi_wr_waitQ);
+ } else
+ conn->wr_state = ISCSI_CONN_WR_STATE_IDLE;
+ spin_unlock_bh(&iscsi_wr_lock);
+ }
+ return;
+}
+
+static int iscsi_xmit_response(struct scst_cmd *scst_cmd)
+{
+ int is_send_status = scst_cmd_get_is_send_status(scst_cmd);
+ struct iscsi_cmnd *req = (struct iscsi_cmnd *)
+ scst_cmd_get_tgt_priv(scst_cmd);
+ struct iscsi_conn *conn = req->conn;
+ int status = scst_cmd_get_status(scst_cmd);
+ u8 *sense = scst_cmd_get_sense_buffer(scst_cmd);
+ int sense_len = scst_cmd_get_sense_buffer_len(scst_cmd);
+
+ if (unlikely(scst_cmd_atomic(scst_cmd)))
+ return SCST_TGT_RES_NEED_THREAD_CTX;
+
+ scst_cmd_set_tgt_priv(scst_cmd, NULL);
+
+ EXTRACHECKS_BUG_ON(req->scst_state != ISCSI_CMD_STATE_RESTARTED);
+
+ if (unlikely(scst_cmd_aborted(scst_cmd)))
+ set_bit(ISCSI_CMD_ABORTED, &req->prelim_compl_flags);
+
+ if (unlikely(req->prelim_compl_flags != 0)) {
+ if (test_bit(ISCSI_CMD_ABORTED, &req->prelim_compl_flags)) {
+ TRACE_MGMT_DBG("req %p (scst_cmd %p) aborted", req,
+ req->scst_cmd);
+ scst_set_delivery_status(req->scst_cmd,
+ SCST_CMD_DELIVERY_ABORTED);
+ req->scst_state = ISCSI_CMD_STATE_PROCESSED;
+ req_cmnd_release_force(req);
+ goto out;
+ }
+
+ TRACE_DBG("Prelim completed req %p", req);
+
+ /*
+ * We could preliminary have finished req before we
+ * knew its device, so check if we return correct sense
+ * format.
+ */
+ scst_check_convert_sense(scst_cmd);
+
+ if (!req->own_sg) {
+ req->sg = scst_cmd_get_sg(scst_cmd);
+ req->sg_cnt = scst_cmd_get_sg_cnt(scst_cmd);
+ }
+ } else {
+ EXTRACHECKS_BUG_ON(req->own_sg);
+ req->sg = scst_cmd_get_sg(scst_cmd);
+ req->sg_cnt = scst_cmd_get_sg_cnt(scst_cmd);
+ }
+
+ req->bufflen = scst_cmd_get_resp_data_len(scst_cmd);
+
+ req->scst_state = ISCSI_CMD_STATE_PROCESSED;
+
+ TRACE_DBG("req %p, is_send_status=%x, req->bufflen=%d, req->sg=%p, "
+ "req->sg_cnt %d", req, is_send_status, req->bufflen, req->sg,
+ req->sg_cnt);
+
+ EXTRACHECKS_BUG_ON(req->hashed);
+ if (req->main_rsp != NULL)
+ EXTRACHECKS_BUG_ON(cmnd_opcode(req->main_rsp) != ISCSI_OP_REJECT);
+
+ if (unlikely((req->bufflen != 0) && !is_send_status)) {
+ PRINT_CRIT_ERROR("%s", "Sending DATA without STATUS is "
+ "unsupported");
+ scst_set_cmd_error(scst_cmd,
+ SCST_LOAD_SENSE(scst_sense_hardw_error));
+ BUG(); /* ToDo */
+ }
+
+ if (req->bufflen != 0) {
+ /*
+ * Check above makes sure that is_send_status is set,
+ * so status is valid here, but in future that could change.
+ * ToDo
+ */
+ if ((status != SAM_STAT_CHECK_CONDITION) &&
+ ((cmnd_hdr(req)->flags & (ISCSI_CMD_WRITE|ISCSI_CMD_READ)) !=
+ (ISCSI_CMD_WRITE|ISCSI_CMD_READ))) {
+ send_data_rsp(req, status, is_send_status);
+ } else {
+ struct iscsi_cmnd *rsp;
+ send_data_rsp(req, 0, 0);
+ if (is_send_status) {
+ rsp = create_status_rsp(req, status, sense,
+ sense_len, true);
+ iscsi_cmnd_init_write(rsp, 0);
+ }
+ }
+ } else if (is_send_status) {
+ struct iscsi_cmnd *rsp;
+ rsp = create_status_rsp(req, status, sense, sense_len, false);
+ iscsi_cmnd_init_write(rsp, 0);
+ }
+#ifdef CONFIG_SCST_EXTRACHECKS
+ else
+ BUG();
+#endif
+
+ /*
+ * "_ordered" here to protect from reorder, which can lead to
+ * preliminary connection destroy in req_cmnd_release(). Just in
+ * case, actually, because reordering shouldn't go so far, but who
+ * knows..
+ */
+ conn_get_ordered(conn);
+ req_cmnd_release(req);
+ iscsi_try_local_processing(conn);
+ conn_put(conn);
+
+out:
+ return SCST_TGT_RES_SUCCESS;
+}
+
+/* Called under sn_lock */
+static bool iscsi_is_delay_tm_resp(struct iscsi_cmnd *rsp)
+{
+ bool res = 0;
+ struct iscsi_task_mgt_hdr *req_hdr =
+ (struct iscsi_task_mgt_hdr *)&rsp->parent_req->pdu.bhs;
+ int function = req_hdr->function & ISCSI_FUNCTION_MASK;
+ struct iscsi_session *sess = rsp->conn->session;
+
+ /* This should be checked for immediate TM commands as well */
+
+ switch (function) {
+ default:
+ if (before(sess->exp_cmd_sn, req_hdr->cmd_sn))
+ res = 1;
+ break;
+ }
+ return res;
+}
+
+/* Called under sn_lock, but might drop it inside, then reaquire */
+static void iscsi_check_send_delayed_tm_resp(struct iscsi_session *sess)
+ __acquires(&sn_lock)
+ __releases(&sn_lock)
+{
+ struct iscsi_cmnd *tm_rsp = sess->tm_rsp;
+
+ if (tm_rsp == NULL)
+ goto out;
+
+ if (iscsi_is_delay_tm_resp(tm_rsp))
+ goto out;
+
+ TRACE_MGMT_DBG("Sending delayed rsp %p", tm_rsp);
+
+ sess->tm_rsp = NULL;
+ sess->tm_active--;
+
+ spin_unlock(&sess->sn_lock);
+
+ BUG_ON(sess->tm_active < 0);
+
+ iscsi_cmnd_init_write(tm_rsp, ISCSI_INIT_WRITE_WAKE);
+
+ spin_lock(&sess->sn_lock);
+
+out:
+ return;
+}
+
+static void iscsi_send_task_mgmt_resp(struct iscsi_cmnd *req, int status)
+{
+ struct iscsi_cmnd *rsp;
+ struct iscsi_task_mgt_hdr *req_hdr =
+ (struct iscsi_task_mgt_hdr *)&req->pdu.bhs;
+ struct iscsi_task_rsp_hdr *rsp_hdr;
+ struct iscsi_session *sess = req->conn->session;
+ int fn = req_hdr->function & ISCSI_FUNCTION_MASK;
+
+ TRACE_MGMT_DBG("TM req %p finished", req);
+ TRACE(TRACE_MGMT, "TM fn %d finished, status %d", fn, status);
+
+ rsp = iscsi_alloc_rsp(req);
+ rsp_hdr = (struct iscsi_task_rsp_hdr *)&rsp->pdu.bhs;
+
+ rsp_hdr->opcode = ISCSI_OP_SCSI_TASK_MGT_RSP;
+ rsp_hdr->flags = ISCSI_FLG_FINAL;
+ rsp_hdr->itt = req_hdr->itt;
+ rsp_hdr->response = status;
+
+ if (fn == ISCSI_FUNCTION_TARGET_COLD_RESET) {
+ rsp->should_close_conn = 1;
+ rsp->should_close_all_conn = 1;
+ }
+
+ BUG_ON(sess->tm_rsp != NULL);
+
+ spin_lock(&sess->sn_lock);
+ if (iscsi_is_delay_tm_resp(rsp)) {
+ TRACE_MGMT_DBG("Delaying TM fn %d response %p "
+ "(req %p), because not all affected commands "
+ "received (TM cmd sn %u, exp sn %u)",
+ req_hdr->function & ISCSI_FUNCTION_MASK, rsp, req,
+ req_hdr->cmd_sn, sess->exp_cmd_sn);
+ sess->tm_rsp = rsp;
+ spin_unlock(&sess->sn_lock);
+ goto out_release;
+ }
+ sess->tm_active--;
+ spin_unlock(&sess->sn_lock);
+
+ BUG_ON(sess->tm_active < 0);
+
+ iscsi_cmnd_init_write(rsp, ISCSI_INIT_WRITE_WAKE);
+
+out_release:
+ req_cmnd_release(req);
+ return;
+}
+
+static inline int iscsi_get_mgmt_response(int status)
+{
+ switch (status) {
+ case SCST_MGMT_STATUS_SUCCESS:
+ return ISCSI_RESPONSE_FUNCTION_COMPLETE;
+
+ case SCST_MGMT_STATUS_TASK_NOT_EXIST:
+ return ISCSI_RESPONSE_UNKNOWN_TASK;
+
+ case SCST_MGMT_STATUS_LUN_NOT_EXIST:
+ return ISCSI_RESPONSE_UNKNOWN_LUN;
+
+ case SCST_MGMT_STATUS_FN_NOT_SUPPORTED:
+ return ISCSI_RESPONSE_FUNCTION_UNSUPPORTED;
+
+ case SCST_MGMT_STATUS_REJECTED:
+ case SCST_MGMT_STATUS_FAILED:
+ default:
+ return ISCSI_RESPONSE_FUNCTION_REJECTED;
+ }
+}
+
+static void iscsi_task_mgmt_fn_done(struct scst_mgmt_cmd *scst_mcmd)
+{
+ int fn = scst_mgmt_cmd_get_fn(scst_mcmd);
+ struct iscsi_cmnd *req = (struct iscsi_cmnd *)
+ scst_mgmt_cmd_get_tgt_priv(scst_mcmd);
+ int status =
+ iscsi_get_mgmt_response(scst_mgmt_cmd_get_status(scst_mcmd));
+
+ if ((status == ISCSI_RESPONSE_UNKNOWN_TASK) &&
+ (fn == SCST_ABORT_TASK)) {
+ /* If we are here, we found the task, so must succeed */
+ status = ISCSI_RESPONSE_FUNCTION_COMPLETE;
+ }
+
+ TRACE_MGMT_DBG("req %p, scst_mcmd %p, fn %d, scst status %d, status %d",
+ req, scst_mcmd, fn, scst_mgmt_cmd_get_status(scst_mcmd),
+ status);
+
+ switch (fn) {
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_ABORT_ALL_TASKS_SESS:
+ /* They are internal */
+ break;
+ default:
+ iscsi_send_task_mgmt_resp(req, status);
+ scst_mgmt_cmd_set_tgt_priv(scst_mcmd, NULL);
+ break;
+ }
+ return;
+}
+
+static int iscsi_scsi_aen(struct scst_aen *aen)
+{
+ int res = SCST_AEN_RES_SUCCESS;
+ uint64_t lun = scst_aen_get_lun(aen);
+ const uint8_t *sense = scst_aen_get_sense(aen);
+ int sense_len = scst_aen_get_sense_len(aen);
+ struct iscsi_session *sess = scst_sess_get_tgt_priv(
+ scst_aen_get_sess(aen));
+ struct iscsi_conn *conn;
+ bool found;
+ struct iscsi_cmnd *fake_req, *rsp;
+ struct iscsi_async_msg_hdr *rsp_hdr;
+ struct scatterlist *sg;
+
+ TRACE_MGMT_DBG("SCSI AEN to sess %p (initiator %s)", sess,
+ sess->initiator_name);
+
+ mutex_lock(&sess->target->target_mutex);
+
+ found = false;
+ list_for_each_entry_reverse(conn, &sess->conn_list, conn_list_entry) {
+ if (!test_bit(ISCSI_CONN_SHUTTINGDOWN, &conn->conn_aflags) &&
+ (conn->conn_reinst_successor == NULL)) {
+ found = true;
+ break;
+ }
+ }
+ if (!found) {
+ TRACE_MGMT_DBG("Unable to find alive conn for sess %p", sess);
+ goto out_err;
+ }
+
+ /* Create a fake request */
+ fake_req = cmnd_alloc(conn, NULL);
+ if (fake_req == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc fake AEN request");
+ goto out_err;
+ }
+
+ mutex_unlock(&sess->target->target_mutex);
+
+ rsp = iscsi_alloc_main_rsp(fake_req);
+ if (rsp == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc AEN rsp");
+ goto out_err_free_req;
+ }
+
+ fake_req->scst_state = ISCSI_CMD_STATE_AEN;
+ fake_req->scst_aen = aen;
+
+ rsp_hdr = (struct iscsi_async_msg_hdr *)&rsp->pdu.bhs;
+
+ rsp_hdr->opcode = ISCSI_OP_ASYNC_MSG;
+ rsp_hdr->flags = ISCSI_FLG_FINAL;
+ rsp_hdr->lun = lun; /* it's already in SCSI form */
+ rsp_hdr->ffffffff = 0xffffffff;
+ rsp_hdr->async_event = ISCSI_ASYNC_SCSI;
+
+ sg = rsp->sg = rsp->rsp_sg;
+ rsp->sg_cnt = 2;
+ rsp->own_sg = 1;
+
+ sg_init_table(sg, 2);
+ sg_set_buf(&sg[0], &rsp->sense_hdr, sizeof(rsp->sense_hdr));
+ sg_set_buf(&sg[1], sense, sense_len);
+
+ rsp->sense_hdr.length = cpu_to_be16(sense_len);
+ rsp->pdu.datasize = sizeof(rsp->sense_hdr) + sense_len;
+ rsp->bufflen = rsp->pdu.datasize;
+
+ req_cmnd_release(fake_req);
+
+out:
+ return res;
+
+out_err_free_req:
+ req_cmnd_release(fake_req);
+
+out_err:
+ mutex_unlock(&sess->target->target_mutex);
+ res = SCST_AEN_RES_FAILED;
+ goto out;
+}
+
+static int iscsi_report_aen(struct scst_aen *aen)
+{
+ int res;
+ int event_fn = scst_aen_get_event_fn(aen);
+
+ switch (event_fn) {
+ case SCST_AEN_SCSI:
+ res = iscsi_scsi_aen(aen);
+ break;
+ default:
+ TRACE_MGMT_DBG("Unsupported AEN %d", event_fn);
+ res = SCST_AEN_RES_NOT_SUPPORTED;
+ break;
+ }
+ return res;
+}
+
+void iscsi_send_nop_in(struct iscsi_conn *conn)
+{
+ struct iscsi_cmnd *req, *rsp;
+ struct iscsi_nop_in_hdr *rsp_hdr;
+
+ req = cmnd_alloc(conn, NULL);
+ if (req == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc fake Nop-In request");
+ goto out_err;
+ }
+
+ rsp = iscsi_alloc_main_rsp(req);
+ if (rsp == NULL) {
+ PRINT_ERROR("%s", "Unable to alloc Nop-In rsp");
+ goto out_err_free_req;
+ }
+
+ cmnd_get(rsp);
+
+ rsp_hdr = (struct iscsi_nop_in_hdr *)&rsp->pdu.bhs;
+ rsp_hdr->opcode = ISCSI_OP_NOP_IN;
+ rsp_hdr->flags = ISCSI_FLG_FINAL;
+ rsp_hdr->itt = cpu_to_be32(ISCSI_RESERVED_TAG);
+ rsp_hdr->ttt = conn->nop_in_ttt++;
+
+ if (conn->nop_in_ttt == cpu_to_be32(ISCSI_RESERVED_TAG))
+ conn->nop_in_ttt = 0;
+
+ /* Supposed that all other fields are zeroed */
+
+ TRACE_DBG("Sending Nop-In request (ttt 0x%08x)", rsp_hdr->ttt);
+ spin_lock_bh(&conn->nop_req_list_lock);
+ list_add_tail(&rsp->nop_req_list_entry, &conn->nop_req_list);
+ spin_unlock_bh(&conn->nop_req_list_lock);
+
+out_err_free_req:
+ req_cmnd_release(req);
+
+out_err:
+ return;
+}
+
+static int iscsi_target_detect(struct scst_tgt_template *templ)
+{
+ /* Nothing to do */
+ return 0;
+}
+
+static int iscsi_target_release(struct scst_tgt *scst_tgt)
+{
+ /* Nothing to do */
+ return 0;
+}
+
+static struct scst_trace_log iscsi_local_trace_tbl[] = {
+ { TRACE_D_WRITE, "d_write" },
+ { TRACE_CONN_OC, "conn" },
+ { TRACE_CONN_OC_DBG, "conn_dbg" },
+ { TRACE_D_IOV, "iov" },
+ { TRACE_D_DUMP_PDU, "pdu" },
+ { TRACE_NET_PG, "net_page" },
+ { 0, NULL }
+};
+
+#define ISCSI_TRACE_TLB_HELP ", d_read, d_write, conn, conn_dbg, iov, pdu, net_page"
+
+#define ISCSI_MGMT_CMD_HELP \
+ " echo \"add_attribute IncomingUser name password\" >mgmt\n" \
+ " echo \"del_attribute IncomingUser name\" >mgmt\n" \
+ " echo \"add_attribute OutgoingUser name password\" >mgmt\n" \
+ " echo \"del_attribute OutgoingUser name\" >mgmt\n" \
+ " echo \"add_target_attribute target_name IncomingUser name password\" >mgmt\n" \
+ " echo \"del_target_attribute target_name IncomingUser name\" >mgmt\n" \
+ " echo \"add_target_attribute target_name OutgoingUser name password\" >mgmt\n" \
+ " echo \"del_target_attribute target_name OutgoingUser name\" >mgmt\n"
+
+struct scst_tgt_template iscsi_template = {
+ .name = "iscsi",
+ .sg_tablesize = 0xFFFF /* no limit */,
+ .threads_num = 0,
+ .no_clustering = 1,
+ .xmit_response_atomic = 0,
+ .tgtt_attrs = iscsi_attrs,
+ .tgt_attrs = iscsi_tgt_attrs,
+ .sess_attrs = iscsi_sess_attrs,
+ .enable_target = iscsi_enable_target,
+ .is_target_enabled = iscsi_is_target_enabled,
+ .add_target = iscsi_sysfs_add_target,
+ .del_target = iscsi_sysfs_del_target,
+ .mgmt_cmd = iscsi_sysfs_mgmt_cmd,
+ .mgmt_cmd_help = ISCSI_MGMT_CMD_HELP,
+#if defined(CONFIG_SCST_DEBUG) || defined(CONFIG_SCST_TRACING)
+ .default_trace_flags = ISCSI_DEFAULT_LOG_FLAGS,
+ .trace_flags = &trace_flag,
+ .trace_tbl = iscsi_local_trace_tbl,
+ .trace_tbl_help = ISCSI_TRACE_TLB_HELP,
+#endif
+ .detect = iscsi_target_detect,
+ .release = iscsi_target_release,
+ .xmit_response = iscsi_xmit_response,
+ .alloc_data_buf = iscsi_alloc_data_buf,
+ .preprocessing_done = iscsi_preprocessing_done,
+ .pre_exec = iscsi_pre_exec,
+ .task_mgmt_affected_cmds_done = iscsi_task_mgmt_affected_cmds_done,
+ .task_mgmt_fn_done = iscsi_task_mgmt_fn_done,
+ .report_aen = iscsi_report_aen,
+};
+
+static __init int iscsi_run_threads(int count, char *name, int (*fn)(void *))
+{
+ int res = 0;
+ int i;
+ struct iscsi_thread_t *thr;
+
+ for (i = 0; i < count; i++) {
+ thr = kmalloc(sizeof(*thr), GFP_KERNEL);
+ if (!thr) {
+ res = -ENOMEM;
+ PRINT_ERROR("Failed to allocate thr %d", res);
+ goto out;
+ }
+ thr->thr = kthread_run(fn, NULL, "%s%d", name, i);
+ if (IS_ERR(thr->thr)) {
+ res = PTR_ERR(thr->thr);
+ PRINT_ERROR("kthread_create() failed: %d", res);
+ kfree(thr);
+ goto out;
+ }
+ list_add_tail(&thr->threads_list_entry, &iscsi_threads_list);
+ }
+
+out:
+ return res;
+}
+
+static void iscsi_stop_threads(void)
+{
+ struct iscsi_thread_t *t, *tmp;
+
+ list_for_each_entry_safe(t, tmp, &iscsi_threads_list,
+ threads_list_entry) {
+ int rc = kthread_stop(t->thr);
+ if (rc < 0)
+ TRACE_MGMT_DBG("kthread_stop() failed: %d", rc);
+ list_del(&t->threads_list_entry);
+ kfree(t);
+ }
+ return;
+}
+
+static int __init iscsi_init(void)
+{
+ int err = 0;
+ int num;
+
+ PRINT_INFO("iSCSI SCST Target - version %s", ISCSI_VERSION_STRING);
+
+ dummy_page = alloc_pages(GFP_KERNEL, 0);
+ if (dummy_page == NULL) {
+ PRINT_ERROR("%s", "Dummy page allocation failed");
+ goto out;
+ }
+
+ sg_init_table(&dummy_sg, 1);
+ sg_set_page(&dummy_sg, dummy_page, PAGE_SIZE, 0);
+
+ ctr_major = register_chrdev(0, ctr_name, &ctr_fops);
+ if (ctr_major < 0) {
+ PRINT_ERROR("failed to register the control device %d",
+ ctr_major);
+ err = ctr_major;
+ goto out_callb;
+ }
+
+ err = event_init();
+ if (err < 0)
+ goto out_reg;
+
+ iscsi_cmnd_cache = KMEM_CACHE(iscsi_cmnd, SCST_SLAB_FLAGS);
+ if (!iscsi_cmnd_cache) {
+ err = -ENOMEM;
+ goto out_event;
+ }
+
+ err = scst_register_target_template(&iscsi_template);
+ if (err < 0)
+ goto out_kmem;
+
+ num = max((int)num_online_cpus(), 2);
+
+ err = iscsi_run_threads(num, "iscsird", istrd);
+ if (err != 0)
+ goto out_thr;
+
+ err = iscsi_run_threads(num, "iscsiwr", istwr);
+ if (err != 0)
+ goto out_thr;
+
+out:
+ return err;
+
+out_thr:
+ iscsi_stop_threads();
+
+ scst_unregister_target_template(&iscsi_template);
+
+out_kmem:
+ kmem_cache_destroy(iscsi_cmnd_cache);
+
+out_event:
+ event_exit();
+
+out_reg:
+ unregister_chrdev(ctr_major, ctr_name);
+
+out_callb:
+ __free_pages(dummy_page, 0);
+ goto out;
+}
+
+static void __exit iscsi_exit(void)
+{
+ iscsi_stop_threads();
+
+ unregister_chrdev(ctr_major, ctr_name);
+
+ event_exit();
+
+ kmem_cache_destroy(iscsi_cmnd_cache);
+
+ scst_unregister_target_template(&iscsi_template);
+
+ __free_pages(dummy_page, 0);
+ return;
+}
+
+module_init(iscsi_init);
+module_exit(iscsi_exit);
+
+MODULE_LICENSE("GPL");
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/nthread.c linux-2.6.33/drivers/scst/iscsi-scst/nthread.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/nthread.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/nthread.c
@@ -0,0 +1,1524 @@
+/*
+ * Network threads.
+ *
+ * Copyright (C) 2004 - 2005 FUJITA Tomonori <tomof@acm.org>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/sched.h>
+#include <linux/file.h>
+#include <linux/kthread.h>
+#include <asm/ioctls.h>
+#include <linux/delay.h>
+#include <net/tcp.h>
+
+#include "iscsi.h"
+#include "digest.h"
+
+enum rx_state {
+ RX_INIT_BHS, /* Must be zero for better "switch" optimization. */
+ RX_BHS,
+ RX_CMD_START,
+ RX_DATA,
+ RX_END,
+
+ RX_CMD_CONTINUE,
+ RX_INIT_HDIGEST,
+ RX_CHECK_HDIGEST,
+ RX_INIT_DDIGEST,
+ RX_CHECK_DDIGEST,
+ RX_AHS,
+ RX_PADDING,
+};
+
+enum tx_state {
+ TX_INIT = 0, /* Must be zero for better "switch" optimization. */
+ TX_BHS_DATA,
+ TX_INIT_PADDING,
+ TX_PADDING,
+ TX_INIT_DDIGEST,
+ TX_DDIGEST,
+ TX_END,
+};
+
+static inline void iscsi_check_closewait(struct iscsi_conn *conn) {};
+
+static void free_pending_commands(struct iscsi_conn *conn)
+{
+ struct iscsi_session *session = conn->session;
+ struct list_head *pending_list = &session->pending_list;
+ int req_freed;
+ struct iscsi_cmnd *cmnd;
+
+ spin_lock(&session->sn_lock);
+ do {
+ req_freed = 0;
+ list_for_each_entry(cmnd, pending_list, pending_list_entry) {
+ TRACE_CONN_CLOSE_DBG("Pending cmd %p"
+ "(conn %p, cmd_sn %u, exp_cmd_sn %u)",
+ cmnd, conn, cmnd->pdu.bhs.sn,
+ session->exp_cmd_sn);
+ if ((cmnd->conn == conn) &&
+ (session->exp_cmd_sn == cmnd->pdu.bhs.sn)) {
+ TRACE_CONN_CLOSE_DBG("Freeing pending cmd %p",
+ cmnd);
+
+ list_del(&cmnd->pending_list_entry);
+ cmnd->pending = 0;
+
+ session->exp_cmd_sn++;
+
+ spin_unlock(&session->sn_lock);
+
+ req_cmnd_release_force(cmnd);
+
+ req_freed = 1;
+ spin_lock(&session->sn_lock);
+ break;
+ }
+ }
+ } while (req_freed);
+ spin_unlock(&session->sn_lock);
+
+ return;
+}
+
+static void free_orphaned_pending_commands(struct iscsi_conn *conn)
+{
+ struct iscsi_session *session = conn->session;
+ struct list_head *pending_list = &session->pending_list;
+ int req_freed;
+ struct iscsi_cmnd *cmnd;
+
+ spin_lock(&session->sn_lock);
+ do {
+ req_freed = 0;
+ list_for_each_entry(cmnd, pending_list, pending_list_entry) {
+ TRACE_CONN_CLOSE_DBG("Pending cmd %p"
+ "(conn %p, cmd_sn %u, exp_cmd_sn %u)",
+ cmnd, conn, cmnd->pdu.bhs.sn,
+ session->exp_cmd_sn);
+ if (cmnd->conn == conn) {
+ PRINT_ERROR("Freeing orphaned pending cmd %p",
+ cmnd);
+
+ list_del(&cmnd->pending_list_entry);
+ cmnd->pending = 0;
+
+ if (session->exp_cmd_sn == cmnd->pdu.bhs.sn)
+ session->exp_cmd_sn++;
+
+ spin_unlock(&session->sn_lock);
+
+ req_cmnd_release_force(cmnd);
+
+ req_freed = 1;
+ spin_lock(&session->sn_lock);
+ break;
+ }
+ }
+ } while (req_freed);
+ spin_unlock(&session->sn_lock);
+
+ return;
+}
+
+#ifdef CONFIG_SCST_DEBUG
+static void trace_conn_close(struct iscsi_conn *conn)
+{
+ struct iscsi_cmnd *cmnd;
+
+#if 0
+ if (time_after(jiffies, start_waiting + 10*HZ))
+ trace_flag |= TRACE_CONN_OC_DBG;
+#endif
+
+ spin_lock_bh(&conn->cmd_list_lock);
+ list_for_each_entry(cmnd, &conn->cmd_list,
+ cmd_list_entry) {
+ TRACE_CONN_CLOSE_DBG(
+ "cmd %p, scst_cmd %p, scst_state %x, scst_cmd state "
+ "%d, r2t_len_to_receive %d, ref_cnt %d, sn %u, "
+ "parent_req %p, pending %d",
+ cmnd, cmnd->scst_cmd, cmnd->scst_state,
+ ((cmnd->parent_req == NULL) && cmnd->scst_cmd) ?
+ cmnd->scst_cmd->state : -1,
+ cmnd->r2t_len_to_receive, atomic_read(&cmnd->ref_cnt),
+ cmnd->pdu.bhs.sn, cmnd->parent_req, cmnd->pending);
+ }
+ spin_unlock_bh(&conn->cmd_list_lock);
+ return;
+}
+#else /* CONFIG_SCST_DEBUG */
+static void trace_conn_close(struct iscsi_conn *conn) {}
+#endif /* CONFIG_SCST_DEBUG */
+
+void iscsi_task_mgmt_affected_cmds_done(struct scst_mgmt_cmd *scst_mcmd)
+{
+ int fn = scst_mgmt_cmd_get_fn(scst_mcmd);
+ void *priv = scst_mgmt_cmd_get_tgt_priv(scst_mcmd);
+
+ TRACE_MGMT_DBG("scst_mcmd %p, fn %d, priv %p", scst_mcmd, fn, priv);
+
+ switch (fn) {
+ case SCST_NEXUS_LOSS_SESS:
+ case SCST_ABORT_ALL_TASKS_SESS:
+ {
+ struct iscsi_conn *conn = (struct iscsi_conn *)priv;
+ struct iscsi_session *sess = conn->session;
+ struct iscsi_conn *c;
+
+ mutex_lock(&sess->target->target_mutex);
+
+ /*
+ * We can't mark sess as shutting down earlier, because until
+ * now it might have pending commands. Otherwise, in case of
+ * reinstatement it might lead to data corruption, because
+ * commands in being reinstated session can be executed
+ * after commands in the new session.
+ */
+ sess->sess_shutting_down = 1;
+ list_for_each_entry(c, &sess->conn_list, conn_list_entry) {
+ if (!test_bit(ISCSI_CONN_SHUTTINGDOWN, &c->conn_aflags)) {
+ sess->sess_shutting_down = 0;
+ break;
+ }
+ }
+
+ if (conn->conn_reinst_successor != NULL) {
+ BUG_ON(!test_bit(ISCSI_CONN_REINSTATING,
+ &conn->conn_reinst_successor->conn_aflags));
+ conn_reinst_finished(conn->conn_reinst_successor);
+ conn->conn_reinst_successor = NULL;
+ } else if (sess->sess_reinst_successor != NULL) {
+ sess_reinst_finished(sess->sess_reinst_successor);
+ sess->sess_reinst_successor = NULL;
+ }
+ mutex_unlock(&sess->target->target_mutex);
+
+ complete_all(&conn->ready_to_free);
+ break;
+ }
+ default:
+ /* Nothing to do */
+ break;
+ }
+
+ return;
+}
+
+/* No locks */
+static void close_conn(struct iscsi_conn *conn)
+{
+ struct iscsi_session *session = conn->session;
+ struct iscsi_target *target = conn->target;
+ typeof(jiffies) start_waiting = jiffies;
+ typeof(jiffies) shut_start_waiting = start_waiting;
+ bool pending_reported = 0, wait_expired = 0, shut_expired = 0;
+ bool reinst;
+
+#define CONN_PENDING_TIMEOUT ((typeof(jiffies))10*HZ)
+#define CONN_WAIT_TIMEOUT ((typeof(jiffies))10*HZ)
+#define CONN_REG_SHUT_TIMEOUT ((typeof(jiffies))125*HZ)
+#define CONN_DEL_SHUT_TIMEOUT ((typeof(jiffies))10*HZ)
+
+ TRACE_MGMT_DBG("Closing connection %p (conn_ref_cnt=%d)", conn,
+ atomic_read(&conn->conn_ref_cnt));
+
+ iscsi_extracheck_is_rd_thread(conn);
+
+ BUG_ON(!conn->closing);
+
+ if (conn->active_close) {
+ /* We want all our already send operations to complete */
+ conn->sock->ops->shutdown(conn->sock, RCV_SHUTDOWN);
+ } else {
+ conn->sock->ops->shutdown(conn->sock,
+ RCV_SHUTDOWN|SEND_SHUTDOWN);
+ }
+
+ mutex_lock(&session->target->target_mutex);
+
+ set_bit(ISCSI_CONN_SHUTTINGDOWN, &conn->conn_aflags);
+ reinst = (conn->conn_reinst_successor != NULL);
+
+ mutex_unlock(&session->target->target_mutex);
+
+ if (reinst) {
+ int rc;
+ int lun = 0;
+
+ /* Abort all outstanding commands */
+ rc = scst_rx_mgmt_fn_lun(session->scst_sess,
+ SCST_ABORT_ALL_TASKS_SESS, (uint8_t *)&lun, sizeof(lun),
+ SCST_NON_ATOMIC, conn);
+ if (rc != 0)
+ PRINT_ERROR("SCST_ABORT_ALL_TASKS_SESS failed %d", rc);
+ } else {
+ int rc;
+ int lun = 0;
+
+ rc = scst_rx_mgmt_fn_lun(session->scst_sess,
+ SCST_NEXUS_LOSS_SESS, (uint8_t *)&lun, sizeof(lun),
+ SCST_NON_ATOMIC, conn);
+ if (rc != 0)
+ PRINT_ERROR("SCST_NEXUS_LOSS_SESS failed %d", rc);
+ }
+
+ if (conn->read_state != RX_INIT_BHS) {
+ struct iscsi_cmnd *cmnd = conn->read_cmnd;
+
+ if (cmnd->scst_state == ISCSI_CMD_STATE_RX_CMD) {
+ TRACE_CONN_CLOSE_DBG("Going to wait for cmnd %p to "
+ "change state from RX_CMD", cmnd);
+ }
+ wait_event(conn->read_state_waitQ,
+ cmnd->scst_state != ISCSI_CMD_STATE_RX_CMD);
+
+ TRACE_CONN_CLOSE_DBG("Releasing conn->read_cmnd %p (conn %p)",
+ conn->read_cmnd, conn);
+
+ conn->read_cmnd = NULL;
+ conn->read_state = RX_INIT_BHS;
+ req_cmnd_release_force(cmnd);
+ }
+
+ conn_abort(conn);
+
+ /* ToDo: not the best way to wait */
+ while (atomic_read(&conn->conn_ref_cnt) != 0) {
+ if (conn->conn_tm_active)
+ iscsi_check_tm_data_wait_timeouts(conn, true);
+
+ mutex_lock(&target->target_mutex);
+ spin_lock(&session->sn_lock);
+ if (session->tm_rsp && session->tm_rsp->conn == conn) {
+ struct iscsi_cmnd *tm_rsp = session->tm_rsp;
+ TRACE_MGMT_DBG("Dropping delayed TM rsp %p", tm_rsp);
+ session->tm_rsp = NULL;
+ session->tm_active--;
+ WARN_ON(session->tm_active < 0);
+ spin_unlock(&session->sn_lock);
+ mutex_unlock(&target->target_mutex);
+
+ rsp_cmnd_release(tm_rsp);
+ } else {
+ spin_unlock(&session->sn_lock);
+ mutex_unlock(&target->target_mutex);
+ }
+
+ /* It's safe to check it without sn_lock */
+ if (!list_empty(&session->pending_list)) {
+ TRACE_CONN_CLOSE_DBG("Disposing pending commands on "
+ "connection %p (conn_ref_cnt=%d)", conn,
+ atomic_read(&conn->conn_ref_cnt));
+
+ free_pending_commands(conn);
+
+ if (time_after(jiffies,
+ start_waiting + CONN_PENDING_TIMEOUT)) {
+ if (!pending_reported) {
+ TRACE_CONN_CLOSE("%s",
+ "Pending wait time expired");
+ pending_reported = 1;
+ }
+ free_orphaned_pending_commands(conn);
+ }
+ }
+
+ iscsi_make_conn_wr_active(conn);
+
+ /* That's for active close only, actually */
+ if (time_after(jiffies, start_waiting + CONN_WAIT_TIMEOUT) &&
+ !wait_expired) {
+ TRACE_CONN_CLOSE("Wait time expired (conn %p, "
+ "sk_state %d)",
+ conn, conn->sock->sk->sk_state);
+ conn->sock->ops->shutdown(conn->sock, SEND_SHUTDOWN);
+ wait_expired = 1;
+ shut_start_waiting = jiffies;
+ }
+
+ if (wait_expired && !shut_expired &&
+ time_after(jiffies, shut_start_waiting +
+ conn->deleting ? CONN_DEL_SHUT_TIMEOUT :
+ CONN_REG_SHUT_TIMEOUT)) {
+ TRACE_CONN_CLOSE("Wait time after shutdown expired "
+ "(conn %p, sk_state %d)", conn,
+ conn->sock->sk->sk_state);
+ conn->sock->sk->sk_prot->disconnect(conn->sock->sk, 0);
+ shut_expired = 1;
+ }
+
+ if (conn->deleting)
+ msleep(200);
+ else
+ msleep(1000);
+
+ TRACE_CONN_CLOSE_DBG("conn %p, conn_ref_cnt %d left, "
+ "wr_state %d, exp_cmd_sn %u",
+ conn, atomic_read(&conn->conn_ref_cnt),
+ conn->wr_state, session->exp_cmd_sn);
+
+ trace_conn_close(conn);
+
+ iscsi_check_closewait(conn);
+ }
+
+ write_lock_bh(&conn->sock->sk->sk_callback_lock);
+ conn->sock->sk->sk_state_change = conn->old_state_change;
+ conn->sock->sk->sk_data_ready = conn->old_data_ready;
+ conn->sock->sk->sk_write_space = conn->old_write_space;
+ write_unlock_bh(&conn->sock->sk->sk_callback_lock);
+
+ while (1) {
+ bool t;
+
+ spin_lock_bh(&iscsi_wr_lock);
+ t = (conn->wr_state == ISCSI_CONN_WR_STATE_IDLE);
+ spin_unlock_bh(&iscsi_wr_lock);
+
+ if (t && (atomic_read(&conn->conn_ref_cnt) == 0))
+ break;
+
+ TRACE_CONN_CLOSE_DBG("Waiting for wr thread (conn %p), "
+ "wr_state %x", conn, conn->wr_state);
+ msleep(50);
+ }
+
+ wait_for_completion(&conn->ready_to_free);
+
+ TRACE_CONN_CLOSE("Notifying user space about closing connection %p",
+ conn);
+ event_send(target->tid, session->sid, conn->cid, 0, E_CONN_CLOSE,
+ NULL, NULL);
+
+ kobject_put(&conn->iscsi_conn_kobj);
+ return;
+}
+
+static int close_conn_thr(void *arg)
+{
+ struct iscsi_conn *conn = (struct iscsi_conn *)arg;
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ /*
+ * To satisfy iscsi_extracheck_is_rd_thread() in functions called
+ * on the connection close. It is safe, because at this point conn
+ * can't be used by any other thread.
+ */
+ conn->rd_task = current;
+#endif
+ close_conn(conn);
+ return 0;
+}
+
+/* No locks */
+static void start_close_conn(struct iscsi_conn *conn)
+{
+ struct task_struct *t;
+
+ t = kthread_run(close_conn_thr, conn, "iscsi_conn_cleanup");
+ if (IS_ERR(t)) {
+ PRINT_ERROR("kthread_run() failed (%ld), closing conn %p "
+ "directly", PTR_ERR(t), conn);
+ close_conn(conn);
+ }
+ return;
+}
+
+static inline void iscsi_conn_init_read(struct iscsi_conn *conn,
+ void __user *data, size_t len)
+{
+ conn->read_iov[0].iov_base = data;
+ conn->read_iov[0].iov_len = len;
+ conn->read_msg.msg_iov = conn->read_iov;
+ conn->read_msg.msg_iovlen = 1;
+ conn->read_size = len;
+ return;
+}
+
+static void iscsi_conn_prepare_read_ahs(struct iscsi_conn *conn,
+ struct iscsi_cmnd *cmnd)
+{
+ int asize = (cmnd->pdu.ahssize + 3) & -4;
+
+ /* ToDo: __GFP_NOFAIL ?? */
+ cmnd->pdu.ahs = kmalloc(asize, __GFP_NOFAIL|GFP_KERNEL);
+ BUG_ON(cmnd->pdu.ahs == NULL);
+ iscsi_conn_init_read(conn, (void __force __user *)cmnd->pdu.ahs, asize);
+ return;
+}
+
+static struct iscsi_cmnd *iscsi_get_send_cmnd(struct iscsi_conn *conn)
+{
+ struct iscsi_cmnd *cmnd = NULL;
+
+ spin_lock_bh(&conn->write_list_lock);
+ if (!list_empty(&conn->write_list)) {
+ cmnd = list_entry(conn->write_list.next, struct iscsi_cmnd,
+ write_list_entry);
+ cmd_del_from_write_list(cmnd);
+ cmnd->write_processing_started = 1;
+ }
+ spin_unlock_bh(&conn->write_list_lock);
+
+ if (unlikely(test_bit(ISCSI_CMD_ABORTED,
+ &cmnd->parent_req->prelim_compl_flags))) {
+ TRACE_MGMT_DBG("Going to send acmd %p (scst cmd %p, "
+ "state %d, parent_req %p)", cmnd, cmnd->scst_cmd,
+ cmnd->scst_state, cmnd->parent_req);
+ }
+
+ if (unlikely(cmnd_opcode(cmnd) == ISCSI_OP_SCSI_TASK_MGT_RSP)) {
+#ifdef CONFIG_SCST_DEBUG
+ struct iscsi_task_mgt_hdr *req_hdr =
+ (struct iscsi_task_mgt_hdr *)&cmnd->parent_req->pdu.bhs;
+ struct iscsi_task_rsp_hdr *rsp_hdr =
+ (struct iscsi_task_rsp_hdr *)&cmnd->pdu.bhs;
+ TRACE_MGMT_DBG("Going to send TM response %p (status %d, "
+ "fn %d, parent_req %p)", cmnd, rsp_hdr->response,
+ req_hdr->function & ISCSI_FUNCTION_MASK,
+ cmnd->parent_req);
+#endif
+ }
+
+ return cmnd;
+}
+
+/* Returns number of bytes left to receive or <0 for error */
+static int do_recv(struct iscsi_conn *conn)
+{
+ int res;
+ mm_segment_t oldfs;
+ struct msghdr msg;
+ int first_len;
+
+ EXTRACHECKS_BUG_ON(conn->read_cmnd == NULL);
+
+ if (unlikely(conn->closing)) {
+ res = -EIO;
+ goto out;
+ }
+
+ /*
+ * We suppose that if sock_recvmsg() returned less data than requested,
+ * then next time it will return -EAGAIN, so there's no point to call
+ * it again.
+ */
+
+restart:
+ memset(&msg, 0, sizeof(msg));
+ msg.msg_iov = conn->read_msg.msg_iov;
+ msg.msg_iovlen = conn->read_msg.msg_iovlen;
+ first_len = msg.msg_iov->iov_len;
+
+ oldfs = get_fs();
+ set_fs(get_ds());
+ res = sock_recvmsg(conn->sock, &msg, conn->read_size,
+ MSG_DONTWAIT | MSG_NOSIGNAL);
+ set_fs(oldfs);
+
+ if (res > 0) {
+ /*
+ * To save some considerable effort and CPU power we
+ * suppose that TCP functions adjust
+ * conn->read_msg.msg_iov and conn->read_msg.msg_iovlen
+ * on amount of copied data. This BUG_ON is intended
+ * to catch if it is changed in the future.
+ */
+ BUG_ON((res >= first_len) &&
+ (conn->read_msg.msg_iov->iov_len != 0));
+ conn->read_size -= res;
+ if (conn->read_size != 0) {
+ if (res >= first_len) {
+ int done = 1 + ((res - first_len) >> PAGE_SHIFT);
+ conn->read_msg.msg_iov += done;
+ conn->read_msg.msg_iovlen -= done;
+ }
+ }
+ res = conn->read_size;
+ } else {
+ switch (res) {
+ case -EAGAIN:
+ TRACE_DBG("EAGAIN received for conn %p", conn);
+ res = conn->read_size;
+ break;
+ case -ERESTARTSYS:
+ TRACE_DBG("ERESTARTSYS received for conn %p", conn);
+ goto restart;
+ default:
+ if (!conn->closing) {
+ PRINT_ERROR("sock_recvmsg() failed: %d", res);
+ mark_conn_closed(conn);
+ }
+ if (res == 0)
+ res = -EIO;
+ break;
+ }
+ }
+
+out:
+ return res;
+}
+
+static int iscsi_rx_check_ddigest(struct iscsi_conn *conn)
+{
+ struct iscsi_cmnd *cmnd = conn->read_cmnd;
+ int res;
+
+ res = do_recv(conn);
+ if (res == 0) {
+ conn->read_state = RX_END;
+
+ if (cmnd->pdu.datasize <= 16*1024) {
+ /*
+ * It's cache hot, so let's compute it inline. The
+ * choice here about what will expose more latency:
+ * possible cache misses or the digest calculation.
+ */
+ TRACE_DBG("cmnd %p, opcode %x: checking RX "
+ "ddigest inline", cmnd, cmnd_opcode(cmnd));
+ cmnd->ddigest_checked = 1;
+ res = digest_rx_data(cmnd);
+ if (unlikely(res != 0)) {
+ mark_conn_closed(conn);
+ goto out;
+ }
+ } else if (cmnd_opcode(cmnd) == ISCSI_OP_SCSI_CMD) {
+ cmd_add_on_rx_ddigest_list(cmnd, cmnd);
+ cmnd_get(cmnd);
+ } else if (cmnd_opcode(cmnd) != ISCSI_OP_SCSI_DATA_OUT) {
+ /*
+ * We could get here only for Nop-Out. ISCSI RFC
+ * doesn't specify how to deal with digest errors in
+ * this case. Is closing connection correct?
+ */
+ TRACE_DBG("cmnd %p, opcode %x: checking NOP RX "
+ "ddigest", cmnd, cmnd_opcode(cmnd));
+ res = digest_rx_data(cmnd);
+ if (unlikely(res != 0)) {
+ mark_conn_closed(conn);
+ goto out;
+ }
+ }
+ }
+
+out:
+ return res;
+}
+
+/* No locks, conn is rd processing */
+static int process_read_io(struct iscsi_conn *conn, int *closed)
+{
+ struct iscsi_cmnd *cmnd = conn->read_cmnd;
+ int res;
+
+ /* In case of error cmnd will be freed in close_conn() */
+
+ do {
+ switch (conn->read_state) {
+ case RX_INIT_BHS:
+ EXTRACHECKS_BUG_ON(conn->read_cmnd != NULL);
+ cmnd = cmnd_alloc(conn, NULL);
+ conn->read_cmnd = cmnd;
+ iscsi_conn_init_read(cmnd->conn,
+ (void __force __user *)&cmnd->pdu.bhs,
+ sizeof(cmnd->pdu.bhs));
+ conn->read_state = RX_BHS;
+ /* go through */
+
+ case RX_BHS:
+ res = do_recv(conn);
+ if (res == 0) {
+ iscsi_cmnd_get_length(&cmnd->pdu);
+ if (cmnd->pdu.ahssize == 0) {
+ if ((conn->hdigest_type & DIGEST_NONE) == 0)
+ conn->read_state = RX_INIT_HDIGEST;
+ else
+ conn->read_state = RX_CMD_START;
+ } else {
+ iscsi_conn_prepare_read_ahs(conn, cmnd);
+ conn->read_state = RX_AHS;
+ }
+ }
+ break;
+
+ case RX_CMD_START:
+ res = cmnd_rx_start(cmnd);
+ if (res == 0) {
+ if (cmnd->pdu.datasize == 0)
+ conn->read_state = RX_END;
+ else
+ conn->read_state = RX_DATA;
+ } else if (res > 0)
+ conn->read_state = RX_CMD_CONTINUE;
+ else
+ BUG_ON(!conn->closing);
+ break;
+
+ case RX_CMD_CONTINUE:
+ if (cmnd->scst_state == ISCSI_CMD_STATE_RX_CMD) {
+ TRACE_DBG("cmnd %p is still in RX_CMD state",
+ cmnd);
+ res = 1;
+ break;
+ }
+ res = cmnd_rx_continue(cmnd);
+ if (unlikely(res != 0))
+ BUG_ON(!conn->closing);
+ else {
+ if (cmnd->pdu.datasize == 0)
+ conn->read_state = RX_END;
+ else
+ conn->read_state = RX_DATA;
+ }
+ break;
+
+ case RX_DATA:
+ res = do_recv(conn);
+ if (res == 0) {
+ int psz = ((cmnd->pdu.datasize + 3) & -4) - cmnd->pdu.datasize;
+ if (psz != 0) {
+ TRACE_DBG("padding %d bytes", psz);
+ iscsi_conn_init_read(conn,
+ (void __force __user *)&conn->rpadding, psz);
+ conn->read_state = RX_PADDING;
+ } else if ((conn->ddigest_type & DIGEST_NONE) != 0)
+ conn->read_state = RX_END;
+ else
+ conn->read_state = RX_INIT_DDIGEST;
+ }
+ break;
+
+ case RX_END:
+ if (unlikely(conn->read_size != 0)) {
+ PRINT_CRIT_ERROR("conn read_size !=0 on RX_END "
+ "(conn %p, op %x, read_size %d)", conn,
+ cmnd_opcode(cmnd), conn->read_size);
+ BUG();
+ }
+ conn->read_cmnd = NULL;
+ conn->read_state = RX_INIT_BHS;
+
+ cmnd_rx_end(cmnd);
+
+ EXTRACHECKS_BUG_ON(conn->read_size != 0);
+
+ /*
+ * To maintain fairness. Res must be 0 here anyway, the
+ * assignment is only to remove compiler warning about
+ * uninitialized variable.
+ */
+ res = 0;
+ goto out;
+
+ case RX_INIT_HDIGEST:
+ iscsi_conn_init_read(conn,
+ (void __force __user *)&cmnd->hdigest, sizeof(u32));
+ conn->read_state = RX_CHECK_HDIGEST;
+ /* go through */
+
+ case RX_CHECK_HDIGEST:
+ res = do_recv(conn);
+ if (res == 0) {
+ res = digest_rx_header(cmnd);
+ if (unlikely(res != 0)) {
+ PRINT_ERROR("rx header digest for "
+ "initiator %s failed (%d)",
+ conn->session->initiator_name,
+ res);
+ mark_conn_closed(conn);
+ } else
+ conn->read_state = RX_CMD_START;
+ }
+ break;
+
+ case RX_INIT_DDIGEST:
+ iscsi_conn_init_read(conn,
+ (void __force __user *)&cmnd->ddigest,
+ sizeof(u32));
+ conn->read_state = RX_CHECK_DDIGEST;
+ /* go through */
+
+ case RX_CHECK_DDIGEST:
+ res = iscsi_rx_check_ddigest(conn);
+ break;
+
+ case RX_AHS:
+ res = do_recv(conn);
+ if (res == 0) {
+ if ((conn->hdigest_type & DIGEST_NONE) == 0)
+ conn->read_state = RX_INIT_HDIGEST;
+ else
+ conn->read_state = RX_CMD_START;
+ }
+ break;
+
+ case RX_PADDING:
+ res = do_recv(conn);
+ if (res == 0) {
+ if ((conn->ddigest_type & DIGEST_NONE) == 0)
+ conn->read_state = RX_INIT_DDIGEST;
+ else
+ conn->read_state = RX_END;
+ }
+ break;
+
+ default:
+ PRINT_CRIT_ERROR("%d %x", conn->read_state, cmnd_opcode(cmnd));
+ res = -1; /* to keep compiler happy */
+ BUG();
+ }
+ } while (res == 0);
+
+ if (unlikely(conn->closing)) {
+ start_close_conn(conn);
+ *closed = 1;
+ }
+
+out:
+ return res;
+}
+
+/*
+ * Called under iscsi_rd_lock and BHs disabled, but will drop it inside,
+ * then reaquire.
+ */
+static void scst_do_job_rd(void)
+ __acquires(&iscsi_rd_lock)
+ __releases(&iscsi_rd_lock)
+{
+
+ /*
+ * We delete/add to tail connections to maintain fairness between them.
+ */
+
+ while (!list_empty(&iscsi_rd_list)) {
+ int closed = 0, rc;
+ struct iscsi_conn *conn = list_entry(iscsi_rd_list.next,
+ typeof(*conn), rd_list_entry);
+
+ list_del(&conn->rd_list_entry);
+
+ BUG_ON(conn->rd_state == ISCSI_CONN_RD_STATE_PROCESSING);
+ conn->rd_data_ready = 0;
+ conn->rd_state = ISCSI_CONN_RD_STATE_PROCESSING;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ conn->rd_task = current;
+#endif
+ spin_unlock_bh(&iscsi_rd_lock);
+
+ rc = process_read_io(conn, &closed);
+
+ spin_lock_bh(&iscsi_rd_lock);
+
+ if (unlikely(closed))
+ continue;
+
+ if (unlikely(conn->conn_tm_active)) {
+ spin_unlock_bh(&iscsi_rd_lock);
+ iscsi_check_tm_data_wait_timeouts(conn, false);
+ spin_lock_bh(&iscsi_rd_lock);
+ }
+
+#ifdef CONFIG_SCST_EXTRACHECKS
+ conn->rd_task = NULL;
+#endif
+ if ((rc == 0) || conn->rd_data_ready) {
+ list_add_tail(&conn->rd_list_entry, &iscsi_rd_list);
+ conn->rd_state = ISCSI_CONN_RD_STATE_IN_LIST;
+ } else
+ conn->rd_state = ISCSI_CONN_RD_STATE_IDLE;
+ }
+ return;
+}
+
+static inline int test_rd_list(void)
+{
+ int res = !list_empty(&iscsi_rd_list) ||
+ unlikely(kthread_should_stop());
+ return res;
+}
+
+int istrd(void *arg)
+{
+
+ PRINT_INFO("Read thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ spin_lock_bh(&iscsi_rd_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_rd_list()) {
+ add_wait_queue_exclusive_head(&iscsi_rd_waitQ, &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_rd_list())
+ break;
+ spin_unlock_bh(&iscsi_rd_lock);
+ schedule();
+ spin_lock_bh(&iscsi_rd_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&iscsi_rd_waitQ, &wait);
+ }
+ scst_do_job_rd();
+ }
+ spin_unlock_bh(&iscsi_rd_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so iscsi_rd_list must be empty.
+ */
+ BUG_ON(!list_empty(&iscsi_rd_list));
+
+ PRINT_INFO("Read thread PID %d finished", current->pid);
+ return 0;
+}
+
+static inline void check_net_priv(struct iscsi_cmnd *cmd, struct page *page) {}
+static inline void __iscsi_get_page_callback(struct iscsi_cmnd *cmd) {}
+static inline void __iscsi_put_page_callback(struct iscsi_cmnd *cmd) {}
+
+void req_add_to_write_timeout_list(struct iscsi_cmnd *req)
+{
+ struct iscsi_conn *conn;
+ unsigned long timeout_time;
+ bool set_conn_tm_active = false;
+
+ if (req->on_write_timeout_list)
+ goto out;
+
+ conn = req->conn;
+
+ TRACE_DBG("Adding req %p to conn %p write_timeout_list",
+ req, conn);
+
+ spin_lock_bh(&conn->write_list_lock);
+
+ /* Recheck, since it can be changed behind us */
+ if (unlikely(req->on_write_timeout_list)) {
+ spin_unlock_bh(&conn->write_list_lock);
+ goto out;
+ }
+
+ req->on_write_timeout_list = 1;
+ req->write_start = jiffies;
+
+ list_add_tail(&req->write_timeout_list_entry,
+ &conn->write_timeout_list);
+
+ if (!timer_pending(&conn->rsp_timer)) {
+ if (unlikely(conn->conn_tm_active ||
+ test_bit(ISCSI_CMD_ABORTED,
+ &req->prelim_compl_flags))) {
+ set_conn_tm_active = true;
+ timeout_time = req->write_start +
+ ISCSI_TM_DATA_WAIT_TIMEOUT +
+ ISCSI_ADD_SCHED_TIME;
+ } else
+ timeout_time = req->write_start +
+ conn->rsp_timeout + ISCSI_ADD_SCHED_TIME;
+
+ TRACE_DBG("Starting timer on %ld (con %p, write_start %ld)",
+ timeout_time, conn, req->write_start);
+
+ conn->rsp_timer.expires = timeout_time;
+ add_timer(&conn->rsp_timer);
+ } else if (unlikely(test_bit(ISCSI_CMD_ABORTED,
+ &req->prelim_compl_flags))) {
+ unsigned long timeout_time = jiffies +
+ ISCSI_TM_DATA_WAIT_TIMEOUT + ISCSI_ADD_SCHED_TIME;
+ set_conn_tm_active = true;
+ if (time_after(conn->rsp_timer.expires, timeout_time)) {
+ TRACE_MGMT_DBG("Mod timer on %ld (conn %p)",
+ timeout_time, conn);
+ mod_timer(&conn->rsp_timer, timeout_time);
+ }
+ }
+
+ spin_unlock_bh(&conn->write_list_lock);
+
+ /*
+ * conn_tm_active can be already cleared by
+ * iscsi_check_tm_data_wait_timeouts(). write_list_lock is an inner
+ * lock for iscsi_rd_lock.
+ */
+ if (unlikely(set_conn_tm_active)) {
+ spin_lock_bh(&iscsi_rd_lock);
+ TRACE_MGMT_DBG("Setting conn_tm_active for conn %p", conn);
+ conn->conn_tm_active = 1;
+ spin_unlock_bh(&iscsi_rd_lock);
+ }
+
+out:
+ return;
+}
+
+static int write_data(struct iscsi_conn *conn)
+{
+ mm_segment_t oldfs;
+ struct file *file;
+ struct iovec *iop;
+ struct socket *sock;
+ ssize_t (*sock_sendpage)(struct socket *, struct page *, int, size_t,
+ int);
+ ssize_t (*sendpage)(struct socket *, struct page *, int, size_t, int);
+ struct iscsi_cmnd *write_cmnd = conn->write_cmnd;
+ struct iscsi_cmnd *ref_cmd;
+ struct page *page;
+ struct scatterlist *sg;
+ int saved_size, size, sendsize;
+ int length, offset, idx;
+ int flags, res, count, sg_size;
+ bool do_put = false, ref_cmd_to_parent;
+
+ iscsi_extracheck_is_wr_thread(conn);
+
+ if (!write_cmnd->own_sg) {
+ ref_cmd = write_cmnd->parent_req;
+ ref_cmd_to_parent = true;
+ } else {
+ ref_cmd = write_cmnd;
+ ref_cmd_to_parent = false;
+ }
+
+ req_add_to_write_timeout_list(write_cmnd->parent_req);
+
+ file = conn->file;
+ size = conn->write_size;
+ saved_size = size;
+ iop = conn->write_iop;
+ count = conn->write_iop_used;
+
+ if (iop) {
+ while (1) {
+ loff_t off = 0;
+ int rest;
+
+ BUG_ON(count > (signed)(sizeof(conn->write_iov) /
+ sizeof(conn->write_iov[0])));
+retry:
+ oldfs = get_fs();
+ set_fs(KERNEL_DS);
+ res = vfs_writev(file,
+ (struct iovec __force __user *)iop,
+ count, &off);
+ set_fs(oldfs);
+ TRACE_WRITE("sid %#Lx, cid %u, res %d, iov_len %ld",
+ (long long unsigned int)conn->session->sid,
+ conn->cid, res, (long)iop->iov_len);
+ if (unlikely(res <= 0)) {
+ if (res == -EAGAIN) {
+ conn->write_iop = iop;
+ conn->write_iop_used = count;
+ goto out_iov;
+ } else if (res == -EINTR)
+ goto retry;
+ goto out_err;
+ }
+
+ rest = res;
+ size -= res;
+ while ((typeof(rest))iop->iov_len <= rest && rest) {
+ rest -= iop->iov_len;
+ iop++;
+ count--;
+ }
+ if (count == 0) {
+ conn->write_iop = NULL;
+ conn->write_iop_used = 0;
+ if (size)
+ break;
+ goto out_iov;
+ }
+ BUG_ON(iop > conn->write_iov + sizeof(conn->write_iov)
+ /sizeof(conn->write_iov[0]));
+ iop->iov_base += rest;
+ iop->iov_len -= rest;
+ }
+ }
+
+ sg = write_cmnd->sg;
+ if (unlikely(sg == NULL)) {
+ PRINT_INFO("WARNING: Data missed (cmd %p)!", write_cmnd);
+ res = 0;
+ goto out;
+ }
+
+ /* To protect from too early transfer completion race */
+ __iscsi_get_page_callback(ref_cmd);
+ do_put = true;
+
+ sock = conn->sock;
+
+ if ((write_cmnd->parent_req->scst_cmd != NULL) &&
+ scst_cmd_get_dh_data_buff_alloced(write_cmnd->parent_req->scst_cmd))
+ sock_sendpage = sock_no_sendpage;
+ else
+ sock_sendpage = sock->ops->sendpage;
+
+ flags = MSG_DONTWAIT;
+ sg_size = size;
+
+ if (sg != write_cmnd->rsp_sg) {
+ offset = conn->write_offset + sg[0].offset;
+ idx = offset >> PAGE_SHIFT;
+ offset &= ~PAGE_MASK;
+ length = min(size, (int)PAGE_SIZE - offset);
+ TRACE_WRITE("write_offset %d, sg_size %d, idx %d, offset %d, "
+ "length %d", conn->write_offset, sg_size, idx, offset,
+ length);
+ } else {
+ idx = 0;
+ offset = conn->write_offset;
+ while (offset >= sg[idx].length) {
+ offset -= sg[idx].length;
+ idx++;
+ }
+ length = sg[idx].length - offset;
+ offset += sg[idx].offset;
+ sock_sendpage = sock_no_sendpage;
+ TRACE_WRITE("rsp_sg: write_offset %d, sg_size %d, idx %d, "
+ "offset %d, length %d", conn->write_offset, sg_size,
+ idx, offset, length);
+ }
+ page = sg_page(&sg[idx]);
+
+ while (1) {
+ sendpage = sock_sendpage;
+
+ sendsize = min(size, length);
+ if (size <= sendsize) {
+retry2:
+ res = sendpage(sock, page, offset, size, flags);
+ TRACE_WRITE("Final %s sid %#Lx, cid %u, res %d (page "
+ "index %lu, offset %u, size %u, cmd %p, "
+ "page %p)", (sendpage != sock_no_sendpage) ?
+ "sendpage" : "sock_no_sendpage",
+ (long long unsigned int)conn->session->sid,
+ conn->cid, res, page->index,
+ offset, size, write_cmnd, page);
+ if (unlikely(res <= 0)) {
+ if (res == -EINTR)
+ goto retry2;
+ else
+ goto out_res;
+ }
+
+ check_net_priv(ref_cmd, page);
+ if (res == size) {
+ conn->write_size = 0;
+ res = saved_size;
+ goto out_put;
+ }
+
+ offset += res;
+ size -= res;
+ goto retry2;
+ }
+
+retry1:
+ res = sendpage(sock, page, offset, sendsize, flags | MSG_MORE);
+ TRACE_WRITE("%s sid %#Lx, cid %u, res %d (page index %lu, "
+ "offset %u, sendsize %u, size %u, cmd %p, page %p)",
+ (sendpage != sock_no_sendpage) ? "sendpage" :
+ "sock_no_sendpage",
+ (unsigned long long)conn->session->sid, conn->cid,
+ res, page->index, offset, sendsize, size,
+ write_cmnd, page);
+ if (unlikely(res <= 0)) {
+ if (res == -EINTR)
+ goto retry1;
+ else
+ goto out_res;
+ }
+
+ check_net_priv(ref_cmd, page);
+
+ size -= res;
+
+ if (res == sendsize) {
+ idx++;
+ EXTRACHECKS_BUG_ON(idx >= ref_cmd->sg_cnt);
+ page = sg_page(&sg[idx]);
+ length = sg[idx].length;
+ offset = sg[idx].offset;
+ } else {
+ offset += res;
+ sendsize -= res;
+ goto retry1;
+ }
+ }
+
+out_off:
+ conn->write_offset += sg_size - size;
+
+out_iov:
+ conn->write_size = size;
+ if ((saved_size == size) && res == -EAGAIN)
+ goto out_put;
+
+ res = saved_size - size;
+
+out_put:
+ if (do_put)
+ __iscsi_put_page_callback(ref_cmd);
+
+out:
+ return res;
+
+out_res:
+ check_net_priv(ref_cmd, page);
+ if (res == -EAGAIN)
+ goto out_off;
+ /* else go through */
+
+out_err:
+#ifndef CONFIG_SCST_DEBUG
+ if (!conn->closing)
+#endif
+ {
+ PRINT_ERROR("error %d at sid:cid %#Lx:%u, cmnd %p", res,
+ (long long unsigned int)conn->session->sid,
+ conn->cid, conn->write_cmnd);
+ }
+ if (ref_cmd_to_parent &&
+ ((ref_cmd->scst_cmd != NULL) || (ref_cmd->scst_aen != NULL))) {
+ if (ref_cmd->scst_state == ISCSI_CMD_STATE_AEN)
+ scst_set_aen_delivery_status(ref_cmd->scst_aen,
+ SCST_AEN_RES_FAILED);
+ else
+ scst_set_delivery_status(ref_cmd->scst_cmd,
+ SCST_CMD_DELIVERY_FAILED);
+ }
+ goto out_put;
+}
+
+static int exit_tx(struct iscsi_conn *conn, int res)
+{
+ iscsi_extracheck_is_wr_thread(conn);
+
+ switch (res) {
+ case -EAGAIN:
+ case -ERESTARTSYS:
+ res = 0;
+ break;
+ default:
+#ifndef CONFIG_SCST_DEBUG
+ if (!conn->closing)
+#endif
+ {
+ PRINT_ERROR("Sending data failed: initiator %s, "
+ "write_size %d, write_state %d, res %d",
+ conn->session->initiator_name,
+ conn->write_size,
+ conn->write_state, res);
+ }
+ conn->write_state = TX_END;
+ conn->write_size = 0;
+ mark_conn_closed(conn);
+ break;
+ }
+ return res;
+}
+
+static int tx_ddigest(struct iscsi_cmnd *cmnd, int state)
+{
+ int res, rest = cmnd->conn->write_size;
+ struct msghdr msg = {.msg_flags = MSG_NOSIGNAL | MSG_DONTWAIT};
+ struct kvec iov;
+
+ iscsi_extracheck_is_wr_thread(cmnd->conn);
+
+ TRACE_DBG("Sending data digest %x (cmd %p)", cmnd->ddigest, cmnd);
+
+ iov.iov_base = (char *)(&cmnd->ddigest) + (sizeof(u32) - rest);
+ iov.iov_len = rest;
+
+ res = kernel_sendmsg(cmnd->conn->sock, &msg, &iov, 1, rest);
+ if (res > 0) {
+ cmnd->conn->write_size -= res;
+ if (!cmnd->conn->write_size)
+ cmnd->conn->write_state = state;
+ } else
+ res = exit_tx(cmnd->conn, res);
+
+ return res;
+}
+
+static void init_tx_hdigest(struct iscsi_cmnd *cmnd)
+{
+ struct iscsi_conn *conn = cmnd->conn;
+ struct iovec *iop;
+
+ iscsi_extracheck_is_wr_thread(conn);
+
+ digest_tx_header(cmnd);
+
+ BUG_ON(conn->write_iop_used >=
+ (signed)(sizeof(conn->write_iov)/sizeof(conn->write_iov[0])));
+
+ iop = &conn->write_iop[conn->write_iop_used];
+ conn->write_iop_used++;
+ iop->iov_base = (void __force __user *)&(cmnd->hdigest);
+ iop->iov_len = sizeof(u32);
+ conn->write_size += sizeof(u32);
+
+ return;
+}
+
+static int tx_padding(struct iscsi_cmnd *cmnd, int state)
+{
+ int res, rest = cmnd->conn->write_size;
+ struct msghdr msg = {.msg_flags = MSG_NOSIGNAL | MSG_DONTWAIT};
+ struct kvec iov;
+ static const uint32_t padding;
+
+ iscsi_extracheck_is_wr_thread(cmnd->conn);
+
+ TRACE_DBG("Sending %d padding bytes (cmd %p)", rest, cmnd);
+
+ iov.iov_base = (char *)(&padding) + (sizeof(uint32_t) - rest);
+ iov.iov_len = rest;
+
+ res = kernel_sendmsg(cmnd->conn->sock, &msg, &iov, 1, rest);
+ if (res > 0) {
+ cmnd->conn->write_size -= res;
+ if (!cmnd->conn->write_size)
+ cmnd->conn->write_state = state;
+ } else
+ res = exit_tx(cmnd->conn, res);
+
+ return res;
+}
+
+static int iscsi_do_send(struct iscsi_conn *conn, int state)
+{
+ int res;
+
+ iscsi_extracheck_is_wr_thread(conn);
+
+ res = write_data(conn);
+ if (res > 0) {
+ if (!conn->write_size)
+ conn->write_state = state;
+ } else
+ res = exit_tx(conn, res);
+
+ return res;
+}
+
+/*
+ * No locks, conn is wr processing.
+ *
+ * IMPORTANT! Connection conn must be protected by additional conn_get()
+ * upon entrance in this function, because otherwise it could be destroyed
+ * inside as a result of cmnd release.
+ */
+int iscsi_send(struct iscsi_conn *conn)
+{
+ struct iscsi_cmnd *cmnd = conn->write_cmnd;
+ int ddigest, res = 0;
+
+ TRACE_DBG("conn %p, write_cmnd %p", conn, cmnd);
+
+ iscsi_extracheck_is_wr_thread(conn);
+
+ ddigest = conn->ddigest_type != DIGEST_NONE ? 1 : 0;
+
+ switch (conn->write_state) {
+ case TX_INIT:
+ BUG_ON(cmnd != NULL);
+ cmnd = conn->write_cmnd = iscsi_get_send_cmnd(conn);
+ if (!cmnd)
+ goto out;
+ cmnd_tx_start(cmnd);
+ if (!(conn->hdigest_type & DIGEST_NONE))
+ init_tx_hdigest(cmnd);
+ conn->write_state = TX_BHS_DATA;
+ case TX_BHS_DATA:
+ res = iscsi_do_send(conn, cmnd->pdu.datasize ?
+ TX_INIT_PADDING : TX_END);
+ if (res <= 0 || conn->write_state != TX_INIT_PADDING)
+ break;
+ case TX_INIT_PADDING:
+ cmnd->conn->write_size = ((cmnd->pdu.datasize + 3) & -4) -
+ cmnd->pdu.datasize;
+ if (cmnd->conn->write_size != 0)
+ conn->write_state = TX_PADDING;
+ else if (ddigest)
+ conn->write_state = TX_INIT_DDIGEST;
+ else
+ conn->write_state = TX_END;
+ break;
+ case TX_PADDING:
+ res = tx_padding(cmnd, ddigest ? TX_INIT_DDIGEST : TX_END);
+ if (res <= 0 || conn->write_state != TX_INIT_DDIGEST)
+ break;
+ case TX_INIT_DDIGEST:
+ cmnd->conn->write_size = sizeof(u32);
+ conn->write_state = TX_DDIGEST;
+ case TX_DDIGEST:
+ res = tx_ddigest(cmnd, TX_END);
+ break;
+ default:
+ PRINT_CRIT_ERROR("%d %d %x", res, conn->write_state,
+ cmnd_opcode(cmnd));
+ BUG();
+ }
+
+ if (res == 0)
+ goto out;
+
+ if (conn->write_state != TX_END)
+ goto out;
+
+ if (unlikely(conn->write_size)) {
+ PRINT_CRIT_ERROR("%d %x %u", res, cmnd_opcode(cmnd),
+ conn->write_size);
+ BUG();
+ }
+ cmnd_tx_end(cmnd);
+
+ rsp_cmnd_release(cmnd);
+
+ conn->write_cmnd = NULL;
+ conn->write_state = TX_INIT;
+
+out:
+ return res;
+}
+
+/* No locks, conn is wr processing.
+ *
+ * IMPORTANT! Connection conn must be protected by additional conn_get()
+ * upon entrance in this function, because otherwise it could be destroyed
+ * inside as a result of iscsi_send(), which releases sent commands.
+ */
+static int process_write_queue(struct iscsi_conn *conn)
+{
+ int res = 0;
+
+ if (likely(test_write_ready(conn)))
+ res = iscsi_send(conn);
+ return res;
+}
+
+/*
+ * Called under iscsi_wr_lock and BHs disabled, but will drop it inside,
+ * then reaquire.
+ */
+static void scst_do_job_wr(void)
+ __acquires(&iscsi_wr_lock)
+ __releases(&iscsi_wr_lock)
+{
+
+ /*
+ * We delete/add to tail connections to maintain fairness between them.
+ */
+
+ while (!list_empty(&iscsi_wr_list)) {
+ int rc;
+ struct iscsi_conn *conn = list_entry(iscsi_wr_list.next,
+ typeof(*conn), wr_list_entry);
+
+ TRACE_DBG("conn %p, wr_state %x, wr_space_ready %d, "
+ "write ready %d", conn, conn->wr_state,
+ conn->wr_space_ready, test_write_ready(conn));
+
+ list_del(&conn->wr_list_entry);
+
+ BUG_ON(conn->wr_state == ISCSI_CONN_WR_STATE_PROCESSING);
+
+ conn->wr_state = ISCSI_CONN_WR_STATE_PROCESSING;
+ conn->wr_space_ready = 0;
+#ifdef CONFIG_SCST_EXTRACHECKS
+ conn->wr_task = current;
+#endif
+ spin_unlock_bh(&iscsi_wr_lock);
+
+ conn_get(conn);
+
+ rc = process_write_queue(conn);
+
+ spin_lock_bh(&iscsi_wr_lock);
+#ifdef CONFIG_SCST_EXTRACHECKS
+ conn->wr_task = NULL;
+#endif
+ if ((rc == -EAGAIN) && !conn->wr_space_ready) {
+ conn->wr_state = ISCSI_CONN_WR_STATE_SPACE_WAIT;
+ goto cont;
+ }
+
+ if (test_write_ready(conn)) {
+ list_add_tail(&conn->wr_list_entry, &iscsi_wr_list);
+ conn->wr_state = ISCSI_CONN_WR_STATE_IN_LIST;
+ } else
+ conn->wr_state = ISCSI_CONN_WR_STATE_IDLE;
+
+cont:
+ conn_put(conn);
+ }
+ return;
+}
+
+static inline int test_wr_list(void)
+{
+ int res = !list_empty(&iscsi_wr_list) ||
+ unlikely(kthread_should_stop());
+ return res;
+}
+
+int istwr(void *arg)
+{
+
+ PRINT_INFO("Write thread started, PID %d", current->pid);
+
+ current->flags |= PF_NOFREEZE;
+
+ spin_lock_bh(&iscsi_wr_lock);
+ while (!kthread_should_stop()) {
+ wait_queue_t wait;
+ init_waitqueue_entry(&wait, current);
+
+ if (!test_wr_list()) {
+ add_wait_queue_exclusive_head(&iscsi_wr_waitQ, &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (test_wr_list())
+ break;
+ spin_unlock_bh(&iscsi_wr_lock);
+ schedule();
+ spin_lock_bh(&iscsi_wr_lock);
+ }
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&iscsi_wr_waitQ, &wait);
+ }
+ scst_do_job_wr();
+ }
+ spin_unlock_bh(&iscsi_wr_lock);
+
+ /*
+ * If kthread_should_stop() is true, we are guaranteed to be
+ * on the module unload, so iscsi_wr_list must be empty.
+ */
+ BUG_ON(!list_empty(&iscsi_wr_list));
+
+ PRINT_INFO("Write thread PID %d finished", current->pid);
+ return 0;
+}
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/param.c linux-2.6.33/drivers/scst/iscsi-scst/param.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/param.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/param.c
@@ -0,0 +1,306 @@
+/*
+ * Copyright (C) 2005 FUJITA Tomonori <tomof@acm.org>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "iscsi.h"
+#include "digest.h"
+
+#define CHECK_PARAM(info, iparams, word, min, max) \
+do { \
+ if (!(info)->partial || ((info)->partial & 1 << key_##word)) { \
+ TRACE_DBG("%s: %u", #word, (iparams)[key_##word]); \
+ if ((iparams)[key_##word] < (min) || \
+ (iparams)[key_##word] > (max)) { \
+ if ((iparams)[key_##word] < (min)) { \
+ (iparams)[key_##word] = (min); \
+ PRINT_WARNING("%s: %u is too small, resetting " \
+ "it to allowed min %u", \
+ #word, (iparams)[key_##word], (min)); \
+ } else { \
+ PRINT_WARNING("%s: %u is too big, resetting " \
+ "it to allowed max %u", \
+ #word, (iparams)[key_##word], (max)); \
+ (iparams)[key_##word] = (max); \
+ } \
+ } \
+ } \
+} while (0)
+
+#define SET_PARAM(params, info, iparams, word) \
+({ \
+ int changed = 0; \
+ if (!(info)->partial || ((info)->partial & 1 << key_##word)) { \
+ if ((params)->word != (iparams)[key_##word]) \
+ changed = 1; \
+ (params)->word = (iparams)[key_##word]; \
+ TRACE_DBG("%s set to %u", #word, (params)->word); \
+ } \
+ changed; \
+})
+
+#define GET_PARAM(params, info, iparams, word) \
+do { \
+ (iparams)[key_##word] = (params)->word; \
+} while (0)
+
+const char *iscsi_get_bool_value(int val)
+{
+ if (val)
+ return "Yes";
+ else
+ return "No";
+}
+
+const char *iscsi_get_digest_name(int val, char *res)
+{
+ int pos = 0;
+
+ if (val & DIGEST_NONE)
+ pos = sprintf(&res[pos], "%s", "None");
+
+ if (val & DIGEST_CRC32C)
+ pos += sprintf(&res[pos], "%s%s", (pos != 0) ? ", " : "",
+ "CRC32C");
+
+ if (pos == 0)
+ sprintf(&res[pos], "%s", "Unknown");
+
+ return res;
+}
+
+static void log_params(struct iscsi_sess_params *params)
+{
+ char digest_name[64];
+
+ PRINT_INFO("Negotiated parameters: InitialR2T %s, ImmediateData %s, "
+ "MaxConnections %d, MaxRecvDataSegmentLength %d, "
+ "MaxXmitDataSegmentLength %d, ",
+ iscsi_get_bool_value(params->initial_r2t),
+ iscsi_get_bool_value(params->immediate_data), params->max_connections,
+ params->max_recv_data_length, params->max_xmit_data_length);
+ PRINT_INFO(" MaxBurstLength %d, FirstBurstLength %d, "
+ "DefaultTime2Wait %d, DefaultTime2Retain %d, ",
+ params->max_burst_length, params->first_burst_length,
+ params->default_wait_time, params->default_retain_time);
+ PRINT_INFO(" MaxOutstandingR2T %d, DataPDUInOrder %s, "
+ "DataSequenceInOrder %s, ErrorRecoveryLevel %d, ",
+ params->max_outstanding_r2t,
+ iscsi_get_bool_value(params->data_pdu_inorder),
+ iscsi_get_bool_value(params->data_sequence_inorder),
+ params->error_recovery_level);
+ PRINT_INFO(" HeaderDigest %s, DataDigest %s, OFMarker %s, "
+ "IFMarker %s, OFMarkInt %d, IFMarkInt %d",
+ iscsi_get_digest_name(params->header_digest, digest_name),
+ iscsi_get_digest_name(params->data_digest, digest_name),
+ iscsi_get_bool_value(params->ofmarker),
+ iscsi_get_bool_value(params->ifmarker),
+ params->ofmarkint, params->ifmarkint);
+}
+
+/* target_mutex supposed to be locked */
+static void sess_params_check(struct iscsi_kern_params_info *info)
+{
+ int32_t *iparams = info->session_params;
+ const int max_len = ISCSI_CONN_IOV_MAX * PAGE_SIZE;
+
+ CHECK_PARAM(info, iparams, initial_r2t, 0, 1);
+ CHECK_PARAM(info, iparams, immediate_data, 0, 1);
+ CHECK_PARAM(info, iparams, max_connections, 1, 1);
+ CHECK_PARAM(info, iparams, max_recv_data_length, 512, max_len);
+ CHECK_PARAM(info, iparams, max_xmit_data_length, 512, max_len);
+ CHECK_PARAM(info, iparams, max_burst_length, 512, max_len);
+ CHECK_PARAM(info, iparams, first_burst_length, 512, max_len);
+ CHECK_PARAM(info, iparams, max_outstanding_r2t, 1, 65535);
+ CHECK_PARAM(info, iparams, error_recovery_level, 0, 0);
+ CHECK_PARAM(info, iparams, data_pdu_inorder, 0, 1);
+ CHECK_PARAM(info, iparams, data_sequence_inorder, 0, 1);
+
+ digest_alg_available(&iparams[key_header_digest]);
+ digest_alg_available(&iparams[key_data_digest]);
+
+ CHECK_PARAM(info, iparams, ofmarker, 0, 0);
+ CHECK_PARAM(info, iparams, ifmarker, 0, 0);
+
+ return;
+}
+
+/* target_mutex supposed to be locked */
+static void sess_params_set(struct iscsi_sess_params *params,
+ struct iscsi_kern_params_info *info)
+{
+ int32_t *iparams = info->session_params;
+
+ SET_PARAM(params, info, iparams, initial_r2t);
+ SET_PARAM(params, info, iparams, immediate_data);
+ SET_PARAM(params, info, iparams, max_connections);
+ SET_PARAM(params, info, iparams, max_recv_data_length);
+ SET_PARAM(params, info, iparams, max_xmit_data_length);
+ SET_PARAM(params, info, iparams, max_burst_length);
+ SET_PARAM(params, info, iparams, first_burst_length);
+ SET_PARAM(params, info, iparams, default_wait_time);
+ SET_PARAM(params, info, iparams, default_retain_time);
+ SET_PARAM(params, info, iparams, max_outstanding_r2t);
+ SET_PARAM(params, info, iparams, data_pdu_inorder);
+ SET_PARAM(params, info, iparams, data_sequence_inorder);
+ SET_PARAM(params, info, iparams, error_recovery_level);
+ SET_PARAM(params, info, iparams, header_digest);
+ SET_PARAM(params, info, iparams, data_digest);
+ SET_PARAM(params, info, iparams, ofmarker);
+ SET_PARAM(params, info, iparams, ifmarker);
+ SET_PARAM(params, info, iparams, ofmarkint);
+ SET_PARAM(params, info, iparams, ifmarkint);
+ return;
+}
+
+static void sess_params_get(struct iscsi_sess_params *params,
+ struct iscsi_kern_params_info *info)
+{
+ int32_t *iparams = info->session_params;
+
+ GET_PARAM(params, info, iparams, initial_r2t);
+ GET_PARAM(params, info, iparams, immediate_data);
+ GET_PARAM(params, info, iparams, max_connections);
+ GET_PARAM(params, info, iparams, max_recv_data_length);
+ GET_PARAM(params, info, iparams, max_xmit_data_length);
+ GET_PARAM(params, info, iparams, max_burst_length);
+ GET_PARAM(params, info, iparams, first_burst_length);
+ GET_PARAM(params, info, iparams, default_wait_time);
+ GET_PARAM(params, info, iparams, default_retain_time);
+ GET_PARAM(params, info, iparams, max_outstanding_r2t);
+ GET_PARAM(params, info, iparams, data_pdu_inorder);
+ GET_PARAM(params, info, iparams, data_sequence_inorder);
+ GET_PARAM(params, info, iparams, error_recovery_level);
+ GET_PARAM(params, info, iparams, header_digest);
+ GET_PARAM(params, info, iparams, data_digest);
+ GET_PARAM(params, info, iparams, ofmarker);
+ GET_PARAM(params, info, iparams, ifmarker);
+ GET_PARAM(params, info, iparams, ofmarkint);
+ GET_PARAM(params, info, iparams, ifmarkint);
+ return;
+}
+
+/* target_mutex supposed to be locked */
+static void tgt_params_check(struct iscsi_session *session,
+ struct iscsi_kern_params_info *info)
+{
+ int32_t *iparams = info->target_params;
+
+ CHECK_PARAM(info, iparams, queued_cmnds, MIN_NR_QUEUED_CMNDS,
+ min_t(int, MAX_NR_QUEUED_CMNDS,
+ scst_get_max_lun_commands(session->scst_sess, NO_SUCH_LUN)));
+ CHECK_PARAM(info, iparams, rsp_timeout, MIN_RSP_TIMEOUT,
+ MAX_RSP_TIMEOUT);
+ CHECK_PARAM(info, iparams, nop_in_interval, MIN_NOP_IN_INTERVAL,
+ MAX_NOP_IN_INTERVAL);
+ return;
+}
+
+/* target_mutex supposed to be locked */
+static int iscsi_tgt_params_set(struct iscsi_session *session,
+ struct iscsi_kern_params_info *info, int set)
+{
+ struct iscsi_tgt_params *params = &session->tgt_params;
+ int32_t *iparams = info->target_params;
+
+ if (set) {
+ struct iscsi_conn *conn;
+
+ tgt_params_check(session, info);
+
+ SET_PARAM(params, info, iparams, queued_cmnds);
+ SET_PARAM(params, info, iparams, rsp_timeout);
+ SET_PARAM(params, info, iparams, nop_in_interval);
+
+ PRINT_INFO("Target parameters set for session %llx: "
+ "QueuedCommands %d, Response timeout %d, Nop-In "
+ "interval %d", session->sid, params->queued_cmnds,
+ params->rsp_timeout, params->nop_in_interval);
+
+ list_for_each_entry(conn, &session->conn_list,
+ conn_list_entry) {
+ conn->rsp_timeout = session->tgt_params.rsp_timeout * HZ;
+ conn->nop_in_interval = session->tgt_params.nop_in_interval * HZ;
+ spin_lock_bh(&iscsi_rd_lock);
+ if (!conn->closing && (conn->nop_in_interval > 0)) {
+ TRACE_DBG("Schedule Nop-In work for conn %p", conn);
+ schedule_delayed_work(&conn->nop_in_delayed_work,
+ conn->nop_in_interval + ISCSI_ADD_SCHED_TIME);
+ }
+ spin_unlock_bh(&iscsi_rd_lock);
+ }
+ } else {
+ GET_PARAM(params, info, iparams, queued_cmnds);
+ GET_PARAM(params, info, iparams, rsp_timeout);
+ GET_PARAM(params, info, iparams, nop_in_interval);
+ }
+
+ return 0;
+}
+
+/* target_mutex supposed to be locked */
+static int iscsi_sess_params_set(struct iscsi_session *session,
+ struct iscsi_kern_params_info *info, int set)
+{
+ struct iscsi_sess_params *params;
+
+ if (set)
+ sess_params_check(info);
+
+ params = &session->sess_params;
+
+ if (set) {
+ sess_params_set(params, info);
+ log_params(params);
+ } else
+ sess_params_get(params, info);
+
+ return 0;
+}
+
+/* target_mutex supposed to be locked */
+int iscsi_params_set(struct iscsi_target *target,
+ struct iscsi_kern_params_info *info, int set)
+{
+ int err;
+ struct iscsi_session *session;
+
+ if (info->sid == 0) {
+ PRINT_ERROR("sid must not be %d", 0);
+ err = -EINVAL;
+ goto out;
+ }
+
+ session = session_lookup(target, info->sid);
+ if (session == NULL) {
+ PRINT_ERROR("Session for sid %llx not found", info->sid);
+ err = -ENOENT;
+ goto out;
+ }
+
+ if (set && !list_empty(&session->conn_list) &&
+ (info->params_type != key_target)) {
+ err = -EBUSY;
+ goto out;
+ }
+
+ if (info->params_type == key_session)
+ err = iscsi_sess_params_set(session, info, set);
+ else if (info->params_type == key_target)
+ err = iscsi_tgt_params_set(session, info, set);
+ else
+ err = -EINVAL;
+
+out:
+ return err;
+}
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/session.c linux-2.6.33/drivers/scst/iscsi-scst/session.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/session.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/session.c
@@ -0,0 +1,482 @@
+/*
+ * Copyright (C) 2002 - 2003 Ardis Technolgies <roman@ardistech.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include "iscsi.h"
+
+/* target_mutex supposed to be locked */
+struct iscsi_session *session_lookup(struct iscsi_target *target, u64 sid)
+{
+ struct iscsi_session *session;
+
+ list_for_each_entry(session, &target->session_list,
+ session_list_entry) {
+ if (session->sid == sid)
+ return session;
+ }
+ return NULL;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int iscsi_session_alloc(struct iscsi_target *target,
+ struct iscsi_kern_session_info *info, struct iscsi_session **result)
+{
+ int err;
+ unsigned int i;
+ struct iscsi_session *session;
+ char *name = NULL;
+
+ session = kzalloc(sizeof(*session), GFP_KERNEL);
+ if (!session)
+ return -ENOMEM;
+
+ session->target = target;
+ session->sid = info->sid;
+ atomic_set(&session->active_cmds, 0);
+ session->exp_cmd_sn = info->exp_cmd_sn;
+
+ session->initiator_name = kstrdup(info->initiator_name, GFP_KERNEL);
+ if (!session->initiator_name) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ name = info->initiator_name;
+
+ INIT_LIST_HEAD(&session->conn_list);
+ INIT_LIST_HEAD(&session->pending_list);
+
+ spin_lock_init(&session->sn_lock);
+
+ spin_lock_init(&session->cmnd_data_wait_hash_lock);
+ for (i = 0; i < ARRAY_SIZE(session->cmnd_data_wait_hash); i++)
+ INIT_LIST_HEAD(&session->cmnd_data_wait_hash[i]);
+
+ session->next_ttt = 1;
+
+ session->scst_sess = scst_register_session(target->scst_tgt, 0,
+ name, NULL, NULL);
+ if (session->scst_sess == NULL) {
+ PRINT_ERROR("%s", "scst_register_session() failed");
+ err = -ENOMEM;
+ goto err;
+ }
+
+ scst_sess_set_tgt_priv(session->scst_sess, session);
+
+ TRACE_MGMT_DBG("Session %p created: target %p, tid %u, sid %#Lx",
+ session, target, target->tid, info->sid);
+
+ *result = session;
+ return 0;
+
+err:
+ if (session) {
+ kfree(session->initiator_name);
+ kfree(session);
+ }
+ return err;
+}
+
+/* target_mutex supposed to be locked */
+void sess_reinst_finished(struct iscsi_session *sess)
+{
+ struct iscsi_conn *c;
+
+ TRACE_MGMT_DBG("Enabling reinstate successor sess %p", sess);
+
+ BUG_ON(!sess->sess_reinstating);
+
+ list_for_each_entry(c, &sess->conn_list, conn_list_entry) {
+ conn_reinst_finished(c);
+ }
+ sess->sess_reinstating = 0;
+ return;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+int __add_session(struct iscsi_target *target,
+ struct iscsi_kern_session_info *info)
+{
+ struct iscsi_session *new_sess = NULL, *sess, *old_sess;
+ int err = 0, i;
+ union iscsi_sid sid;
+ bool reinstatement = false;
+ struct iscsi_kern_params_info *params_info;
+
+ TRACE_MGMT_DBG("Adding session SID %llx", info->sid);
+
+ err = iscsi_session_alloc(target, info, &new_sess);
+ if (err != 0)
+ goto out;
+
+ mutex_lock(&target->target_mutex);
+
+ sess = session_lookup(target, info->sid);
+ if (sess != NULL) {
+ PRINT_ERROR("Attempt to add session with existing SID %llx",
+ info->sid);
+ err = -EEXIST;
+ goto out_err_unlock;
+ }
+
+ params_info = kmalloc(sizeof(*params_info), GFP_KERNEL);
+ if (params_info == NULL) {
+ PRINT_ERROR("Unable to allocate params info (size %zd)",
+ sizeof(*params_info));
+ err = -ENOMEM;
+ goto out_err_unlock;
+ }
+
+ sid = *(union iscsi_sid *)&info->sid;
+ sid.id.tsih = 0;
+ old_sess = NULL;
+
+ /*
+ * We need to find the latest session to correctly handle
+ * multi-reinstatements
+ */
+ list_for_each_entry_reverse(sess, &target->session_list,
+ session_list_entry) {
+ union iscsi_sid i = *(union iscsi_sid *)&sess->sid;
+ i.id.tsih = 0;
+ if ((sid.id64 == i.id64) &&
+ (strcmp(info->initiator_name, sess->initiator_name) == 0)) {
+ if (!sess->sess_shutting_down) {
+ /* session reinstatement */
+ old_sess = sess;
+ }
+ break;
+ }
+ }
+ sess = NULL;
+
+ list_add_tail(&new_sess->session_list_entry, &target->session_list);
+
+ memset(params_info, 0, sizeof(*params_info));
+ params_info->tid = target->tid;
+ params_info->sid = info->sid;
+ params_info->params_type = key_session;
+ for (i = 0; i < session_key_last; i++)
+ params_info->session_params[i] = info->session_params[i];
+
+ err = iscsi_params_set(target, params_info, 1);
+ if (err != 0)
+ goto out_del;
+
+ memset(params_info, 0, sizeof(*params_info));
+ params_info->tid = target->tid;
+ params_info->sid = info->sid;
+ params_info->params_type = key_target;
+ for (i = 0; i < target_key_last; i++)
+ params_info->target_params[i] = info->target_params[i];
+
+ err = iscsi_params_set(target, params_info, 1);
+ if (err != 0)
+ goto out_del;
+
+ kfree(params_info);
+ params_info = NULL;
+
+ if (old_sess != NULL) {
+ reinstatement = true;
+
+ TRACE_MGMT_DBG("Reinstating sess %p with SID %llx (old %p, "
+ "SID %llx)", new_sess, new_sess->sid, old_sess,
+ old_sess->sid);
+
+ new_sess->sess_reinstating = 1;
+ old_sess->sess_reinst_successor = new_sess;
+
+ target_del_session(old_sess->target, old_sess, 0);
+ }
+
+ mutex_unlock(&target->target_mutex);
+
+ if (reinstatement) {
+ /*
+ * Mutex target_mgmt_mutex won't allow to add connections to
+ * the new session after target_mutex was dropped, so it's safe
+ * to replace the initial UA without it. We can't do it under
+ * target_mutex, because otherwise we can establish a
+ * circular locking dependency between target_mutex and
+ * scst_mutex in SCST core (iscsi_report_aen() called by
+ * SCST core under scst_mutex).
+ */
+ scst_set_initial_UA(new_sess->scst_sess,
+ SCST_LOAD_SENSE(scst_sense_nexus_loss_UA));
+ }
+
+out:
+ return err;
+
+out_del:
+ list_del(&new_sess->session_list_entry);
+ kfree(params_info);
+
+out_err_unlock:
+ mutex_unlock(&target->target_mutex);
+
+ scst_unregister_session(new_sess->scst_sess, 1, NULL);
+ new_sess->scst_sess = NULL;
+
+ mutex_lock(&target->target_mutex);
+ session_free(new_sess, false);
+ mutex_unlock(&target->target_mutex);
+ goto out;
+}
+
+static void __session_free(struct iscsi_session *session)
+{
+ kfree(session->initiator_name);
+ kfree(session);
+}
+
+static void iscsi_unreg_sess_done(struct scst_session *scst_sess)
+{
+ struct iscsi_session *session;
+
+ session = (struct iscsi_session *)scst_sess_get_tgt_priv(scst_sess);
+
+ session->scst_sess = NULL;
+ __session_free(session);
+ return;
+}
+
+/* target_mutex supposed to be locked */
+int session_free(struct iscsi_session *session, bool del)
+{
+ unsigned int i;
+
+ TRACE_MGMT_DBG("Freeing session %p (SID %llx)",
+ session, session->sid);
+
+ BUG_ON(!list_empty(&session->conn_list));
+ if (unlikely(atomic_read(&session->active_cmds) != 0)) {
+ PRINT_CRIT_ERROR("active_cmds not 0 (%d)!!",
+ atomic_read(&session->active_cmds));
+ BUG();
+ }
+
+ for (i = 0; i < ARRAY_SIZE(session->cmnd_data_wait_hash); i++)
+ BUG_ON(!list_empty(&session->cmnd_data_wait_hash[i]));
+
+ if (session->sess_reinst_successor != NULL)
+ sess_reinst_finished(session->sess_reinst_successor);
+
+ if (session->sess_reinstating) {
+ struct iscsi_session *s;
+ TRACE_MGMT_DBG("Freeing being reinstated sess %p", session);
+ list_for_each_entry(s, &session->target->session_list,
+ session_list_entry) {
+ if (s->sess_reinst_successor == session) {
+ s->sess_reinst_successor = NULL;
+ break;
+ }
+ }
+ }
+
+ if (del)
+ list_del(&session->session_list_entry);
+
+ if (session->scst_sess != NULL) {
+ /*
+ * We must NOT call scst_unregister_session() in the waiting
+ * mode, since we are under target_mutex. Otherwise we can
+ * establish a circular locking dependency between target_mutex
+ * and scst_mutex in SCST core (iscsi_report_aen() called by
+ * SCST core under scst_mutex).
+ */
+ scst_unregister_session(session->scst_sess, 0,
+ iscsi_unreg_sess_done);
+ } else
+ __session_free(session);
+
+ return 0;
+}
+
+/* target_mutex supposed to be locked */
+int __del_session(struct iscsi_target *target, u64 sid)
+{
+ struct iscsi_session *session;
+
+ session = session_lookup(target, sid);
+ if (!session)
+ return -ENOENT;
+
+ if (!list_empty(&session->conn_list)) {
+ PRINT_ERROR("%llu still have connections",
+ (long long unsigned int)session->sid);
+ return -EBUSY;
+ }
+
+ return session_free(session, true);
+}
+
+#define ISCSI_SESS_BOOL_PARAM_ATTR(name, exported_name) \
+static ssize_t iscsi_sess_show_##name(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buf) \
+{ \
+ int pos; \
+ struct scst_session *scst_sess; \
+ struct iscsi_session *sess; \
+ \
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj); \
+ sess = (struct iscsi_session *)scst_sess_get_tgt_priv(scst_sess); \
+ \
+ pos = sprintf(buf, "%s\n", \
+ iscsi_get_bool_value(sess->sess_params.name)); \
+ \
+ return pos; \
+} \
+ \
+static struct kobj_attribute iscsi_sess_attr_##name = \
+ __ATTR(exported_name, S_IRUGO, iscsi_sess_show_##name, NULL);
+
+#define ISCSI_SESS_INT_PARAM_ATTR(name, exported_name) \
+static ssize_t iscsi_sess_show_##name(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buf) \
+{ \
+ int pos; \
+ struct scst_session *scst_sess; \
+ struct iscsi_session *sess; \
+ \
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj); \
+ sess = (struct iscsi_session *)scst_sess_get_tgt_priv(scst_sess); \
+ \
+ pos = sprintf(buf, "%d\n", sess->sess_params.name); \
+ \
+ return pos; \
+} \
+ \
+static struct kobj_attribute iscsi_sess_attr_##name = \
+ __ATTR(exported_name, S_IRUGO, iscsi_sess_show_##name, NULL);
+
+#define ISCSI_SESS_DIGEST_PARAM_ATTR(name, exported_name) \
+static ssize_t iscsi_sess_show_##name(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buf) \
+{ \
+ int pos; \
+ struct scst_session *scst_sess; \
+ struct iscsi_session *sess; \
+ char digest_name[64]; \
+ \
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj); \
+ sess = (struct iscsi_session *)scst_sess_get_tgt_priv(scst_sess); \
+ \
+ pos = sprintf(buf, "%s\n", iscsi_get_digest_name( \
+ sess->sess_params.name, digest_name)); \
+ \
+ return pos; \
+} \
+ \
+static struct kobj_attribute iscsi_sess_attr_##name = \
+ __ATTR(exported_name, S_IRUGO, iscsi_sess_show_##name, NULL);
+
+ISCSI_SESS_BOOL_PARAM_ATTR(initial_r2t, InitialR2T);
+ISCSI_SESS_BOOL_PARAM_ATTR(immediate_data, ImmediateData);
+ISCSI_SESS_INT_PARAM_ATTR(max_recv_data_length, MaxRecvDataSegmentLength);
+ISCSI_SESS_INT_PARAM_ATTR(max_xmit_data_length, MaxXmitDataSegmentLength);
+ISCSI_SESS_INT_PARAM_ATTR(max_burst_length, MaxBurstLength);
+ISCSI_SESS_INT_PARAM_ATTR(first_burst_length, FirstBurstLength);
+ISCSI_SESS_INT_PARAM_ATTR(max_outstanding_r2t, MaxOutstandingR2T);
+ISCSI_SESS_DIGEST_PARAM_ATTR(header_digest, HeaderDigest);
+ISCSI_SESS_DIGEST_PARAM_ATTR(data_digest, DataDigest);
+
+static ssize_t iscsi_sess_sid_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct scst_session *scst_sess;
+ struct iscsi_session *sess;
+
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj);
+ sess = (struct iscsi_session *)scst_sess_get_tgt_priv(scst_sess);
+
+ pos = sprintf(buf, "%llx\n", sess->sid);
+ return pos;
+}
+
+static struct kobj_attribute iscsi_attr_sess_sid =
+ __ATTR(sid, S_IRUGO, iscsi_sess_sid_show, NULL);
+
+static ssize_t iscsi_sess_reinstating_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct scst_session *scst_sess;
+ struct iscsi_session *sess;
+
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj);
+ sess = (struct iscsi_session *)scst_sess_get_tgt_priv(scst_sess);
+
+ pos = sprintf(buf, "%d\n", sess->sess_reinstating ? 1 : 0);
+ return pos;
+}
+
+static struct kobj_attribute iscsi_sess_attr_reinstating =
+ __ATTR(reinstating, S_IRUGO, iscsi_sess_reinstating_show, NULL);
+
+static ssize_t iscsi_sess_force_close_store(struct kobject *kobj,
+ struct kobj_attribute *attr, const char *buf, size_t count)
+{
+ int res;
+ struct scst_session *scst_sess;
+ struct iscsi_session *sess;
+ struct iscsi_conn *conn;
+
+ scst_sess = container_of(kobj, struct scst_session, sess_kobj);
+ sess = (struct iscsi_session *)scst_sess_get_tgt_priv(scst_sess);
+
+ if (mutex_lock_interruptible(&sess->target->target_mutex) != 0) {
+ res = -EINTR;
+ goto out;
+ }
+
+ PRINT_INFO("Deleting session %llu with initiator %s (%p)",
+ (long long unsigned int)sess->sid, sess->initiator_name, sess);
+
+ list_for_each_entry(conn, &sess->conn_list, conn_list_entry) {
+ TRACE_MGMT_DBG("Deleting connection with initiator %p", conn);
+ __mark_conn_closed(conn, ISCSI_CONN_ACTIVE_CLOSE|ISCSI_CONN_DELETING);
+ }
+
+ mutex_unlock(&sess->target->target_mutex);
+
+ res = count;
+
+out:
+ return res;
+}
+
+static struct kobj_attribute iscsi_sess_attr_force_close =
+ __ATTR(force_close, S_IWUSR, NULL, iscsi_sess_force_close_store);
+
+const struct attribute *iscsi_sess_attrs[] = {
+ &iscsi_sess_attr_initial_r2t.attr,
+ &iscsi_sess_attr_immediate_data.attr,
+ &iscsi_sess_attr_max_recv_data_length.attr,
+ &iscsi_sess_attr_max_xmit_data_length.attr,
+ &iscsi_sess_attr_max_burst_length.attr,
+ &iscsi_sess_attr_first_burst_length.attr,
+ &iscsi_sess_attr_max_outstanding_r2t.attr,
+ &iscsi_sess_attr_header_digest.attr,
+ &iscsi_sess_attr_data_digest.attr,
+ &iscsi_attr_sess_sid.attr,
+ &iscsi_sess_attr_reinstating.attr,
+ &iscsi_sess_attr_force_close.attr,
+ NULL,
+};
+
diff -uprN orig/linux-2.6.33/drivers/scst/iscsi-scst/target.c linux-2.6.33/drivers/scst/iscsi-scst/target.c
--- orig/linux-2.6.33/drivers/scst/iscsi-scst/target.c
+++ linux-2.6.33/drivers/scst/iscsi-scst/target.c
@@ -0,0 +1,500 @@
+/*
+ * Copyright (C) 2002 - 2003 Ardis Technolgies <roman@ardistech.com>
+ * Copyright (C) 2007 - 2010 Vladislav Bolkhovitin
+ * Copyright (C) 2007 - 2010 ID7 Ltd.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation, version 2
+ * of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/delay.h>
+
+#include "iscsi.h"
+#include "digest.h"
+
+#define MAX_NR_TARGETS (1UL << 30)
+
+DEFINE_MUTEX(target_mgmt_mutex);
+
+/* All 3 protected by target_mgmt_mutex */
+static LIST_HEAD(target_list);
+static u32 next_target_id;
+static u32 nr_targets;
+
+/* target_mgmt_mutex supposed to be locked */
+struct iscsi_target *target_lookup_by_id(u32 id)
+{
+ struct iscsi_target *target;
+
+ list_for_each_entry(target, &target_list, target_list_entry) {
+ if (target->tid == id)
+ return target;
+ }
+ return NULL;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static struct iscsi_target *target_lookup_by_name(const char *name)
+{
+ struct iscsi_target *target;
+
+ list_for_each_entry(target, &target_list, target_list_entry) {
+ if (!strcmp(target->name, name))
+ return target;
+ }
+ return NULL;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+static int iscsi_target_create(struct iscsi_kern_target_info *info, u32 tid,
+ struct iscsi_target **out_target)
+{
+ int err = -EINVAL, len;
+ char *name = info->name;
+ struct iscsi_target *target;
+
+ TRACE_MGMT_DBG("Creating target tid %u, name %s", tid, name);
+
+ len = strlen(name);
+ if (!len) {
+ PRINT_ERROR("The length of the target name is zero %u", tid);
+ goto out;
+ }
+
+ if (!try_module_get(THIS_MODULE)) {
+ PRINT_ERROR("Fail to get module %u", tid);
+ goto out;
+ }
+
+ target = kzalloc(sizeof(*target), GFP_KERNEL);
+ if (!target) {
+ err = -ENOMEM;
+ goto out_put;
+ }
+
+ target->tid = info->tid = tid;
+
+ strlcpy(target->name, name, sizeof(target->name));
+
+ mutex_init(&target->target_mutex);
+ INIT_LIST_HEAD(&target->session_list);
+ INIT_LIST_HEAD(&target->attrs_list);
+
+ target->scst_tgt = scst_register_target(&iscsi_template, target->name);
+ if (!target->scst_tgt) {
+ PRINT_ERROR("%s", "scst_register_target() failed");
+ err = -EBUSY;
+ goto out_free;
+ }
+
+ scst_tgt_set_tgt_priv(target->scst_tgt, target);
+
+ list_add_tail(&target->target_list_entry, &target_list);
+
+ *out_target = target;
+
+ return 0;
+
+out_free:
+ kfree(target);
+
+out_put:
+ module_put(THIS_MODULE);
+
+out:
+ return err;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+int __add_target(struct iscsi_kern_target_info *info)
+{
+ int err;
+ u32 tid = info->tid;
+ struct iscsi_target *target;
+ struct iscsi_kern_params_info *params_info;
+ struct iscsi_kern_attr *attr_info;
+ union add_info_union {
+ struct iscsi_kern_params_info params_info;
+ struct iscsi_kern_attr attr_info;
+ } *add_info;
+ int i, rc;
+ unsigned long attrs_ptr_long;
+ struct iscsi_kern_attr __user *attrs_ptr;
+
+ if (nr_targets > MAX_NR_TARGETS) {
+ err = -EBUSY;
+ goto out;
+ }
+
+ if (target_lookup_by_name(info->name)) {
+ PRINT_ERROR("Target %s already exist!", info->name);
+ err = -EEXIST;
+ goto out;
+ }
+
+ if (tid && target_lookup_by_id(tid)) {
+ PRINT_ERROR("Target %u already exist!", tid);
+ err = -EEXIST;
+ goto out;
+ }
+
+ add_info = kmalloc(sizeof(*add_info), GFP_KERNEL);
+ if (add_info == NULL) {
+ PRINT_ERROR("Unable to allocate additional info (size %zd)",
+ sizeof(*add_info));
+ err = -ENOMEM;
+ goto out;
+ }
+ params_info = (struct iscsi_kern_params_info *)add_info;
+ attr_info = (struct iscsi_kern_attr *)add_info;
+
+ if (tid == 0) {
+ do {
+ if (!++next_target_id)
+ ++next_target_id;
+ } while (target_lookup_by_id(next_target_id));
+
+ tid = next_target_id;
+ }
+
+ err = iscsi_target_create(info, tid, &target);
+ if (err != 0)
+ goto out_free;
+
+ nr_targets++;
+
+ mutex_lock(&target->target_mutex);
+
+ attrs_ptr_long = info->attrs_ptr;
+ attrs_ptr = (struct iscsi_kern_attr __user *)attrs_ptr_long;
+ for (i = 0; i < info->attrs_num; i++) {
+ memset(attr_info, 0, sizeof(*attr_info));
+
+ rc = copy_from_user(attr_info, attrs_ptr, sizeof(*attr_info));
+ if (rc != 0) {
+ PRINT_ERROR("Failed to copy users of target %s "
+ "failed", info->name);
+ err = -EFAULT;
+ goto out_del_unlock;
+ }
+
+ attr_info->name[sizeof(attr_info->name)-1] = '\0';
+
+ err = iscsi_add_attr(target, attr_info);
+ if (err != 0)
+ goto out_del_unlock;
+
+ attrs_ptr++;
+ }
+
+ mutex_unlock(&target->target_mutex);
+
+ err = tid;
+
+out_free:
+ kfree(add_info);
+
+out:
+ return err;
+
+out_del_unlock:
+ mutex_unlock(&target->target_mutex);
+ __del_target(tid);
+ goto out_free;
+}
+
+static void target_destroy(struct iscsi_target *target)
+{
+ struct iscsi_attr *attr, *t;
+
+ TRACE_MGMT_DBG("Destroying target tid %u", target->tid);
+
+ list_for_each_entry_safe(attr, t, &target->attrs_list,
+ attrs_list_entry) {
+ __iscsi_del_attr(target, attr);
+ }
+
+ scst_unregister_target(target->scst_tgt);
+
+ kfree(target);
+
+ module_put(THIS_MODULE);
+ return;
+}
+
+/* target_mgmt_mutex supposed to be locked */
+int __del_target(u32 id)
+{
+ struct iscsi_target *target;
+ int err;
+
+ target = target_lookup_by_id(id);
+ if (!target) {
+ err = -ENOENT;
+ goto out;
+ }
+
+ mutex_lock(&target->target_mutex);
+
+ if (!list_empty(&target->session_list)) {
+ err = -EBUSY;
+ goto out_unlock;
+ }
+
+ list_del(&target->target_list_entry);
+ nr_targets--;
+
+ mutex_unlock(&target->target_mutex);
+
+ target_destroy(target);
+ return 0;
+
+out_unlock:
+ mutex_unlock(&target->target_mutex);
+
+out:
+ return err;
+}
+
+/* target_mutex supposed to be locked */
+void target_del_session(struct iscsi_target *target,
+ struct iscsi_session *session, int flags)
+{
+
+ TRACE_MGMT_DBG("Deleting session %p", session);
+
+ if (!list_empty(&session->conn_list)) {
+ struct iscsi_conn *conn, *tc;
+ list_for_each_entry_safe(conn, tc, &session->conn_list,
+ conn_list_entry) {
+ TRACE_MGMT_DBG("Mark conn %p closing", conn);
+ __mark_conn_closed(conn, flags);
+ }
+ } else {
+ TRACE_MGMT_DBG("Freeing session %p without connections",
+ session);
+ __del_session(target, session->sid);
+ }
+ return;
+}
+
+/* target_mutex supposed to be locked */
+void target_del_all_sess(struct iscsi_target *target, int flags)
+{
+ struct iscsi_session *session, *ts;
+
+ if (!list_empty(&target->session_list)) {
+ TRACE_MGMT_DBG("Deleting all sessions from target %p", target);
+ list_for_each_entry_safe(session, ts, &target->session_list,
+ session_list_entry) {
+ target_del_session(target, session, flags);
+ }
+ }
+ return;
+}
+
+void target_del_all(void)
+{
+ struct iscsi_target *target, *t;
+ bool first = true;
+
+ TRACE_MGMT_DBG("%s", "Deleting all targets");
+
+ /* Not the best, ToDo */
+ while (1) {
+ mutex_lock(&target_mgmt_mutex);
+
+ if (list_empty(&target_list))
+ break;
+
+ /*
+ * In the first iteration we won't delete targets to go at
+ * first through all sessions of all targets and close their
+ * connections. Otherwise we can stuck for noticeable time
+ * waiting during a target's unregistration for the activities
+ * suspending over active connection. This can especially got
+ * bad if any being wait connection itself stuck waiting for
+ * something and can be recovered only by connection close.
+ * Let's for such cases not wait while such connection recover
+ * theyself, but act in advance.
+ */
+
+ list_for_each_entry_safe(target, t, &target_list,
+ target_list_entry) {
+ mutex_lock(&target->target_mutex);
+
+ if (!list_empty(&target->session_list)) {
+ target_del_all_sess(target,
+ ISCSI_CONN_ACTIVE_CLOSE |
+ ISCSI_CONN_DELETING);
+ } else if (!first) {
+ TRACE_MGMT_DBG("Deleting target %p", target);
+ list_del(&target->target_list_entry);
+ nr_targets--;
+ mutex_unlock(&target->target_mutex);
+ target_destroy(target);
+ continue;
+ }
+
+ mutex_unlock(&target->target_mutex);
+ }
+ mutex_unlock(&target_mgmt_mutex);
+ msleep(100);
+
+ first = false;
+ }
+
+ mutex_unlock(&target_mgmt_mutex);
+
+ TRACE_MGMT_DBG("%s", "Deleting all targets finished");
+ return;
+}
+
+static ssize_t iscsi_tgt_tid_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int pos;
+ struct scst_tgt *scst_tgt;
+ struct iscsi_target *tgt;
+
+ scst_tgt = container_of(kobj, struct scst_tgt, tgt_kobj);
+ tgt = (struct iscsi_target *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ pos = sprintf(buf, "%u\n", tgt->tid);
+ return pos;
+}
+
+static struct kobj_attribute iscsi_tgt_attr_tid =
+ __ATTR(tid, S_IRUGO, iscsi_tgt_tid_show, NULL);
+
+const struct attribute *iscsi_tgt_attrs[] = {
+ &iscsi_tgt_attr_tid.attr,
+ NULL,
+};
+
+ssize_t iscsi_sysfs_send_event(uint32_t tid, enum iscsi_kern_event_code code,
+ const char *param1, const char *param2, void **data)
+{
+ int res;
+ struct scst_sysfs_user_info *info;
+
+ if (ctr_open_state != ISCSI_CTR_OPEN_STATE_OPEN) {
+ PRINT_ERROR("%s", "User space process not connected");
+ res = -EPERM;
+ goto out;
+ }
+
+ res = scst_sysfs_user_add_info(&info);
+ if (res != 0)
+ goto out;
+
+ TRACE_DBG("Sending event %d (tid %d, param1 %s, param2 %s, cookie %d, "
+ "info %p)", tid, code, param1, param2, info->info_cookie, info);
+
+ res = event_send(tid, 0, 0, info->info_cookie, code, param1, param2);
+ if (res <= 0) {
+ PRINT_ERROR("event_send() failed: %d", res);
+ if (res == 0)
+ res = -EFAULT;
+ goto out_free;
+ }
+
+ /*
+ * It may wait 30 secs in blocking connect to an unreacheable
+ * iSNS server. It must be fixed, but not now. ToDo.
+ */
+ res = scst_wait_info_completion(info, 31 * HZ);
+
+ if (data != NULL)
+ *data = info->data;
+
+out_free:
+ scst_sysfs_user_del_info(info);
+
+out:
+ return res;
+}
+
+int iscsi_enable_target(struct scst_tgt *scst_tgt, bool enable)
+{
+ struct iscsi_target *tgt =
+ (struct iscsi_target *)scst_tgt_get_tgt_priv(scst_tgt);
+ int res;
+ uint32_t type;
+
+ if (enable)
+ type = E_ENABLE_TARGET;
+ else
+ type = E_DISABLE_TARGET;
+
+ TRACE_DBG("%s target %d", enable ? "Enabling" : "Disabling", tgt->tid);
+
+ res = iscsi_sysfs_send_event(tgt->tid, type, NULL, NULL, NULL);
+ return res;
+}
+
+bool iscsi_is_target_enabled(struct scst_tgt *scst_tgt)
+{
+ struct iscsi_target *tgt =
+ (struct iscsi_target *)scst_tgt_get_tgt_priv(scst_tgt);
+
+ return tgt->tgt_enabled;
+}
+
+ssize_t iscsi_sysfs_add_target(const char *target_name, char *params)
+{
+ int res;
+
+ res = iscsi_sysfs_send_event(0, E_ADD_TARGET, target_name,
+ params, NULL);
+ if (res > 0) {
+ /* It's tid */
+ res = 0;
+ }
+ return res;
+}
+
+ssize_t iscsi_sysfs_del_target(const char *target_name)
+{
+ int res = 0, tid;
+
+ /* We don't want to have tgt visible after the mutex unlock */
+ {
+ struct iscsi_target *tgt;
+ mutex_lock(&target_mgmt_mutex);
+ tgt = target_lookup_by_name(target_name);
+ if (tgt == NULL) {
+ PRINT_ERROR("Target %s not found", target_name);
+ mutex_unlock(&target_mgmt_mutex);
+ res = -ENOENT;
+ goto out;
+ }
+ tid = tgt->tid;
+ mutex_unlock(&target_mgmt_mutex);
+ }
+
+ TRACE_DBG("Deleting target %s (tid %d)", target_name, tid);
+
+ res = iscsi_sysfs_send_event(tid, E_DEL_TARGET, NULL, NULL, NULL);
+
+out:
+ return res;
+}
+
+ssize_t iscsi_sysfs_mgmt_cmd(char *cmd)
+{
+ int res;
+
+ TRACE_DBG("Sending mgmt cmd %s", cmd);
+
+ res = iscsi_sysfs_send_event(0, E_MGMT_CMD, cmd, NULL, NULL);
+ return res;
+}
+
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH][RFC 4/4/4/5] iSCSI-SCST's README file
[not found] ` <4BC45EF7.7010304@vlnb.net>
2010-04-13 13:10 ` [PATCH][RFC 3/4/4/5] iSCSI-SCST's implementation files Vladislav Bolkhovitin
@ 2010-04-13 13:10 ` Vladislav Bolkhovitin
1 sibling, 0 replies; 18+ messages in thread
From: Vladislav Bolkhovitin @ 2010-04-13 13:10 UTC (permalink / raw)
To: linux-scsi
Cc: linux-kernel, scst-devel, James Bottomley, Andrew Morton,
FUJITA Tomonori, Mike Christie, Jeff Garzik, Linus Torvalds,
Bart Van Assche
This patch contains iSCSI-SCST's README file.
Signed-off-by: Vladislav Bolkhovitin <vst@vlnb.net>
---
README.iscsi | 585 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 585 insertions(+)
diff -uprN orig/linux-2.6.33/Documentation/scst/README.iscsi linux-2.6.33/Documentation/scst/README.iscsi
--- orig/linux-2.6.33/Documentation/scst/README.iscsi
+++ linux-2.6.33/Documentation/scst/README.iscsi
@@ -0,0 +1,585 @@
+iSCSI SCST target driver
+========================
+
+ISCSI-SCST is a deeply reworked fork of iSCSI Enterprise Target (IET)
+(http://iscsitarget.sourceforge.net). Reasons of the fork were:
+
+ - To be able to use full power of SCST core.
+
+ - To fix all the problems, corner cases issues and iSCSI standard
+ violations which IET has.
+
+See more info at http://iscsi-scst.sourceforge.net.
+
+Usage
+-----
+
+See in http://iscsi-scst.sourceforge.net/iscsi-scst-howto.txt how to
+configure iSCSI-SCST.
+
+In 2.0.0 usage of iscsi-scstd.conf as well as iscsi-scst-adm utility is
+obsolete. Use the sysfs interface facilities instead.
+
+It is recommended to use TEST UNIT READY ("tur") command to check if
+iSCSI-SCST target is alive in MPIO configurations.
+
+Also see SCST README file how to tune for the best performance.
+
+CAUTION: Working of target and initiator on the same host isn't fully
+======= supported. See SCST README file for details.
+
+Sysfs interface
+---------------
+
+Root of SCST sysfs interface is /sys/kernel/scst_tgt. Root of iSCSI-SCST
+is /sys/kernel/scst_tgt/targets/iscsi. It has the following entries:
+
+ - None, one or more subdirectories for targets with name equal to names
+ of the corresponding targets.
+
+ - IncomingUser[num] - optional one or more attributes containing user
+ name and password for incoming discovery user name. Not exist by
+ default and can be added through "mgmt" entry, see below.
+
+ - OutgoingUser - optional attribute containing user name and password
+ for outgoing discovery user name. Not exist by default and can be
+ added through "mgmt" entry, see below.
+
+ - iSNSServer - contains name or IP address of iSNS server with optional
+ "AccessControl" attribute, which allows to enable iSNS access
+ control. Empty by default.
+
+ - enabled - using this attribute you can enable or disable iSCSI-SCST
+ accept new connections. It allows to finish configuring global
+ iSCSI-SCST attributes before it starts accepting new connections. 0
+ by default.
+
+ - open_state - read-only attribute, which allows to see if the user
+ space part of iSCSI-SCST connected to the kernel part.
+
+ - trace_level - allows to enable and disable various tracing
+ facilities. See content of this file for help how to use it.
+
+ - version - read-only attribute, which allows to see version of
+ iSCSI-SCST and enabled optional features.
+
+ - mgmt - main management entry, which allows to configure iSCSI-SCST.
+ Namely, add/delete targets as well as add/delete optional global and
+ per-target attributes. See content of this file for help how to use
+ it.
+
+Each iSCSI-SCST sysfs file (attribute) can contain in the last line mark
+"[key]". It is automatically added mark used to allow scstadmin to see
+which attributes it should save in the config file. You can ignore it.
+
+Each target subdirectory contains the following entries:
+
+ - ini_groups - subdirectory defining initiator groups for this target,
+ used to define per-initiator access control. See SCST core README for
+ more details.
+
+ - luns - subdirectory defining LUNs of this target. See SCST core
+ README for more details.
+
+ - sessions - subdirectory containing connected to this target sessions.
+
+ - IncomingUser[num] - optional one or more attributes containing user
+ name and password for incoming user name. Not exist by default and can
+ be added through the "mgmt" entry, see above.
+
+ - OutgoingUser - optional attribute containing user name and password
+ for outgoing user name. Not exist by default and can be added through
+ the "mgmt" entry, see above.
+
+ - Entries defining default iSCSI parameters values used during iSCSI
+ parameters negotiation. Only entries which can be changed or make
+ sense are listed there.
+
+ - QueuedCommands - defines maximum number of commands queued to any
+ session of this target. Default is 32 commands.
+
+ - RspTimeout - defines the maximum time in seconds a command can wait for
+ response from initiator, otherwise the corresponding connection will
+ be closed. For performance reasons it is implemented as a timer,
+ which once in RspTimeout time checks the oldest command waiting for
+ response and, if it's older than RspTimeout, then it closes the
+ connection. Hence, a stalled connection will be closed in time
+ between RspTimeout and 2*RspTimeout. Default is 30 seconds.
+
+ - NopInInterval - defines interval between NOP-In requests, which the
+ target will send on idle connections to check if the initiator is
+ still alive. If there is no NOP-Out reply from the initiator in
+ RspTimeout time, the corresponding connection will be closed. Default
+ is 30 seconds. If it's set to 0, then NOP-In requests are disabled.
+
+ - enabled - using this attribute you can enable or disable iSCSI-SCST
+ accept new connections to this target. It allows to finish
+ configuring it before it starts accepting new connections. 0 by
+ default.
+
+ - rel_tgt_id - allows to read or write SCSI Relative Target Port
+ Identifier attribute. This identifier is used to identify SCSI Target
+ Ports by some SCSI commands, mainly by Persistent Reservations
+ commands. This identifier must be unique among all SCST targets, but
+ for convenience SCST allows disabled targets to have not unique
+ rel_tgt_id. In this case SCST will not allow to enable this target
+ until rel_tgt_id becomes unique. This attribute initialized unique by
+ SCST by default.
+
+ - tid - TID of this target.
+
+Subdirectory "sessions" contains one subdirectory for each connected
+session with name equal to name of the connected initiator.
+
+Each session subdirectory contains the following entries:
+
+ - One subdirectory for each TCP connection in this session. ISCSI-SCST
+ supports 1 connection per session, but the session subdirectory can
+ contain several connections: one active and other being closed.
+
+ - Entries defining negotiated iSCSI parameters. Only parameters which
+ can be changed or make sense are listed there.
+
+ - initiator_name - contains initiator name
+
+ - sid - contains SID of this session
+
+ - reinstating - contains reinstatement state of this session
+
+ - force_close - write-only attribute, which allows to force close this
+ session. This is the only writable session attribute.
+
+ - active_commands - contains number of active, i.e. not yet or being
+ executed, SCSI commands in this session.
+
+ - commands - contains overall number of SCSI commands in this session.
+
+Each connection subdirectory contains the following entries:
+
+ - cid - contains CID of this connection.
+
+ - ip - contains IP address of the connected initiator.
+
+ - state - contains processing state of this connection.
+
+Below is a sample script, which configures 1 virtual disk "disk1" using
+/disk1 image and one target iqn.2006-10.net.vlnb:tgt with all default
+parameters:
+
+#!/bin/bash
+
+modprobe scst
+modprobe scst_vdisk
+
+echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
+
+service iscsi-scst start
+
+echo "add_target iqn.2006-10.net.vlnb:tgt" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+echo "add disk1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
+
+echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/enabled
+echo 1 >/sys/kernel/scst_tgt/targets/iscsi/enabled
+
+Below is more advanced sample script, which configures more virtual
+devices of various types, including virtual CDROM and 2 targets, one
+with all default parameters, another one with some not default
+parameters, incoming and outgoing user names for CHAP authentification,
+and special permissions for initiator iqn.2005-03.org.open-iscsi:cacdcd2520,
+which will see another set of devices. Also this sample configures CHAP
+authentication for discovery sessions and iSNS server with access control.
+
+#!/bin/bash
+
+modprobe scst
+modprobe scst_vdisk
+
+echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
+echo "add_device disk2 filename=/disk2; blocksize=4096; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
+echo "add_device blockio filename=/dev/sda5" >/sys/kernel/scst_tgt/handlers/vdisk_blockio/mgmt
+echo "add_device nullio" >/sys/kernel/scst_tgt/handlers/vdisk_nullio/mgmt
+echo "add_device cdrom" >/sys/kernel/scst_tgt/handlers/vcdrom/mgmt
+
+service iscsi-scst start
+
+echo "192.168.1.16 AccessControl" >/sys/kernel/scst_tgt/targets/iscsi/iSNSServer
+echo "add_attribute IncomingUser joeD 12charsecret" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+echo "add_attribute OutgoingUser jackD 12charsecret1" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+
+echo "add_target iqn.2006-10.net.vlnb:tgt" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+
+echo "add disk1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
+echo "add cdrom 1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
+
+echo "add_target iqn.2006-10.net.vlnb:tgt1" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+echo "add_target_attribute iqn.2006-10.net.vlnb:tgt1 IncomingUser1 joe2 12charsecret2" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+echo "add_target_attribute iqn.2006-10.net.vlnb:tgt1 IncomingUser joe 12charsecret" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+echo "add_target_attribute iqn.2006-10.net.vlnb:tgt1 OutgoingUser jim1 12charpasswd" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
+echo "No" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/InitialR2T
+echo "Yes" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ImmediateData
+echo "8192" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxRecvDataSegmentLength
+echo "8192" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxXmitDataSegmentLength
+echo "131072" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxBurstLength
+echo "32768" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/FirstBurstLength
+echo "1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxOutstandingR2T
+echo "CRC32C,None" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/HeaderDigest
+echo "CRC32C,None" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/DataDigest
+echo "32" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/QueuedCommands
+
+echo "add disk2 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/mgmt
+echo "add nullio 26" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/mgmt
+
+echo "create special_ini" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/mgmt
+echo "add blockio 0 read_only=1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/special_ini/luns/mgmt
+echo "add iqn.2005-03.org.open-iscsi:cacdcd2520" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/special_ini/initiators/mgmt
+
+echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/enabled
+echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/enabled
+
+echo 1 >/sys/kernel/scst_tgt/targets/iscsi/enabled
+
+The resulting overall SCST sysfs hierarchy with an initiator connected to
+both iSCSI-SCST targets will look like:
+
+/sys/kernel/scst_tgt
+|-- devices
+| |-- blockio
+| | |-- blocksize
+| | |-- exported
+| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/special_ini/luns/0
+| | |-- filename
+| | |-- handler -> ../../handlers/vdisk_blockio
+| | |-- read_only
+| | |-- removable
+| | |-- resync_size
+| | |-- size_mb
+| | |-- t10_dev_id
+| | |-- threads_num
+| | |-- threads_pool_type
+| | |-- type
+| | `-- usn
+| |-- cdrom
+| | |-- exported
+| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/1
+| | |-- filename
+| | |-- handler -> ../../handlers/vcdrom
+| | |-- size_mb
+| | |-- t10_dev_id
+| | |-- threads_num
+| | |-- threads_pool_type
+| | |-- type
+| | `-- usn
+| |-- disk1
+| | |-- blocksize
+| | |-- exported
+| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/0
+| | |-- filename
+| | |-- handler -> ../../handlers/vdisk_fileio
+| | |-- nv_cache
+| | |-- o_direct
+| | |-- read_only
+| | |-- removable
+| | |-- resync_size
+| | |-- size_mb
+| | |-- t10_dev_id
+| | |-- type
+| | |-- usn
+| | `-- write_through
+| |-- disk2
+| | |-- blocksize
+| | |-- exported
+| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/0
+| | |-- filename
+| | |-- handler -> ../../handlers/vdisk_fileio
+| | |-- nv_cache
+| | |-- o_direct
+| | |-- read_only
+| | |-- removable
+| | |-- resync_size
+| | |-- size_mb
+| | |-- t10_dev_id
+| | |-- threads_num
+| | |-- threads_pool_type
+| | |-- threads_num
+| | |-- threads_pool_type
+| | |-- type
+| | |-- usn
+| | `-- write_through
+| `-- nullio
+| |-- blocksize
+| |-- exported
+| | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/26
+| |-- handler -> ../../handlers/vdisk_nullio
+| |-- read_only
+| |-- removable
+| |-- size_mb
+| |-- t10_dev_id
+| |-- threads_num
+| |-- threads_pool_type
+| |-- type
+| `-- usn
+|-- handlers
+| |-- vcdrom
+| | |-- cdrom -> ../../devices/cdrom
+| | |-- mgmt
+| | |-- trace_level
+| | `-- type
+| |-- vdisk_blockio
+| | |-- blockio -> ../../devices/blockio
+| | |-- mgmt
+| | |-- trace_level
+| | `-- type
+| |-- vdisk_fileio
+| | |-- disk1 -> ../../devices/disk1
+| | |-- disk2 -> ../../devices/disk2
+| | |-- mgmt
+| | |-- trace_level
+| | `-- type
+| `-- vdisk_nullio
+| |-- mgmt
+| |-- nullio -> ../../devices/nullio
+| |-- trace_level
+| `-- type
+|-- sgv
+| |-- global_stats
+| |-- sgv
+| | `-- stats
+| |-- sgv-clust
+| | `-- stats
+| `-- sgv-dma
+| `-- stats
+|-- targets
+| `-- iscsi
+| |-- IncomingUser
+| |-- OutgoingUser
+| |-- enabled
+| |-- iSNSServer
+| |-- iqn.2006-10.net.vlnb:tgt
+| | |-- DataDigest
+| | |-- FirstBurstLength
+| | |-- HeaderDigest
+| | |-- ImmediateData
+| | |-- InitialR2T
+| | |-- MaxBurstLength
+| | |-- MaxOutstandingR2T
+| | |-- MaxRecvDataSegmentLength
+| | |-- MaxXmitDataSegmentLength
+| | |-- NopInInterval
+| | |-- QueuedCommands
+| | |-- RspTimeout
+| | |-- enabled
+| | |-- ini_groups
+| | | `-- mgmt
+| | |-- luns
+| | | |-- 0
+| | | | |-- device -> ../../../../../devices/disk1
+| | | | `-- read_only
+| | | |-- 1
+| | | | |-- device -> ../../../../../devices/cdrom
+| | | | `-- read_only
+| | | `-- mgmt
+| | |-- rel_tgt_id
+| | |-- sessions
+| | | `-- iqn.2005-03.org.open-iscsi:cacdcd2520
+| | | |-- 10.170.75.2
+| | | | |-- cid
+| | | | |-- ip
+| | | | `-- state
+| | | |-- DataDigest
+| | | |-- FirstBurstLength
+| | | |-- HeaderDigest
+| | | |-- ImmediateData
+| | | |-- InitialR2T
+| | | |-- MaxBurstLength
+| | | |-- MaxOutstandingR2T
+| | | |-- MaxRecvDataSegmentLength
+| | | |-- MaxXmitDataSegmentLength
+| | | |-- active_commands
+| | | |-- commands
+| | | |-- force_close
+| | | |-- initiator_name
+| | | |-- luns -> ../../luns
+| | | |-- reinstating
+| | | `-- sid
+| | `-- tid
+| |-- iqn.2006-10.net.vlnb:tgt1
+| | |-- DataDigest
+| | |-- FirstBurstLength
+| | |-- HeaderDigest
+| | |-- ImmediateData
+| | |-- IncomingUser
+| | |-- IncomingUser1
+| | |-- InitialR2T
+| | |-- MaxBurstLength
+| | |-- MaxOutstandingR2T
+| | |-- MaxRecvDataSegmentLength
+| | |-- MaxXmitDataSegmentLength
+| | |-- OutgoingUser
+| | |-- NopInInterval
+| | |-- QueuedCommands
+| | |-- RspTimeout
+| | |-- enabled
+| | |-- ini_groups
+| | | |-- mgmt
+| | | `-- special_ini
+| | | |-- initiators
+| | | | |-- iqn.2005-03.org.open-iscsi:cacdcd2520
+| | | | `-- mgmt
+| | | `-- luns
+| | | |-- 0
+| | | | |-- device -> ../../../../../../../devices/blockio
+| | | | `-- read_only
+| | | `-- mgmt
+| | |-- luns
+| | | |-- 0
+| | | | |-- device -> ../../../../../devices/disk2
+| | | | `-- read_only
+| | | |-- 26
+| | | | |-- device -> ../../../../../devices/nullio
+| | | | `-- read_only
+| | | `-- mgmt
+| | |-- rel_tgt_id
+| | |-- sessions
+| | | `-- iqn.2005-03.org.open-iscsi:cacdcd2520
+| | | |-- 10.170.75.2
+| | | | |-- cid
+| | | | |-- ip
+| | | | `-- state
+| | | |-- DataDigest
+| | | |-- FirstBurstLength
+| | | |-- HeaderDigest
+| | | |-- ImmediateData
+| | | |-- InitialR2T
+| | | |-- MaxBurstLength
+| | | |-- MaxOutstandingR2T
+| | | |-- MaxRecvDataSegmentLength
+| | | |-- MaxXmitDataSegmentLength
+| | | |-- active_commands
+| | | |-- commands
+| | | |-- force_close
+| | | |-- initiator_name
+| | | |-- luns -> ../../ini_groups/special_ini/luns
+| | | |-- reinstating
+| | | `-- sid
+| | `-- tid
+| |-- mgmt
+| |-- open_state
+| |-- trace_level
+| `-- version
+|-- threads
+|-- trace_level
+`-- version
+
+Troubleshooting
+---------------
+
+If you have any problems, start troubleshooting from looking at the
+kernel and system logs. In the kernel log iSCSI-SCST and SCST core send
+their messages, in the system log iscsi-scstd sends its messages. In
+most Linux distributions both those logs are put to /var/log/messages
+file.
+
+Then, it might be helpful to increase level of logging. For kernel
+modules you should make the debug build by enabling CONFIG_SCST_DEBUG.
+
+If after looking on the logs the reason of your problem is still unclear
+for you, report to SCST mailing list scst-devel@lists.sourceforge.net.
+
+Work if target's backstorage or link is too slow
+------------------------------------------------
+
+In some cases you can experience I/O stalls or see in the kernel log
+abort or reset messages. It can happen under high I/O load, when your
+target's backstorage gets overloaded, or working over a slow link, when
+the link can't serve all the queued commands on time,
+
+To workaround it you can reduce QueuedCommands parameter for the
+corresponding target to some lower value, like 8 (default is 32).
+
+Also see SCST README file for more details about that issue and ways to
+prevent it.
+
+Performance advices
+-------------------
+
+1. If you use Windows XP or Windows 2003+ as initiators, you should
+consider to decrease TcpAckFrequency parameter to 1. See
+http://support.microsoft.com/kb/328890/ or google for "TcpAckFrequency"
+for more details.
+
+2. See how to get the maximum throughput from iSCSI, for instance, at
+http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-mutual-iscsi-customers-using-vmware.html.
+It's about VMware, but its recommendations apply to other environments
+as well.
+
+3. ISCSI initiators built in pre-CentOS/RHEL 5 reported to have some
+performance problems. If you use it, it is strongly advised to upgrade.
+
+4. If you are going to use your target in an VM environment, for
+instance as a shared storage with VMware, make sure all your VMs
+connected to the target via *separate* sessions, i.e. each VM has own
+connection to the target, not all VMs connected using a single
+connection. You can check it using SCST sysfs interface. If you
+miss it, you can greatly loose performance of parallel access to your
+target from different VMs. This isn't related to the case if your VMs
+are using the same shared storage, like with VMFS, for instance. In this
+case all your VM hosts will be connected to the target via separate
+sessions, which is enough.
+
+5. Many dual port network adapters are not able to transfer data
+simultaneously on both ports, i.e. they transfer data via both ports on
+the same speed as via any single port. Thus, using such adapters in MPIO
+configuration can't improve performance. To allow MPIO to have double
+performance you should either use separate network adapters, or find a
+dual-port adapter capable to to transfer data simultaneously on both
+ports. You can check it by running 2 iperf's through both ports in
+parallel.
+
+6. Since network offload works much better in the write direction, than
+for reading (simplifying, in the read direction often there's additional
+data copy) in many cases with 10GbE in a single initiator-target pair
+the initiator's CPU is a bottleneck, so you can see the initiator can
+read data on much slower rate, than write. You can check it by watching
+*each particular* CPU load to find out if any of them is close to 100%
+load, including IRQ processing load. Note, many tools like vmstat give
+aggregate load on all CPUs, so with 4 cores 25% corresponds to 100% load
+of any single CPU.
+
+7. See SCST core's README for more advices. Especially pay attention to
+have io_grouping_type option set correctly.
+
+Compilation options
+-------------------
+
+There are the following compilation options, that could be commented
+in/out in the kernel's module Makefile:
+
+ - CONFIG_SCST_DEBUG - turns on some debugging code, including some logging.
+ Makes the driver considerably bigger and slower, producing large amount of
+ log data.
+
+ - CONFIG_SCST_TRACING - turns on ability to log events. Makes the driver
+ considerably bigger and leads to some performance loss.
+
+ - CONFIG_SCST_EXTRACHECKS - adds extra validity checks in the various places.
+
+ - CONFIG_SCST_ISCSI_DEBUG_DIGEST_FAILURES - simulates digest failures in
+ random places.
+
+Credits
+-------
+
+Thanks to:
+
+ * Ming Zhang <blackmagic02881@gmail.com> for fixes
+
+ * Krzysztof Blaszkowski <kb@sysmikro.com.pl> for many fixes
+
+ * Alexey Kuznetsov <kuznet@ms2.inr.ac.ru> for comments and help in
+ debugging
+
+ * Tomasz Chmielewski <mangoo@wpkg.org> for testing and suggestions
+
+ * Bart Van Assche <bart.vanassche@gmail.com> for a lot of help
+
+Vladislav Bolkhovitin <vst@vlnb.net>, http://scst.sourceforge.net
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2010-04-13 13:11 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <4BC44A49.7070307@vlnb.net>
2010-04-13 13:04 ` [PATCH][RFC 0/12/1/5] SCST core Vladislav Bolkhovitin
[not found] ` <4BC44D08.4060907@vlnb.net>
2010-04-13 13:04 ` [PATCH][RFC 1/12/1/5] SCST core's Makefile and Kconfig Vladislav Bolkhovitin
2010-04-13 13:04 ` [PATCH][RFC 2/12/1/5] SCST core's external headers Vladislav Bolkhovitin
2010-04-13 13:04 ` [PATCH][RFC 3/12/1/5] SCST core's scst_main.c Vladislav Bolkhovitin
2010-04-13 13:05 ` [PATCH][RFC 4/12/1/5] SCST core's scst_targ.c Vladislav Bolkhovitin
2010-04-13 13:05 ` [PATCH][RFC 5/12/1/5] SCST core's scst_lib.c Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 6/12/1/5] SCST core's private header Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 7/12/1/5] SCST SGV cache Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 8/12/1/5] SCST sysfs interface Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 9/12/1/5] SCST debugging support Vladislav Bolkhovitin
2010-04-13 13:06 ` [PATCH][RFC 10/12/1/5] SCST external modules support Vladislav Bolkhovitin
[not found] ` <4BC45841.2090707@vlnb.net>
2010-04-13 13:07 ` [PATCH][RFC 11/12/1/5] SCST core's README Vladislav Bolkhovitin
2010-04-13 13:07 ` [PATCH][RFC 12/12/1/5] SCST sysfs rules Vladislav Bolkhovitin
2010-04-13 13:09 ` [PATCH][RFC 0/4/4/5] iSCSI-SCST Vladislav Bolkhovitin
[not found] ` <4BC45AFA.7060403@vlnb.net>
2010-04-13 13:10 ` [PATCH][RFC 1/4/4/5] iSCSI-SCST's Makefile and Kconfig Vladislav Bolkhovitin
[not found] ` <4BC45E87.6000901@vlnb.net>
2010-04-13 13:10 ` [PATCH][RFC 2/4/4/5] iSCSI-SCST's header files Vladislav Bolkhovitin
[not found] ` <4BC45EF7.7010304@vlnb.net>
2010-04-13 13:10 ` [PATCH][RFC 3/4/4/5] iSCSI-SCST's implementation files Vladislav Bolkhovitin
2010-04-13 13:10 ` [PATCH][RFC 4/4/4/5] iSCSI-SCST's README file Vladislav Bolkhovitin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).