* [PATCH-v2 1/4] iscsi-target: Fix iscsit_transport reference leak during NP thread reset
2013-08-18 23:36 [PATCH-v2 0/4] iscsi-target: Add support for login multi-plexing support Nicholas A. Bellinger
@ 2013-08-18 23:36 ` Nicholas A. Bellinger
2013-08-18 23:36 ` [PATCH-v2 2/4] iscsi-target: Prepare login code for multi-plexing support Nicholas A. Bellinger
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Nicholas A. Bellinger @ 2013-08-18 23:36 UTC (permalink / raw)
To: target-devel; +Cc: linux-kernel, Nicholas Bellinger
From: Nicholas Bellinger <nab@linux-iscsi.org>
This patch fixes a bug in __iscsi_target_login_thread() where an explicit
network portal thread reset ends up leaking the iscsit_transport module
reference, along with the associated iscsi_conn allocation.
This manifests itself with iser-target where a NP reset causes the extra
iscsit_transport reference to be taken in iscsit_conn_set_transport()
during the reset, which prevents the ib_isert module from being unloaded
after the NP thread shutdown has finished.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
drivers/target/iscsi/iscsi_target_login.c | 9 ++++-----
1 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/drivers/target/iscsi/iscsi_target_login.c b/drivers/target/iscsi/iscsi_target_login.c
index 0e85238..4c17f83 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -1171,12 +1171,11 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
if (np->np_thread_state == ISCSI_NP_THREAD_RESET) {
spin_unlock_bh(&np->np_thread_lock);
complete(&np->np_restart_comp);
- if (ret == -ENODEV) {
- iscsit_put_transport(conn->conn_transport);
- kfree(conn);
- conn = NULL;
+ iscsit_put_transport(conn->conn_transport);
+ kfree(conn);
+ conn = NULL;
+ if (ret == -ENODEV)
goto out;
- }
/* Get another socket */
return 1;
}
--
1.7.2.5
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH-v2 2/4] iscsi-target: Prepare login code for multi-plexing support
2013-08-18 23:36 [PATCH-v2 0/4] iscsi-target: Add support for login multi-plexing support Nicholas A. Bellinger
2013-08-18 23:36 ` [PATCH-v2 1/4] iscsi-target: Fix iscsit_transport reference leak during NP thread reset Nicholas A. Bellinger
@ 2013-08-18 23:36 ` Nicholas A. Bellinger
2013-08-18 23:36 ` [PATCH-v2 3/4] iscsi-target: Add login negotiation " Nicholas A. Bellinger
2013-08-18 23:36 ` [PATCH 4/4] iser-target: Updates for " Nicholas A. Bellinger
3 siblings, 0 replies; 5+ messages in thread
From: Nicholas A. Bellinger @ 2013-08-18 23:36 UTC (permalink / raw)
To: target-devel; +Cc: linux-kernel, Nicholas Bellinger
From: Nicholas Bellinger <nab@linux-iscsi.org>
This patch prepares the iscsi-target login code for multi-plexing
support. This includes:
- Adding iscsi_tpg_np->tpg_np_kref + iscsit_login_kref_put() for
handling callback of iscsi_tpg_np->tpg_np_comp
- Adding kref_put() in iscsit_deaccess_np()
- Adding kref_put() and wait_for_completion() in
iscsit_reset_np_thread()
- Refactor login failure path release logic into
iscsi_target_login_sess_out()
- Update __iscsi_target_login_thread() to handle
iscsi_post_login_handler() asynchronous completion
- Add shutdown parameter for iscsit_clear_tpg_np_login_thread*()
v2 changes:
- Remove duplicate call to iscsi_post_login_handler() in
__iscsi_target_login_thread()
- Drop unused iscsi_np->np_login_tpg
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
drivers/target/iscsi/iscsi_target.c | 53 +++++-----
drivers/target/iscsi/iscsi_target.h | 6 +-
drivers/target/iscsi/iscsi_target_core.h | 11 ++-
drivers/target/iscsi/iscsi_target_login.c | 162 +++++++++++++++++------------
drivers/target/iscsi/iscsi_target_login.h | 3 +
drivers/target/iscsi/iscsi_target_tpg.c | 16 ++-
drivers/target/iscsi/iscsi_target_tpg.h | 2 +-
7 files changed, 149 insertions(+), 104 deletions(-)
diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index c4aeac3..7228cc7 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -220,11 +220,6 @@ int iscsit_access_np(struct iscsi_np *np, struct iscsi_portal_group *tpg)
spin_unlock_bh(&np->np_thread_lock);
return -1;
}
- if (np->np_login_tpg) {
- pr_err("np->np_login_tpg() is not NULL!\n");
- spin_unlock_bh(&np->np_thread_lock);
- return -1;
- }
spin_unlock_bh(&np->np_thread_lock);
/*
* Determine if the portal group is accepting storage traffic.
@@ -243,23 +238,35 @@ int iscsit_access_np(struct iscsi_np *np, struct iscsi_portal_group *tpg)
if ((ret != 0) || signal_pending(current))
return -1;
- spin_lock_bh(&np->np_thread_lock);
- np->np_login_tpg = tpg;
- spin_unlock_bh(&np->np_thread_lock);
+ spin_lock_bh(&tpg->tpg_state_lock);
+ if (tpg->tpg_state != TPG_STATE_ACTIVE) {
+ spin_unlock_bh(&tpg->tpg_state_lock);
+ mutex_unlock(&tpg->np_login_lock);
+ return -1;
+ }
+ spin_unlock_bh(&tpg->tpg_state_lock);
return 0;
}
-int iscsit_deaccess_np(struct iscsi_np *np, struct iscsi_portal_group *tpg)
+void iscsit_login_kref_put(struct kref *kref)
{
- struct iscsi_tiqn *tiqn = tpg->tpg_tiqn;
+ struct iscsi_tpg_np *tpg_np = container_of(kref,
+ struct iscsi_tpg_np, tpg_np_kref);
- spin_lock_bh(&np->np_thread_lock);
- np->np_login_tpg = NULL;
- spin_unlock_bh(&np->np_thread_lock);
+ complete(&tpg_np->tpg_np_comp);
+}
+
+int iscsit_deaccess_np(struct iscsi_np *np, struct iscsi_portal_group *tpg,
+ struct iscsi_tpg_np *tpg_np)
+{
+ struct iscsi_tiqn *tiqn = tpg->tpg_tiqn;
mutex_unlock(&tpg->np_login_lock);
+ if (tpg_np)
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);
+
if (tiqn)
iscsit_put_tiqn_for_login(tiqn);
@@ -410,20 +417,10 @@ struct iscsi_np *iscsit_add_np(
int iscsit_reset_np_thread(
struct iscsi_np *np,
struct iscsi_tpg_np *tpg_np,
- struct iscsi_portal_group *tpg)
+ struct iscsi_portal_group *tpg,
+ bool shutdown)
{
spin_lock_bh(&np->np_thread_lock);
- if (tpg && tpg_np) {
- /*
- * The reset operation need only be performed when the
- * passed struct iscsi_portal_group has a login in progress
- * to one of the network portals.
- */
- if (tpg_np->tpg_np->np_login_tpg != tpg) {
- spin_unlock_bh(&np->np_thread_lock);
- return 0;
- }
- }
if (np->np_thread_state == ISCSI_NP_THREAD_INACTIVE) {
spin_unlock_bh(&np->np_thread_lock);
return 0;
@@ -438,6 +435,12 @@ int iscsit_reset_np_thread(
}
spin_unlock_bh(&np->np_thread_lock);
+ if (tpg_np && shutdown) {
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);
+
+ wait_for_completion(&tpg_np->tpg_np_comp);
+ }
+
return 0;
}
diff --git a/drivers/target/iscsi/iscsi_target.h b/drivers/target/iscsi/iscsi_target.h
index 2c437cb..f82f627 100644
--- a/drivers/target/iscsi/iscsi_target.h
+++ b/drivers/target/iscsi/iscsi_target.h
@@ -7,13 +7,15 @@ extern void iscsit_put_tiqn_for_login(struct iscsi_tiqn *);
extern struct iscsi_tiqn *iscsit_add_tiqn(unsigned char *);
extern void iscsit_del_tiqn(struct iscsi_tiqn *);
extern int iscsit_access_np(struct iscsi_np *, struct iscsi_portal_group *);
-extern int iscsit_deaccess_np(struct iscsi_np *, struct iscsi_portal_group *);
+extern void iscsit_login_kref_put(struct kref *);
+extern int iscsit_deaccess_np(struct iscsi_np *, struct iscsi_portal_group *,
+ struct iscsi_tpg_np *);
extern bool iscsit_check_np_match(struct __kernel_sockaddr_storage *,
struct iscsi_np *, int);
extern struct iscsi_np *iscsit_add_np(struct __kernel_sockaddr_storage *,
char *, int);
extern int iscsit_reset_np_thread(struct iscsi_np *, struct iscsi_tpg_np *,
- struct iscsi_portal_group *);
+ struct iscsi_portal_group *, bool);
extern int iscsit_del_np(struct iscsi_np *);
extern int iscsit_reject_cmd(struct iscsi_cmd *cmd, u8, unsigned char *);
extern void iscsit_set_unsoliticed_dataout(struct iscsi_cmd *);
diff --git a/drivers/target/iscsi/iscsi_target_core.h b/drivers/target/iscsi/iscsi_target_core.h
index 4f77a78..711a028 100644
--- a/drivers/target/iscsi/iscsi_target_core.h
+++ b/drivers/target/iscsi/iscsi_target_core.h
@@ -554,6 +554,11 @@ struct iscsi_conn {
struct completion rx_half_close_comp;
/* socket used by this connection */
struct socket *sock;
+ void (*orig_data_ready)(struct sock *, int);
+#define LOGIN_FLAGS_READ_ACTIVE 1
+ unsigned long login_flags;
+ struct delayed_work login_work;
+ struct iscsi_login *login;
struct timer_list nopin_timer;
struct timer_list nopin_response_timer;
struct timer_list transport_timer;
@@ -584,6 +589,7 @@ struct iscsi_conn {
void *context;
struct iscsi_login_thread_s *login_thread;
struct iscsi_portal_group *tpg;
+ struct iscsi_tpg_np *tpg_np;
/* Pointer to parent session */
struct iscsi_session *sess;
/* Pointer to thread_set in use for this conn's threads */
@@ -682,6 +688,7 @@ struct iscsi_login {
u8 version_max;
u8 login_complete;
u8 login_failed;
+ bool zero_tsih;
char isid[6];
u32 cmd_sn;
itt_t init_task_tag;
@@ -694,6 +701,7 @@ struct iscsi_login {
char *req_buf;
char *rsp_buf;
struct iscsi_conn *conn;
+ struct iscsi_np *np;
} ____cacheline_aligned;
struct iscsi_node_attrib {
@@ -773,7 +781,6 @@ struct iscsi_np {
struct __kernel_sockaddr_storage np_sockaddr;
struct task_struct *np_thread;
struct timer_list np_login_timer;
- struct iscsi_portal_group *np_login_tpg;
void *np_context;
struct iscsit_transport *np_transport;
struct list_head np_list;
@@ -788,6 +795,8 @@ struct iscsi_tpg_np {
struct list_head tpg_np_parent_list;
struct se_tpg_np se_tpg_np;
spinlock_t tpg_np_parent_lock;
+ struct completion tpg_np_comp;
+ struct kref tpg_np_kref;
};
struct iscsi_portal_group {
diff --git a/drivers/target/iscsi/iscsi_target_login.c b/drivers/target/iscsi/iscsi_target_login.c
index 4c17f83..c29fffd 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -50,6 +50,7 @@ static struct iscsi_login *iscsi_login_init_conn(struct iscsi_conn *conn)
pr_err("Unable to allocate memory for struct iscsi_login.\n");
return NULL;
}
+ conn->login = login;
login->conn = conn;
login->first_request = 1;
@@ -684,7 +685,7 @@ static void iscsi_post_login_start_timers(struct iscsi_conn *conn)
iscsit_start_nopin_timer(conn);
}
-static int iscsi_post_login_handler(
+int iscsi_post_login_handler(
struct iscsi_np *np,
struct iscsi_conn *conn,
u8 zero_tsih)
@@ -1124,6 +1125,77 @@ iscsit_conn_set_transport(struct iscsi_conn *conn, struct iscsit_transport *t)
return 0;
}
+void iscsi_target_login_sess_out(struct iscsi_conn *conn,
+ struct iscsi_np *np, bool zero_tsih, bool new_sess)
+{
+ if (new_sess == false)
+ goto old_sess_out;
+
+ pr_err("iSCSI Login negotiation failed.\n");
+ iscsit_collect_login_stats(conn, ISCSI_STATUS_CLS_INITIATOR_ERR,
+ ISCSI_LOGIN_STATUS_INIT_ERR);
+ if (!zero_tsih || !conn->sess)
+ goto old_sess_out;
+ if (conn->sess->se_sess)
+ transport_free_session(conn->sess->se_sess);
+ if (conn->sess->session_index != 0) {
+ spin_lock_bh(&sess_idr_lock);
+ idr_remove(&sess_idr, conn->sess->session_index);
+ spin_unlock_bh(&sess_idr_lock);
+ }
+ kfree(conn->sess->sess_ops);
+ kfree(conn->sess);
+
+old_sess_out:
+ iscsi_stop_login_thread_timer(np);
+ /*
+ * If login negotiation fails check if the Time2Retain timer
+ * needs to be restarted.
+ */
+ if (!zero_tsih && conn->sess) {
+ spin_lock_bh(&conn->sess->conn_lock);
+ if (conn->sess->session_state == TARG_SESS_STATE_FAILED) {
+ struct se_portal_group *se_tpg =
+ &ISCSI_TPG_C(conn)->tpg_se_tpg;
+
+ atomic_set(&conn->sess->session_continuation, 0);
+ spin_unlock_bh(&conn->sess->conn_lock);
+ spin_lock_bh(&se_tpg->session_lock);
+ iscsit_start_time2retain_handler(conn->sess);
+ spin_unlock_bh(&se_tpg->session_lock);
+ } else
+ spin_unlock_bh(&conn->sess->conn_lock);
+ iscsit_dec_session_usage_count(conn->sess);
+ }
+
+ if (!IS_ERR(conn->conn_rx_hash.tfm))
+ crypto_free_hash(conn->conn_rx_hash.tfm);
+ if (!IS_ERR(conn->conn_tx_hash.tfm))
+ crypto_free_hash(conn->conn_tx_hash.tfm);
+
+ if (conn->conn_cpumask)
+ free_cpumask_var(conn->conn_cpumask);
+
+ kfree(conn->conn_ops);
+
+ if (conn->param_list) {
+ iscsi_release_param_list(conn->param_list);
+ conn->param_list = NULL;
+ }
+ iscsi_target_nego_release(conn);
+
+ if (conn->sock) {
+ sock_release(conn->sock);
+ conn->sock = NULL;
+ }
+
+ if (conn->conn_transport->iscsit_free_conn)
+ conn->conn_transport->iscsit_free_conn(conn);
+
+ iscsit_put_transport(conn->conn_transport);
+ kfree(conn);
+}
+
static int __iscsi_target_login_thread(struct iscsi_np *np)
{
u8 *buffer, zero_tsih = 0;
@@ -1132,6 +1204,8 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
struct iscsi_login *login;
struct iscsi_portal_group *tpg = NULL;
struct iscsi_login_req *pdu;
+ struct iscsi_tpg_np *tpg_np;
+ bool new_sess = false;
flush_signals(current);
@@ -1272,6 +1346,7 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
tpg = conn->tpg;
goto new_sess_out;
}
+ login->zero_tsih = zero_tsih;
tpg = conn->tpg;
if (!tpg) {
@@ -1287,7 +1362,8 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
goto old_sess_out;
}
- if (iscsi_target_start_negotiation(login, conn) < 0)
+ ret = iscsi_target_start_negotiation(login, conn);
+ if (ret < 0)
goto new_sess_out;
if (!conn->sess) {
@@ -1300,84 +1376,32 @@ static int __iscsi_target_login_thread(struct iscsi_np *np)
if (signal_pending(current))
goto new_sess_out;
- ret = iscsi_post_login_handler(np, conn, zero_tsih);
+ if (ret == 1) {
+ tpg_np = conn->tpg_np;
- if (ret < 0)
- goto new_sess_out;
+ ret = iscsi_post_login_handler(np, conn, zero_tsih);
+ if (ret < 0)
+ goto new_sess_out;
+
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ }
- iscsit_deaccess_np(np, tpg);
tpg = NULL;
+ tpg_np = NULL;
/* Get another socket */
return 1;
new_sess_out:
- pr_err("iSCSI Login negotiation failed.\n");
- iscsit_collect_login_stats(conn, ISCSI_STATUS_CLS_INITIATOR_ERR,
- ISCSI_LOGIN_STATUS_INIT_ERR);
- if (!zero_tsih || !conn->sess)
- goto old_sess_out;
- if (conn->sess->se_sess)
- transport_free_session(conn->sess->se_sess);
- if (conn->sess->session_index != 0) {
- spin_lock_bh(&sess_idr_lock);
- idr_remove(&sess_idr, conn->sess->session_index);
- spin_unlock_bh(&sess_idr_lock);
- }
- kfree(conn->sess->sess_ops);
- kfree(conn->sess);
+ new_sess = true;
old_sess_out:
- iscsi_stop_login_thread_timer(np);
- /*
- * If login negotiation fails check if the Time2Retain timer
- * needs to be restarted.
- */
- if (!zero_tsih && conn->sess) {
- spin_lock_bh(&conn->sess->conn_lock);
- if (conn->sess->session_state == TARG_SESS_STATE_FAILED) {
- struct se_portal_group *se_tpg =
- &ISCSI_TPG_C(conn)->tpg_se_tpg;
-
- atomic_set(&conn->sess->session_continuation, 0);
- spin_unlock_bh(&conn->sess->conn_lock);
- spin_lock_bh(&se_tpg->session_lock);
- iscsit_start_time2retain_handler(conn->sess);
- spin_unlock_bh(&se_tpg->session_lock);
- } else
- spin_unlock_bh(&conn->sess->conn_lock);
- iscsit_dec_session_usage_count(conn->sess);
- }
-
- if (!IS_ERR(conn->conn_rx_hash.tfm))
- crypto_free_hash(conn->conn_rx_hash.tfm);
- if (!IS_ERR(conn->conn_tx_hash.tfm))
- crypto_free_hash(conn->conn_tx_hash.tfm);
-
- if (conn->conn_cpumask)
- free_cpumask_var(conn->conn_cpumask);
-
- kfree(conn->conn_ops);
-
- if (conn->param_list) {
- iscsi_release_param_list(conn->param_list);
- conn->param_list = NULL;
- }
- iscsi_target_nego_release(conn);
-
- if (conn->sock) {
- sock_release(conn->sock);
- conn->sock = NULL;
- }
-
- if (conn->conn_transport->iscsit_free_conn)
- conn->conn_transport->iscsit_free_conn(conn);
-
- iscsit_put_transport(conn->conn_transport);
-
- kfree(conn);
+ tpg_np = conn->tpg_np;
+ iscsi_target_login_sess_out(conn, np, zero_tsih, new_sess);
+ new_sess = false;
if (tpg) {
- iscsit_deaccess_np(np, tpg);
+ iscsit_deaccess_np(np, tpg, tpg_np);
tpg = NULL;
+ tpg_np = NULL;
}
out:
diff --git a/drivers/target/iscsi/iscsi_target_login.h b/drivers/target/iscsi/iscsi_target_login.h
index 63efd28..29d0983 100644
--- a/drivers/target/iscsi/iscsi_target_login.h
+++ b/drivers/target/iscsi/iscsi_target_login.h
@@ -12,6 +12,9 @@ extern int iscsit_accept_np(struct iscsi_np *, struct iscsi_conn *);
extern int iscsit_get_login_rx(struct iscsi_conn *, struct iscsi_login *);
extern int iscsit_put_login_tx(struct iscsi_conn *, struct iscsi_login *, u32);
extern void iscsit_free_conn(struct iscsi_np *, struct iscsi_conn *);
+extern int iscsi_post_login_handler(struct iscsi_np *, struct iscsi_conn *, u8);
+extern void iscsi_target_login_sess_out(struct iscsi_conn *, struct iscsi_np *,
+ bool, bool);
extern int iscsi_target_login_thread(void *);
extern int iscsi_login_disable_FIM_keys(struct iscsi_param_list *, struct iscsi_conn *);
diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index 439260b..c1b1067 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -175,18 +175,20 @@ void iscsit_put_tpg(struct iscsi_portal_group *tpg)
static void iscsit_clear_tpg_np_login_thread(
struct iscsi_tpg_np *tpg_np,
- struct iscsi_portal_group *tpg)
+ struct iscsi_portal_group *tpg,
+ bool shutdown)
{
if (!tpg_np->tpg_np) {
pr_err("struct iscsi_tpg_np->tpg_np is NULL!\n");
return;
}
- iscsit_reset_np_thread(tpg_np->tpg_np, tpg_np, tpg);
+ iscsit_reset_np_thread(tpg_np->tpg_np, tpg_np, tpg, shutdown);
}
void iscsit_clear_tpg_np_login_threads(
- struct iscsi_portal_group *tpg)
+ struct iscsi_portal_group *tpg,
+ bool shutdown)
{
struct iscsi_tpg_np *tpg_np;
@@ -197,7 +199,7 @@ void iscsit_clear_tpg_np_login_threads(
continue;
}
spin_unlock(&tpg->tpg_np_lock);
- iscsit_clear_tpg_np_login_thread(tpg_np, tpg);
+ iscsit_clear_tpg_np_login_thread(tpg_np, tpg, shutdown);
spin_lock(&tpg->tpg_np_lock);
}
spin_unlock(&tpg->tpg_np_lock);
@@ -268,6 +270,8 @@ int iscsit_tpg_del_portal_group(
tpg->tpg_state = TPG_STATE_INACTIVE;
spin_unlock(&tpg->tpg_state_lock);
+ iscsit_clear_tpg_np_login_threads(tpg, true);
+
if (iscsit_release_sessions_for_tpg(tpg, force) < 0) {
pr_err("Unable to delete iSCSI Target Portal Group:"
" %hu while active sessions exist, and force=0\n",
@@ -368,7 +372,7 @@ int iscsit_tpg_disable_portal_group(struct iscsi_portal_group *tpg, int force)
tpg->tpg_state = TPG_STATE_INACTIVE;
spin_unlock(&tpg->tpg_state_lock);
- iscsit_clear_tpg_np_login_threads(tpg);
+ iscsit_clear_tpg_np_login_threads(tpg, false);
if (iscsit_release_sessions_for_tpg(tpg, force) < 0) {
spin_lock(&tpg->tpg_state_lock);
@@ -520,7 +524,7 @@ static int iscsit_tpg_release_np(
struct iscsi_portal_group *tpg,
struct iscsi_np *np)
{
- iscsit_clear_tpg_np_login_thread(tpg_np, tpg);
+ iscsit_clear_tpg_np_login_thread(tpg_np, tpg, true);
pr_debug("CORE[%s] - Removed Network Portal: %s:%hu,%hu on %s\n",
tpg->tpg_tiqn->tiqn, np->np_ip, np->np_port, tpg->tpgt,
diff --git a/drivers/target/iscsi/iscsi_target_tpg.h b/drivers/target/iscsi/iscsi_target_tpg.h
index dda48c1..60ef4a9 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.h
+++ b/drivers/target/iscsi/iscsi_target_tpg.h
@@ -8,7 +8,7 @@ extern struct iscsi_portal_group *iscsit_get_tpg_from_np(struct iscsi_tiqn *,
struct iscsi_np *);
extern int iscsit_get_tpg(struct iscsi_portal_group *);
extern void iscsit_put_tpg(struct iscsi_portal_group *);
-extern void iscsit_clear_tpg_np_login_threads(struct iscsi_portal_group *);
+extern void iscsit_clear_tpg_np_login_threads(struct iscsi_portal_group *, bool);
extern void iscsit_tpg_dump_params(struct iscsi_portal_group *);
extern int iscsit_tpg_add_portal_group(struct iscsi_tiqn *, struct iscsi_portal_group *);
extern int iscsit_tpg_del_portal_group(struct iscsi_tiqn *, struct iscsi_portal_group *,
--
1.7.2.5
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH-v2 3/4] iscsi-target: Add login negotiation multi-plexing support
2013-08-18 23:36 [PATCH-v2 0/4] iscsi-target: Add support for login multi-plexing support Nicholas A. Bellinger
2013-08-18 23:36 ` [PATCH-v2 1/4] iscsi-target: Fix iscsit_transport reference leak during NP thread reset Nicholas A. Bellinger
2013-08-18 23:36 ` [PATCH-v2 2/4] iscsi-target: Prepare login code for multi-plexing support Nicholas A. Bellinger
@ 2013-08-18 23:36 ` Nicholas A. Bellinger
2013-08-18 23:36 ` [PATCH 4/4] iser-target: Updates for " Nicholas A. Bellinger
3 siblings, 0 replies; 5+ messages in thread
From: Nicholas A. Bellinger @ 2013-08-18 23:36 UTC (permalink / raw)
To: target-devel; +Cc: linux-kernel, Nicholas Bellinger
From: Nicholas Bellinger <nab@linux-iscsi.org>
This patch adds support for login negotiation multi-plexing in
iscsi-target code.
This involves handling the first login request PDU + payload and
login response PDU + payload within __iscsi_target_login_thread()
process context, and then changing struct sock->sk_data_ready()
so that all subsequent exchanges are handled by workqueue process
context, to allow other incoming login requests to be received
in parallel by __iscsi_target_login_thread().
Upon login negotiation completion (or failure), ->sk_data_ready()
is replaced with the original kernel sockets handler saved in
iscsi_conn->orig_data_ready.
v2 changes:
- Add login_timer in iscsi_target_do_login_rx() to avoid
possible endless sleep with MSG_WAITALL for traditional
iscsi-target in certain network configurations.
- Convert lprintk() -> pr_debug()
- Remove forward declarations of iscsi_target_set_sock_callbacks(),
iscsi_target_restore_sock_callbacks() and iscsi_target_sk_data_ready()
- Make iscsi_target_set_sock_callbacks + iscsi_target_restore_sock_callbacks()
static (Fengguang)
- Make iscsi_target_do_login_rx() safe for iser-target w/o conn->sock
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
drivers/target/iscsi/iscsi_target_core.h | 1 +
drivers/target/iscsi/iscsi_target_nego.c | 209 ++++++++++++++++++++++++++++-
drivers/target/iscsi/iscsi_target_tpg.c | 7 +-
drivers/target/iscsi/iscsi_target_tpg.h | 2 +-
4 files changed, 209 insertions(+), 10 deletions(-)
diff --git a/drivers/target/iscsi/iscsi_target_core.h b/drivers/target/iscsi/iscsi_target_core.h
index 711a028..8a4c32d 100644
--- a/drivers/target/iscsi/iscsi_target_core.h
+++ b/drivers/target/iscsi/iscsi_target_core.h
@@ -562,6 +562,7 @@ struct iscsi_conn {
struct timer_list nopin_timer;
struct timer_list nopin_response_timer;
struct timer_list transport_timer;
+ struct task_struct *login_kworker;
/* Spinlock used for add/deleting cmd's from conn_cmd_list */
spinlock_t cmd_lock;
spinlock_t conn_usage_lock;
diff --git a/drivers/target/iscsi/iscsi_target_nego.c b/drivers/target/iscsi/iscsi_target_nego.c
index c4675b4..daebe32 100644
--- a/drivers/target/iscsi/iscsi_target_nego.c
+++ b/drivers/target/iscsi/iscsi_target_nego.c
@@ -377,14 +377,191 @@ static int iscsi_target_do_tx_login_io(struct iscsi_conn *conn, struct iscsi_log
return 0;
}
+static void iscsi_target_sk_data_ready(struct sock *sk, int count)
+{
+ struct iscsi_conn *conn = sk->sk_user_data;
+ bool rc;
+
+ pr_debug("Entering iscsi_target_sk_data_ready: conn: %p\n", conn);
+
+ read_lock_bh(&sk->sk_callback_lock);
+ if (!sk->sk_user_data) {
+ read_unlock_bh(&sk->sk_callback_lock);
+ return;
+ }
+
+ if (test_and_set_bit(LOGIN_FLAGS_READ_ACTIVE, &conn->login_flags)) {
+ read_unlock_bh(&sk->sk_callback_lock);
+ pr_debug("Got LOGIN_FLAGS_READ_ACTIVE=1, conn: %p >>>>\n", conn);
+ return;
+ }
+
+ rc = schedule_delayed_work(&conn->login_work, 0);
+ if (rc == false) {
+ pr_debug("iscsi_target_sk_data_ready, schedule_delayed_work"
+ " got false\n");
+ }
+ read_unlock_bh(&sk->sk_callback_lock);
+}
+
+static void iscsi_target_set_sock_callbacks(struct iscsi_conn *conn)
+{
+ struct sock *sk;
+
+ if (!conn->sock)
+ return;
+
+ sk = conn->sock->sk;
+ pr_debug("Entering iscsi_target_set_sock_callbacks: conn: %p\n", conn);
+
+ write_lock_bh(&sk->sk_callback_lock);
+ sk->sk_user_data = conn;
+ conn->orig_data_ready = sk->sk_data_ready;
+ sk->sk_data_ready = iscsi_target_sk_data_ready;
+ write_unlock_bh(&sk->sk_callback_lock);
+}
+
+static void iscsi_target_restore_sock_callbacks(struct iscsi_conn *conn)
+{
+ struct sock *sk;
+
+ if (!conn->sock)
+ return;
+
+ sk = conn->sock->sk;
+ pr_debug("Entering iscsi_target_restore_sock_callbacks: conn: %p\n", conn);
+
+ write_lock_bh(&sk->sk_callback_lock);
+ if (!sk->sk_user_data) {
+ write_unlock_bh(&sk->sk_callback_lock);
+ return;
+ }
+ sk->sk_user_data = NULL;
+ sk->sk_data_ready = conn->orig_data_ready;
+ write_unlock_bh(&sk->sk_callback_lock);
+}
+
+static int iscsi_target_do_login(struct iscsi_conn *, struct iscsi_login *);
+
+static bool iscsi_target_sk_state_check(struct sock *sk)
+{
+ if (sk->sk_state == TCP_CLOSE_WAIT || sk->sk_state == TCP_CLOSE) {
+ pr_debug("iscsi_target_sk_state_check: TCP_CLOSE_WAIT|TCP_CLOSE,"
+ "returning FALSE\n");
+ return false;
+ }
+ return true;
+}
+
+static void iscsi_target_login_drop(struct iscsi_conn *conn, struct iscsi_login *login)
+{
+ struct iscsi_np *np = login->np;
+ bool zero_tsih = login->zero_tsih;
+
+ iscsi_remove_failed_auth_entry(conn);
+ iscsi_target_nego_release(conn);
+ iscsi_target_login_sess_out(conn, np, zero_tsih, true);
+}
+
+static void iscsi_target_login_timeout(unsigned long data)
+{
+ struct iscsi_conn *conn = (struct iscsi_conn *)data;
+
+ pr_debug("Entering iscsi_target_login_timeout >>>>>>>>>>>>>>>>>>>\n");
+
+ if (conn->login_kworker) {
+ pr_debug("Sending SIGINT to conn->login_kworker %s/%d\n",
+ conn->login_kworker->comm, conn->login_kworker->pid);
+ send_sig(SIGINT, conn->login_kworker, 1);
+ }
+}
+
+static void iscsi_target_do_login_rx(struct work_struct *work)
+{
+ struct iscsi_conn *conn = container_of(work,
+ struct iscsi_conn, login_work.work);
+ struct iscsi_login *login = conn->login;
+ struct iscsi_np *np = login->np;
+ struct iscsi_portal_group *tpg = conn->tpg;
+ struct iscsi_tpg_np *tpg_np = conn->tpg_np;
+ struct timer_list login_timer;
+ int rc, zero_tsih = login->zero_tsih;
+ bool state;
+
+ pr_debug("entering iscsi_target_do_login_rx, conn: %p, %s:%d\n",
+ conn, current->comm, current->pid);
+
+ spin_lock(&tpg->tpg_state_lock);
+ state = (tpg->tpg_state == TPG_STATE_ACTIVE);
+ spin_unlock(&tpg->tpg_state_lock);
+
+ if (state == false) {
+ pr_debug("iscsi_target_do_login_rx: tpg_state != TPG_STATE_ACTIVE\n");
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ return;
+ }
+
+ if (conn->sock) {
+ struct sock *sk = conn->sock->sk;
+
+ read_lock_bh(&sk->sk_callback_lock);
+ state = iscsi_target_sk_state_check(sk);
+ read_unlock_bh(&sk->sk_callback_lock);
+
+ if (state == false) {
+ pr_debug("iscsi_target_do_login_rx, TCP state CLOSE\n");
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ return;
+ }
+ }
+
+ conn->login_kworker = current;
+ allow_signal(SIGINT);
+
+ init_timer(&login_timer);
+ login_timer.expires = (get_jiffies_64() + TA_LOGIN_TIMEOUT * HZ);
+ login_timer.data = (unsigned long)conn;
+ login_timer.function = iscsi_target_login_timeout;
+ add_timer(&login_timer);
+ pr_debug("Starting login_timer for %s/%d\n", current->comm, current->pid);
+
+ rc = conn->conn_transport->iscsit_get_login_rx(conn, login);
+ del_timer_sync(&login_timer);
+ flush_signals(current);
+ conn->login_kworker = NULL;
+
+ if (rc < 0) {
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ return;
+ }
+
+ pr_debug("iscsi_target_do_login_rx after rx_login_io, %p, %s:%d\n",
+ conn, current->comm, current->pid);
+
+ rc = iscsi_target_do_login(conn, login);
+ clear_bit(LOGIN_FLAGS_READ_ACTIVE, &conn->login_flags);
+ if (rc < 0) {
+ iscsi_target_restore_sock_callbacks(conn);
+ iscsi_target_login_drop(conn, login);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ } else if (rc == 1) {
+ iscsi_target_nego_release(conn);
+ iscsi_post_login_handler(np, conn, zero_tsih);
+ iscsit_deaccess_np(np, tpg, tpg_np);
+ }
+}
+
static int iscsi_target_do_login_io(struct iscsi_conn *conn, struct iscsi_login *login)
{
if (iscsi_target_do_tx_login_io(conn, login) < 0)
return -1;
- if (conn->conn_transport->iscsit_get_login_rx(conn, login) < 0)
- return -1;
-
return 0;
}
@@ -643,10 +820,11 @@ static int iscsi_target_do_login(struct iscsi_conn *conn, struct iscsi_login *lo
if (login_rsp->flags & ISCSI_FLAG_LOGIN_TRANSIT) {
login->tsih = conn->sess->tsih;
login->login_complete = 1;
+ iscsi_target_restore_sock_callbacks(conn);
if (iscsi_target_do_tx_login_io(conn,
login) < 0)
return -1;
- return 0;
+ return 1;
}
break;
default:
@@ -663,6 +841,7 @@ static int iscsi_target_do_login(struct iscsi_conn *conn, struct iscsi_login *lo
login_rsp->flags &= ~ISCSI_FLAG_LOGIN_TRANSIT;
login_rsp->flags &= ~ISCSI_FLAG_LOGIN_NEXT_STAGE_MASK;
}
+ break;
}
return 0;
@@ -695,10 +874,13 @@ int iscsi_target_locate_portal(
char *tmpbuf, *start = NULL, *end = NULL, *key, *value;
struct iscsi_session *sess = conn->sess;
struct iscsi_tiqn *tiqn;
+ struct iscsi_tpg_np *tpg_np = NULL;
struct iscsi_login_req *login_req;
u32 payload_length;
int sessiontype = 0, ret = 0;
+ login->np = np;
+
login_req = (struct iscsi_login_req *) login->req;
payload_length = ntoh24(login_req->dlength);
@@ -822,7 +1004,7 @@ get_target:
/*
* Locate Target Portal Group from Storage Node.
*/
- conn->tpg = iscsit_get_tpg_from_np(tiqn, np);
+ conn->tpg = iscsit_get_tpg_from_np(tiqn, np, &tpg_np);
if (!conn->tpg) {
pr_err("Unable to locate Target Portal Group"
" on %s\n", tiqn->tiqn);
@@ -832,12 +1014,16 @@ get_target:
ret = -1;
goto out;
}
+ conn->tpg_np = tpg_np;
pr_debug("Located Portal Group Object: %hu\n", conn->tpg->tpgt);
/*
* Setup crc32c modules from libcrypto
*/
if (iscsi_login_setup_crypto(conn) < 0) {
pr_err("iscsi_login_setup_crypto() failed\n");
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);
+ iscsit_put_tiqn_for_login(tiqn);
+ conn->tpg = NULL;
ret = -1;
goto out;
}
@@ -846,11 +1032,12 @@ get_target:
* process login attempt.
*/
if (iscsit_access_np(np, conn->tpg) < 0) {
+ kref_put(&tpg_np->tpg_np_kref, iscsit_login_kref_put);
iscsit_put_tiqn_for_login(tiqn);
iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
ISCSI_LOGIN_STATUS_SVC_UNAVAILABLE);
- ret = -1;
conn->tpg = NULL;
+ ret = -1;
goto out;
}
@@ -896,11 +1083,17 @@ int iscsi_target_start_negotiation(
{
int ret;
+ INIT_DELAYED_WORK(&conn->login_work, iscsi_target_do_login_rx);
+ iscsi_target_set_sock_callbacks(conn);
+
ret = iscsi_target_do_login(conn, login);
- if (ret != 0)
+ if (ret < 0) {
+ iscsi_target_restore_sock_callbacks(conn);
iscsi_remove_failed_auth_entry(conn);
+ }
+ if (ret != 0)
+ iscsi_target_nego_release(conn);
- iscsi_target_nego_release(conn);
return ret;
}
diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index c1b1067..6beed70 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -129,7 +129,8 @@ void iscsit_release_discovery_tpg(void)
struct iscsi_portal_group *iscsit_get_tpg_from_np(
struct iscsi_tiqn *tiqn,
- struct iscsi_np *np)
+ struct iscsi_np *np,
+ struct iscsi_tpg_np **tpg_np_out)
{
struct iscsi_portal_group *tpg = NULL;
struct iscsi_tpg_np *tpg_np;
@@ -147,6 +148,8 @@ struct iscsi_portal_group *iscsit_get_tpg_from_np(
spin_lock(&tpg->tpg_np_lock);
list_for_each_entry(tpg_np, &tpg->tpg_gnp_list, tpg_np_list) {
if (tpg_np->tpg_np == np) {
+ *tpg_np_out = tpg_np;
+ kref_get(&tpg_np->tpg_np_kref);
spin_unlock(&tpg->tpg_np_lock);
spin_unlock(&tiqn->tiqn_tpg_lock);
return tpg;
@@ -494,6 +497,8 @@ struct iscsi_tpg_np *iscsit_tpg_add_network_portal(
INIT_LIST_HEAD(&tpg_np->tpg_np_child_list);
INIT_LIST_HEAD(&tpg_np->tpg_np_parent_list);
spin_lock_init(&tpg_np->tpg_np_parent_lock);
+ init_completion(&tpg_np->tpg_np_comp);
+ kref_init(&tpg_np->tpg_np_kref);
tpg_np->tpg_np = np;
tpg_np->tpg = tpg;
diff --git a/drivers/target/iscsi/iscsi_target_tpg.h b/drivers/target/iscsi/iscsi_target_tpg.h
index 60ef4a9..b77693e 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.h
+++ b/drivers/target/iscsi/iscsi_target_tpg.h
@@ -5,7 +5,7 @@ extern struct iscsi_portal_group *iscsit_alloc_portal_group(struct iscsi_tiqn *,
extern int iscsit_load_discovery_tpg(void);
extern void iscsit_release_discovery_tpg(void);
extern struct iscsi_portal_group *iscsit_get_tpg_from_np(struct iscsi_tiqn *,
- struct iscsi_np *);
+ struct iscsi_np *, struct iscsi_tpg_np **);
extern int iscsit_get_tpg(struct iscsi_portal_group *);
extern void iscsit_put_tpg(struct iscsi_portal_group *);
extern void iscsit_clear_tpg_np_login_threads(struct iscsi_portal_group *, bool);
--
1.7.2.5
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 4/4] iser-target: Updates for login negotiation multi-plexing support
2013-08-18 23:36 [PATCH-v2 0/4] iscsi-target: Add support for login multi-plexing support Nicholas A. Bellinger
` (2 preceding siblings ...)
2013-08-18 23:36 ` [PATCH-v2 3/4] iscsi-target: Add login negotiation " Nicholas A. Bellinger
@ 2013-08-18 23:36 ` Nicholas A. Bellinger
3 siblings, 0 replies; 5+ messages in thread
From: Nicholas A. Bellinger @ 2013-08-18 23:36 UTC (permalink / raw)
To: target-devel; +Cc: linux-kernel, Nicholas Bellinger
From: Nicholas Bellinger <nab@linux-iscsi.org>
This patch updates iser-target code to support login negotiation
multi-plexing. This includes only using isert_conn->conn_login_comp
for the first login request PDU, pushing the subsequent processing
to iscsi_conn->login_work -> iscsi_target_do_login_rx(), and turning
isert_get_login_rx() into a NOP.
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
drivers/infiniband/ulp/isert/ib_isert.c | 17 ++++++++++++++++-
1 files changed, 16 insertions(+), 1 deletions(-)
diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
index 3f62041..d17ff13 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -869,7 +869,14 @@ isert_rx_login_req(struct iser_rx_desc *rx_desc, int rx_buflen,
size, rx_buflen, MAX_KEY_VALUE_PAIRS);
memcpy(login->req_buf, &rx_desc->data[0], size);
- complete(&isert_conn->conn_login_comp);
+ if (login->first_request) {
+ complete(&isert_conn->conn_login_comp);
+ return;
+ }
+ if (test_and_set_bit(LOGIN_FLAGS_READ_ACTIVE, &conn->login_flags))
+ return;
+
+ schedule_delayed_work(&conn->login_work, 0);
}
static void
@@ -2224,6 +2231,14 @@ isert_get_login_rx(struct iscsi_conn *conn, struct iscsi_login *login)
int ret;
pr_debug("isert_get_login_rx before conn_login_comp conn: %p\n", conn);
+ /*
+ * For login requests after the first PDU, isert_rx_login_req() will
+ * kick schedule_delayed_work(&conn->login_work) as the packet is
+ * received, which turns this callback from iscsi_target_do_login_rx()
+ * into a NOP.
+ */
+ if (!login->first_request)
+ return 0;
ret = wait_for_completion_interruptible(&isert_conn->conn_login_comp);
if (ret)
--
1.7.2.5
^ permalink raw reply related [flat|nested] 5+ messages in thread