[ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Nicholas A. Bellinger]

The following is the first public release of the PyX Technologies iSCSI
Initiator Core Stack v1.6.1.20 for Linux 2.6.11-rc4.  This is a full
featured iSCSI Initiator stack that is capable of mulitplexing coast to
coast across multiple independant backbone providers using various
network transports _TODAY_.  This is the first release of the core stack
and assoicated userspace tools, the accompanying authentication daemon
will be released shortly.

There is ongoing work in taking advantage of the very latest 2.6
kernel's SCSI functionality, namely drivers/scsi/scsi_transport_iscsi.c.
The near term work for the iSCSI Initiator Core Stack includes taking
advantage of these new abstractions in scsi_transport_scsi.c in a manner
to reduce code duplication across the iSCSI Initiator Core Stack and
linux-sfnet implementations.  An initial patch to scsi_transport_scsi.c
will be provided in order stay in line with the iSCSI Session/Connection
contexts definied in section 12 of RFC 3720.  Additionally this patch
will add initial support for iSCSI Extentions for RDMA (iSER)
optertional keys in order to help facilitate development of iSCSI with
RDMA Capable Protocols (RCP) in the Linux 2.6 environment.

The patch against 2.6.11-rc4 in diff format can be found at:

http://www.kernel.org/pub/linux/kernel/people/nab/iscsi-initiator-core

The assoicated userspace tools and documentation:

http://www.kernel.org/pub/linux/utils/storage/iscsi

For more information on the Internet Small Computer Systems Interface
(iSCSI) standard, please see:

http://www.ietf.org/rfc/rfc3720.txt

The iSCSI Initiator Core Stack contains, but is not limited to the
following features:

1) Full support for Internet Small Computer Systems Interface (iSCSI)
state machine as defined by RFC 3720.
2) Full support for ErrorRecoveryLevel=0 feature set as defined for RFC
3720.
3) Full support for Multiple Connections per Session that can be brought
up and down on the fly.
4) Full support for DataPDUInOrder=No, DataSequenceInOrder=No,
MaxOutstandingR2T>1.
5) Full support for Sync and Steering using Fixed Interval Markers. (See
RFC 3720 Appendix A)
6) Full support for TCP and SCTP network transports.
7) Robust set of iSCSI Channel management options for iSCSI SAN
Administrators.
8) Tested on UP/SMP 32/64-bit Big/Little Endian architectures.


-- 
Nicholas A. Bellinger <nick@pyxtechnologies.com>
Chief Architect, PyX Technologies, Inc.

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Jeff Garzik]

Comments from an initial scan of the code.  This does not include any 
review of iSCSI interaction itself.


1) the TRACE stuff uses too much stack space


2) style:  way too many interal headers; feel free to disagree, though, 
this is maintainer's preference.


3) non-standard function definitions:

+extern void iscsi_non_scsi_queue_cmd (
+       iscsi_cmd_t *cmd,
+       iscsi_session_t *sess)
+{

Use the standard style instead.


4) My initial reaction upon reading the following code is, "where is the 
document describing this mess of locking?"

+       if (atomic_read(&sess->nconn) == 1) {
+               up(&sess->schedule_conn->tx_sem);
+               spin_unlock_irqrestore(&sess->conn_schedule_lock, flags);
+               return;
+       }

Presumably, sess->nconn need not be atomic?

In practice, I have found that use of atomic_t often _creates_ races.


5) style:  these function headers are completely pointless, conveying no 
additional insight or information:

+/*     iscsi_add_nopout_noqueue():
+ *
+ *
+ */


6) This is very worrisome, at the beginning of your queuecommand hook:

+       /*
+       * Get the assoicated iSCSI Channel for this active SCSI Task.
+       */
+       spin_unlock_irq(sc->device->host->host_lock);


It's fairly dangerous to assume that this will work, if, e.g. 
->queuecommand() is called from BH context.

Also, it means you are accessing the internal LUN list without any 
locking at all.


7) leak on error:

+       if (!(cmd = iscsi_allocate_cmd(sess, NULL)))
+               return(-1);
+
+       if ((padding = ((text_len) & 3)) != 0) {
+               TRACE(TRACE_ISCSI, "Attaching %u additional bytes for"
+                       " padding.\n", padding);
+       }
+
+       if (!(text = (unsigned char *) kmalloc((text_len + padding), 
GFP_ATOMIC)
)) {
+               TRACE_ERROR("Unable to allocate memory for outgoing 
text\n");
+               return(-1);
+       }



8) iscsi_build_scsi_cmd() appears to leak cmd->buf_ptr on error, but I 
could be wrong


9) style:  Linux kernel style discourages use of "foo_t", and prefers 
"struct foo"


10) iscsi_build_dataout() leak on error:

+               if (!(pdu = iscsi_get_pdu_holder_from_r2t(cmd, r2t)))
+                       return(-1);
...
+               if ((iov_ret = iscsi_map_scatterlists((void *)&map_sg,
+                                       (void *)unmap_sg)) < 0)
+                       return(-1);


11) excessive stack usage.  will cause crash with 4K stacks:

+       unsigned char ping_data[ISCSI_MAX_PING_DATA_BYTES];


12) general comment:  It is damned difficult to figure out which context 
these functions are operating in.  In iscsi_build_text_req() I see you 
grab a spinlock with spin_lock().  May we presume that local_irq_save() 
is active?  Have you audited this code for ABBA deadlocks?


13) delete all of the following #ifdefs:

+#ifdef CRYPTO_API_CRC32C

and replace with a requirement in Kconfig.


14) Many instances where iscsi_find_cmd_from_itt_or_dump() returns a 
command, and then function returns on error without releasing cmd.

This -may- be a leak on error, maybe not.


15) locking bug: conn->state_lock is locked with spin_lock() in process 
context, spin_lock_bh() in other contexts.


16) stack usage:

+extern int iscsi_initiator_rx_thread (void *arg)
+{
+       int ret;
+       u8 buffer[ISCSI_HDR_LEN], opcode;


17) potential race in iscsi_create_connection():

+       if ((atomic_read(&sess->nconn) + 
atomic_read(&c->connection_count)) >


18) excessive stack usage:

+extern int iscsi_free_session (iscsi_session_t *sess)
+{
+       int i = 0, j = 0;
+       iscsi_conn_t *conn, *conn_next = NULL;
+       iscsi_conn_t *conn_array[ISCSI_MAX_CONNECTIONS];


19) ditto:

+extern int iscsi_stop_session (iscsi_session_t *sess, int 
session_sleep, int co
nnection_sleep)
+{
+       int i = 0, j = 0;
+       iscsi_conn_t *conn, *conn_next = NULL;
+       iscsi_conn_t *conn_array[ISCSI_MAX_CONNECTIONS];



20) numerous code bloat, by checking for NULL before calling kfree():

+               if (c)
+                       kfree(c);

kfree(NULL) is ok.


21) delete FULL_PARAMS_COUNT constant, use ARRAY_SIZE() macro in 
linux/kernel.h


22) excessive stack usage:

+static int iscsi_setup_full_params (iscsi_channel_t *c)
+{
+       unsigned char buf[512];


23) ditto:

+extern int iscsi_init_channel (iscsi_channel_t *channel, u32 
max_sectors, int f
ull_init)
+{
+       unsigned char buf[512];



I stopped reading at this point.

In general,

a) locking is completely unreviewable, incomprehensible and 
unmaintainable by anyone except the author(s).  Either a locking 
rewrite, or a locking proof document, is needed.

b) aside from locking, stack usage, and leaks on error, the code seems 
reasonably clean.

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Nicholas A. Bellinger]

Hi Jeff,

On Wed, 2005-02-23 at 16:03 -0500, Jeff Garzik wrote:
> Comments from an initial scan of the code.  This does not include any 
> review of iSCSI interaction itself.
> 
> 
> 1) the TRACE stuff uses too much stack space


Aside from TRACE_ERROR, the other TRACE_**** defs are not compiled in
per default and not intended for production use. (This is set with
DEBUG_ISCSI=1 in drivers/scsi/iscsi-initiator-core/Makefile).  I will
also be sure to add this as a Kconfig item.  As far as reducing stack
usage in the DEBUG_ISCSI=0 case, what do you recommend?

> 
> 2) style:  way too many interal headers; feel free to disagree, though, 
> this is maintainer's preference.
> 

I will look into thinning this out a bit.

> 
> 3) non-standard function definitions:
> 
> +extern void iscsi_non_scsi_queue_cmd (
> +       iscsi_cmd_t *cmd,
> +       iscsi_session_t *sess)
> +{
> 
> Use the standard style instead.
> 

Ok.

> 
> 4) My initial reaction upon reading the following code is, "where is the 
> document describing this mess of locking?"
> 
> +       if (atomic_read(&sess->nconn) == 1) {
> +               up(&sess->schedule_conn->tx_sem);
> +               spin_unlock_irqrestore(&sess->conn_schedule_lock, flags);
> +               return;
> +       }
> 
> Presumably, sess->nconn need not be atomic?
> 
> In practice, I have found that use of atomic_t often _creates_ races.
> 

These bits are the simple round robin scheduling algortim to determine
which connection TX thread needs to get activated in order to do work.
This lock is only held in two other locations:

A) When a new connection moves into Full Feature Phase (LOGGED_IN
state).

B) When a connection leaves a LOGGED_IN state for various exception
reasons.

This is to ensure that the above function (which as you know can be
called from the SCSI bottom half to send SCSI tasks) does not get a
stale iscsi_conn_t entry.

> 
> 5) style:  these function headers are completely pointless, conveying no 
> additional insight or information:
> 
> +/*     iscsi_add_nopout_noqueue():
> + *
> + *
> + */
> 

Ok.

> 
> 6) This is very worrisome, at the beginning of your queuecommand hook:
> 
> +       /*
> +       * Get the assoicated iSCSI Channel for this active SCSI Task.
> +       */
> +       spin_unlock_irq(sc->device->host->host_lock);
> 
> 
> It's fairly dangerous to assume that this will work, if, e.g. 
> ->queuecommand() is called from BH context.
> 
> Also, it means you are accessing the internal LUN list without any 
> locking at all.
> 

This in fact is the only thing that would not cause a deadlock. Having
to disable hard interrupts with struct Scsi_Host->host_lock while
needing to access a single iscsi_channel_t and three iscsi_session_t
spinlocks from within an bottom half is a nightmare.  I recall the sfnet
implementation doing the same thing to avoid a related deadlock issue.
Also, nothing aside from getting struct scsi_cmnd->device is being
accessed from within ->queuecommand().  Everything else related to
accessing the LUN list is protected by the usual means.

I am open to other suggestions on this one.


> 
> 7) leak on error:
> 
> +       if (!(cmd = iscsi_allocate_cmd(sess, NULL)))
> +               return(-1);
> +
> +       if ((padding = ((text_len) & 3)) != 0) {
> +               TRACE(TRACE_ISCSI, "Attaching %u additional bytes for"
> +                       " padding.\n", padding);
> +       }
> +
> +       if (!(text = (unsigned char *) kmalloc((text_len + padding), 
> GFP_ATOMIC)
> )) {
> +               TRACE_ERROR("Unable to allocate memory for outgoing 
> text\n");
> +               return(-1);
> +       }
> 
> 

Ok.

> 
> 8) iscsi_build_scsi_cmd() appears to leak cmd->buf_ptr on error, but I 
> could be wrong
> 
> 
> 9) style:  Linux kernel style discourages use of "foo_t", and prefers 
> "struct foo"
> 

Ok, I will make an effort take care of this in the next release.

> 
> 10) iscsi_build_dataout() leak on error:
> 
> +               if (!(pdu = iscsi_get_pdu_holder_from_r2t(cmd, r2t)))
> +                       return(-1);
> ...
> +               if ((iov_ret = iscsi_map_scatterlists((void *)&map_sg,
> +                                       (void *)unmap_sg)) < 0)
> +                       return(-1);
> 
> 

Ok.

> 11) excessive stack usage.  will cause crash with 4K stacks:
> 
> +       unsigned char ping_data[ISCSI_MAX_PING_DATA_BYTES];
> 

Will change to dynamic allocation.

> 
> 12) general comment:  It is damned difficult to figure out which context 
> these functions are operating in.  In iscsi_build_text_req() I see you 
> grab a spinlock with spin_lock().  May we presume that local_irq_save() 
> is active?  Have you audited this code for ABBA deadlocks?
> 

Yes, a huge amount of effort has gone into making sure the locks are
aquired and released in the correct order.  Another area that has been
worked on extensively is the interaction between the locks that need to
be held in the ->queuecommand() codepath.  I can document these locks as
well.

> 
> 13) delete all of the following #ifdefs:
> 
> +#ifdef CRYPTO_API_CRC32C
> 
> and replace with a requirement in Kconfig.
> 
> 

Ok.

> 14) Many instances where iscsi_find_cmd_from_itt_or_dump() returns a 
> command, and then function returns on error without releasing cmd.
> 
> This -may- be a leak on error, maybe not.
> 

When iscsi_cmd_t cannot be located by iscsi_find_cmd_from_itt_or_dump()
the datain buffer gets dumped and returns.  This is not a leak error and
is intented to handle various within command recovery situations.  One
example is re-requesting datain that was presumed lost, but in fact was
intact and its assoicated SCSI task completed.  Once the recovery DataIn
arrived for the SCSI task that has already been completed this function
will dump the length of the DataIn after an Initiator Task Tag (ITT)
cannot be found.

> 
> 15) locking bug: conn->state_lock is locked with spin_lock() in process 
> context, spin_lock_bh() in other contexts.
> 

Ok, double check and fix.

> 
> 16) stack usage:
> 
> +extern int iscsi_initiator_rx_thread (void *arg)
> +{
> +       int ret;
> +       u8 buffer[ISCSI_HDR_LEN], opcode;
> 

This one is 48 bytes.  If I could keep anyone statically allocated it
would be this one.

> 
> 17) potential race in iscsi_create_connection():
> 
> +       if ((atomic_read(&sess->nconn) + 
> atomic_read(&c->connection_count)) >
> 

Will look into this one.

> 
> 18) excessive stack usage:
> 
> +extern int iscsi_free_session (iscsi_session_t *sess)
> +{
> +       int i = 0, j = 0;
> +       iscsi_conn_t *conn, *conn_next = NULL;
> +       iscsi_conn_t *conn_array[ISCSI_MAX_CONNECTIONS];
> 

Will change to dynamically allocated memory.

> 
> 19) ditto:
> 
> +extern int iscsi_stop_session (iscsi_session_t *sess, int 
> session_sleep, int co
> nnection_sleep)
> +{
> +       int i = 0, j = 0;
> +       iscsi_conn_t *conn, *conn_next = NULL;
> +       iscsi_conn_t *conn_array[ISCSI_MAX_CONNECTIONS];
> 

Will change to dynamically allocated memory.

> 
> 
> 20) numerous code bloat, by checking for NULL before calling kfree():
> 
> +               if (c)
> +                       kfree(c);
> 
> kfree(NULL) is ok.
> 

Ok, will spend time going through and replacing this.

> 
> 21) delete FULL_PARAMS_COUNT constant, use ARRAY_SIZE() macro in 
> linux/kernel.h
> 

Ok.

> 
> 22) excessive stack usage:
> 
> +static int iscsi_setup_full_params (iscsi_channel_t *c)
> +{
> +       unsigned char buf[512];
> 

Will change to dynamically allocated memory.

> 
> 23) ditto:
> 
> +extern int iscsi_init_channel (iscsi_channel_t *channel, u32 
> max_sectors, int f
> ull_init)
> +{
> +       unsigned char buf[512];
> 
> 

Will change to dynamically allocated memory.

> 
> I stopped reading at this point.
> 
> In general,
> 
> a) locking is completely unreviewable, incomprehensible and 
> unmaintainable by anyone except the author(s).  Either a locking 
> rewrite, or a locking proof document, is needed.
> 

I can spend time working on a document that outlines the important locks
in iscsi_cmd_t, iscsi_conn_t, iscsi_session_t and iscsi_channel_t in
order to give reviewers a clearer idea of how locking is handled.  Also
a better explanation of the limitiations of disabling hard interrupts
with struct Scsi_Host->host_lock as it applies to the iSCSI Initiator
Core can be included.

> b) aside from locking, stack usage, and leaks on error, the code seems 
> reasonably clean.
> 

Thanks for your input!!

-- 
Nicholas A. Bellinger <nick@pyxtechnologies.com>

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[AJ Lewis]

"Nicholas A. Bellinger" <nick@pyxtechnologies.com> writes:

> The following is the first public release of the PyX Technologies iSCSI
> Initiator Core Stack v1.6.1.20 for Linux 2.6.11-rc4.  This is a full
> featured iSCSI Initiator stack that is capable of mulitplexing coast to
> coast across multiple independant backbone providers using various
> network transports _TODAY_.  This is the first release of the core stack
> and assoicated userspace tools, the accompanying authentication daemon
> will be released shortly.

Very cool - glad you're releasing this. I've gotten this setup and
running, but I can't seem to talk to my iscsi target.  The initiator
detects it, but it can't recognize the type:

iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping SecurityNegotiation phase.
iCHANNEL[0] - iSCSI login successful on CID: 0 to 192.168.44.19:3260,2
iCHANNEL[0] - Incremented iSCSI connection count to 1 to node: iqn.1992-08.com.netapp:sn.33615311
iCHANNEL[0] - Established iSCSI session to node: iqn.1992-08.com.netapp:sn.33615311
iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
scsi: unknown device type 31
  Vendor: NETAPP    Model: LUN               Rev: 0.2 
  Type:   Unknown                            ANSI SCSI revision: 04

Any ideas on why this wouldn't be working?  Is there a configuration
step I'm missing?

Also, are you planning on updating the docs for the user tools at all?
There are several steps missing in the current README, and the man
page lists some parameters incorrectly and misses other entirely.

I'm looking forward to getting this working.
-- 
AJ Lewis                                   Voice:  612-638-0500
Red Hat Inc.                               E-Mail: alewis@redhat.com
720 Washington Ave. SE, Suite 200
Minneapolis, MN 55414
   
Current GPG fingerprint = D9F8 EDCE 4242 855F A03D  9B63 F50C 54A8 578C 8715
Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the
many keyservers out there...

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Nicholas A. Bellinger]

On Thu, 2005-02-24 at 14:51 -0600, AJ Lewis wrote:
> "Nicholas A. Bellinger" <nick@pyxtechnologies.com> writes:
> 
> > The following is the first public release of the PyX Technologies iSCSI
> > Initiator Core Stack v1.6.1.20 for Linux 2.6.11-rc4.  This is a full
> > featured iSCSI Initiator stack that is capable of mulitplexing coast to
> > coast across multiple independant backbone providers using various
> > network transports _TODAY_.  This is the first release of the core stack
> > and assoicated userspace tools, the accompanying authentication daemon
> > will be released shortly.
> 
> Very cool - glad you're releasing this. I've gotten this setup and
> running, but I can't seem to talk to my iscsi target.  The initiator
> detects it, but it can't recognize the type:
> 
> iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping SecurityNegotiation phase.
> iCHANNEL[0] - iSCSI login successful on CID: 0 to 192.168.44.19:3260,2
> iCHANNEL[0] - Incremented iSCSI connection count to 1 to node: iqn.1992-08.com.netapp:sn.33615311
> iCHANNEL[0] - Established iSCSI session to node: iqn.1992-08.com.netapp:sn.33615311
> iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
> scsi: unknown device type 31
>   Vendor: NETAPP    Model: LUN               Rev: 0.2 
>   Type:   Unknown                            ANSI SCSI revision: 04
> 
> Any ideas on why this wouldn't be working?  Is there a configuration
> step I'm missing?
> 

This looks like an unknown/unsupported SCSI Peripheral device type in
the INQUIRY response data.  Is this LU just a sequential access device
that is setting those bits to some different value?

It would certainly be interesting to see with Ethereal what is in the
INQUIRY response data that Linux could possibly not support. :-)


> Also, are you planning on updating the docs for the user tools at all?
> There are several steps missing in the current README, and the man
> page lists some parameters incorrectly and misses other entirely.
> 

There are a couple of things I would like to be accomplished in the near
feature for the user tools:

1) Complete documentation of iSCSI Channel Management optertions by way
of updated initiator-ctl manual pages and basic HOWTO for iSCSI Logical
Unit Management Screnarios for iSCSI SAN Adminstitrators.
2) Initial release of authentication daemon using CHAP, with eventual
support for RFC 3720 defined SRP.
3) Work towards supporting the Internet Storage Naming Service (iSNS)
standard and assoicated open source implementations for discovery and
device information for iSCSI Target Nodes and other storage resources.

Thanks for the feedback!

-- 
Nicholas A. Bellinger <nick@pyxtechnologies.com>

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[AJ Lewis]

[-- Attachment #1: Type: Text/Plain, Size: 3638 bytes --]

On Thu, 24 Feb 2005 23:50:29 -0800, "Nicholas A. Bellinger" <nick@pyxtechnologies.com> wrote:
> On Thu, 2005-02-24 at 14:51 -0600, AJ Lewis wrote:
> > I've gotten this setup and
> > running, but I can't seem to talk to my iscsi target.  The initiator
> > detects it, but it can't recognize the type:
> > 
> > iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping SecurityNegotiation phase.
> > iCHANNEL[0] - iSCSI login successful on CID: 0 to 192.168.44.19:3260,2
> > iCHANNEL[0] - Incremented iSCSI connection count to 1 to node: iqn.1992-08.com.netapp:sn.33615311
> > iCHANNEL[0] - Established iSCSI session to node: iqn.1992-08.com.netapp:sn.33615311
> > iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
> > scsi: unknown device type 31
> >   Vendor: NETAPP    Model: LUN               Rev: 0.2 
> >   Type:   Unknown                            ANSI SCSI revision: 04
> > 
> > Any ideas on why this wouldn't be working?  Is there a configuration
> > step I'm missing?
> > 
> 
> This looks like an unknown/unsupported SCSI Peripheral device type in
> the INQUIRY response data.  Is this LU just a sequential access device
> that is setting those bits to some different value?

Well, the cisco linux-iscsi initiator works just fine with it, so I
don't think it's an issue with the NetApp and linux.  With the
linux-iscsi initiator, /proc/scsi/scsi looks like this:

Host: scsi4 Channel: 00 Id: 00 Lun: 00
  Vendor: NETAPP   Model: LUN              Rev: 0.2 
  Type:   Direct-Access                    ANSI SCSI revision: 04


With the pyx initiator, it looks like this:
 Host: scsi6 Channel: 00 Id: 00 Lun: 00
  Vendor: NETAPP   Model: LUN              Rev: 0.2 
  Type:   Unknown                          ANSI SCSI revision: 04

> It would certainly be interesting to see with Ethereal what is in the
> INQUIRY response data that Linux could possibly not support. :-)

Hrm...don't have anything setup to do that ATM, but I could probably
get something going...

> > Also, are you planning on updating the docs for the user tools at all?
> > There are several steps missing in the current README, and the man
> > page lists some parameters incorrectly and misses other entirely.
> > 
> 
> There are a couple of things I would like to be accomplished in the near
> feature for the user tools:
> 
> 1) Complete documentation of iSCSI Channel Management optertions by way
> of updated initiator-ctl manual pages and basic HOWTO for iSCSI Logical
> Unit Management Screnarios for iSCSI SAN Adminstitrators.
> 2) Initial release of authentication daemon using CHAP, with eventual
> support for RFC 3720 defined SRP.
> 3) Work towards supporting the Internet Storage Naming Service (iSNS)
> standard and assoicated open source implementations for discovery and
> device information for iSCSI Target Nodes and other storage resources.

And for the kernel side, it looks like:

 1) /proc -> sysfs conversion
 2) ioctl -> sysfs conversion
 Both of which include integration of the iscsi transport class discussed
 on this list

Out of curiousity, what does this initiator offer that the linux-iscsi
initiator doesn't?  (http://linux-iscsi.sourceforge.net/) I'm just
curious what you see as advantages this implementation has.

Regards,
--
AJ Lewis                                   Voice:  612-638-0500
Red Hat Inc.                               E-Mail: alewis@redhat.com
720 Washington Ave. SE, Suite 200
Minneapolis, MN 55414
   
Current GPG fingerprint = D9F8 EDCE 4242 855F A03D  9B63 F50C 54A8 578C 8715
Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the
many keyservers out there...



[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Nicholas A. Bellinger]

On Fri, 2005-02-25 at 09:18 -0600, AJ Lewis wrote:
> On Thu, 24 Feb 2005 23:50:29 -0800, "Nicholas A. Bellinger" <nick@pyxtechnologies.com> wrote:
> > On Thu, 2005-02-24 at 14:51 -0600, AJ Lewis wrote:
> > > I've gotten this setup and
> > > running, but I can't seem to talk to my iscsi target.  The initiator
> > > detects it, but it can't recognize the type:
> > > 
> > > iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping SecurityNegotiation phase.
> > > iCHANNEL[0] - iSCSI login successful on CID: 0 to 192.168.44.19:3260,2
> > > iCHANNEL[0] - Incremented iSCSI connection count to 1 to node: iqn.1992-08.com.netapp:sn.33615311
> > > iCHANNEL[0] - Established iSCSI session to node: iqn.1992-08.com.netapp:sn.33615311
> > > iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
> > > scsi: unknown device type 31
> > >   Vendor: NETAPP    Model: LUN               Rev: 0.2 
> > >   Type:   Unknown                            ANSI SCSI revision: 04
> > > 
> > > Any ideas on why this wouldn't be working?  Is there a configuration
> > > step I'm missing?
> > > 
> > 
> > This looks like an unknown/unsupported SCSI Peripheral device type in
> > the INQUIRY response data.  Is this LU just a sequential access device
> > that is setting those bits to some different value?
> 
> Well, the cisco linux-iscsi initiator works just fine with it, so I
> don't think it's an issue with the NetApp and linux.  With the
> linux-iscsi initiator, /proc/scsi/scsi looks like this:
> 
> Host: scsi4 Channel: 00 Id: 00 Lun: 00
>   Vendor: NETAPP   Model: LUN              Rev: 0.2 
>   Type:   Direct-Access                    ANSI SCSI revision: 04
> 
> 
> With the pyx initiator, it looks like this:
>  Host: scsi6 Channel: 00 Id: 00 Lun: 00
>   Vendor: NETAPP   Model: LUN              Rev: 0.2 
>   Type:   Unknown                          ANSI SCSI revision: 04
>

This is strange indeed.  I am not sure how the peripheral device type
could get set incorrectly during the DataIN transfer, but the
Vendor/Model/Rev still be intact.

> > It would certainly be interesting to see with Ethereal what is in the
> > INQUIRY response data that Linux could possibly not support. :-)
> 
> Hrm...don't have anything setup to do that ATM, but I could probably
> get something going...
> 

This would definately help.

> > > Also, are you planning on updating the docs for the user tools at all?
> > > There are several steps missing in the current README, and the man
> > > page lists some parameters incorrectly and misses other entirely.
> > > 
> > 
> > There are a couple of things I would like to be accomplished in the near
> > feature for the user tools:
> > 
> > 1) Complete documentation of iSCSI Channel Management optertions by way
> > of updated initiator-ctl manual pages and basic HOWTO for iSCSI Logical
> > Unit Management Screnarios for iSCSI SAN Adminstitrators.
> > 2) Initial release of authentication daemon using CHAP, with eventual
> > support for RFC 3720 defined SRP.
> > 3) Work towards supporting the Internet Storage Naming Service (iSNS)
> > standard and assoicated open source implementations for discovery and
> > device information for iSCSI Target Nodes and other storage resources.
> 
> And for the kernel side, it looks like:
> 
>  1) /proc -> sysfs conversion

I am working on this one.

>  2) ioctl -> sysfs conversion

This may take a bit longer than #1.

>  Both of which include integration of the iscsi transport class discussed
>  on this list

I have a patch for that that I am working on as well.

> Out of curiousity, what does this initiator offer that the linux-iscsi
> initiator doesn't?  (http://linux-iscsi.sourceforge.net/) I'm just
> curious what you see as advantages this implementation has.

I took the the important iSCSI features from RELEASE_NOTES:

3) Full support for Multiple Connections per Session that can be brought
up and down on the fly.
4) Full support for DataPDUInOrder=No, DataSequenceInOrder=No,
MaxOutstandingR2T>1.
5) Full support for Sync and Steering using Fixed Interval Markers. (See
RFC 3720 Appendix A) 
6) Full support for TCP and SCTP network transports.
7) Robust set of iSCSI Channel management options for iSCSI SAN
Administrators
*8) Persisant mapping of iSCSI Logical Units to mount points.

Also, a fair amount of code not directly related to but required for
ERL=1 logic has been included as well.  Over time we will be releasing
parts of ERL=1 functionality until all of within command and within
connection recovery as defined by 6.1.4.1 and 6.1.4.2 of RFC 3720 is
supported.

Thanks,

-- 
Nicholas A. Bellinger <nick@pyxtechnologies.com>

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Dave Wysochanski]

AJ Lewis wrote:

> "Nicholas A. Bellinger" <nick@pyxtechnologies.com> writes:
>
> > The following is the first public release of the PyX Technologies iSCSI
> > Initiator Core Stack v1.6.1.20 for Linux 2.6.11-rc4.  This is a full
> > featured iSCSI Initiator stack that is capable of mulitplexing coast to
> > coast across multiple independant backbone providers using various
> > network transports _TODAY_.  This is the first release of the core 
> stack
> > and assoicated userspace tools, the accompanying authentication daemon
> > will be released shortly.
>
> Very cool - glad you're releasing this. I've gotten this setup and
> running, but I can't seem to talk to my iscsi target.  The initiator
> detects it, but it can't recognize the type:
>
> iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping 
> SecurityNegotiation phase.
> iCHANNEL[0] - iSCSI login successful on CID: 0 to 192.168.44.19:3260,2
> iCHANNEL[0] - Incremented iSCSI connection count to 1 to node: 
> iqn.1992-08.com.netapp:sn.33615311
> iCHANNEL[0] - Established iSCSI session to node: 
> iqn.1992-08.com.netapp:sn.33615311
> iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
> scsi: unknown device type 31
>   Vendor: NETAPP    Model: LUN               Rev: 0.2
>   Type:   Unknown                            ANSI SCSI revision: 04
>
> Any ideas on why this wouldn't be working?  Is there a configuration
> step I'm missing?
>
Make sure the initiator name is in the igroup on the filer for
any luns that you want mapped to the intiator.

I think what you're seeing is not a real LUN, but the config LUN
or something like that, probably the result of the initiator name
not being in the igroup (see "igroup add" command on the filer).

> Also, are you planning on updating the docs for the user tools at all?
> There are several steps missing in the current README, and the man
> page lists some parameters incorrectly and misses other entirely.
>
> I'm looking forward to getting this working.
> -- 
> AJ Lewis                                   Voice:  612-638-0500
> Red Hat Inc.                               E-Mail: alewis@redhat.com
> 720 Washington Ave. SE, Suite 200
> Minneapolis, MN 55414
>   
> Current GPG fingerprint = D9F8 EDCE 4242 855F A03D  9B63 F50C 54A8 
> 578C 8715
> Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the
> many keyservers out there...
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Andre Hedrick]

Hi Dave,

If the NetAPP target is misconfigured, why would it even allow any unknown
initiator to go beyond discovery session?  It appears that a
semi-successful lun scan has happened or am mis reading the report?

If I read deeper into your comments, does the target support ACL's?
If yes, then how did any access to the target by an unauthorized initiator
happen?

Also would you be kind enough to reveal what device class 31 is to conform
to under scsi?

Cheers,

Andre

On Sun, 27 Feb 2005, Dave Wysochanski wrote:

> AJ Lewis wrote:
> 
> > "Nicholas A. Bellinger" <nick@pyxtechnologies.com> writes:
> >
> > > The following is the first public release of the PyX Technologies iSCSI
> > > Initiator Core Stack v1.6.1.20 for Linux 2.6.11-rc4.  This is a full
> > > featured iSCSI Initiator stack that is capable of mulitplexing coast to
> > > coast across multiple independant backbone providers using various
> > > network transports _TODAY_.  This is the first release of the core 
> > stack
> > > and assoicated userspace tools, the accompanying authentication daemon
> > > will be released shortly.
> >
> > Very cool - glad you're releasing this. I've gotten this setup and
> > running, but I can't seem to talk to my iscsi target.  The initiator
> > detects it, but it can't recognize the type:
> >
> > iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping 
> > SecurityNegotiation phase.
> > iCHANNEL[0] - iSCSI login successful on CID: 0 to 192.168.44.19:3260,2
> > iCHANNEL[0] - Incremented iSCSI connection count to 1 to node: 
> > iqn.1992-08.com.netapp:sn.33615311
> > iCHANNEL[0] - Established iSCSI session to node: 
> > iqn.1992-08.com.netapp:sn.33615311
> > iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
> > scsi: unknown device type 31
> >   Vendor: NETAPP    Model: LUN               Rev: 0.2
> >   Type:   Unknown                            ANSI SCSI revision: 04
> >
> > Any ideas on why this wouldn't be working?  Is there a configuration
> > step I'm missing?
> >
> Make sure the initiator name is in the igroup on the filer for
> any luns that you want mapped to the intiator.
> 
> I think what you're seeing is not a real LUN, but the config LUN
> or something like that, probably the result of the initiator name
> not being in the igroup (see "igroup add" command on the filer).
> 
> > Also, are you planning on updating the docs for the user tools at all?
> > There are several steps missing in the current README, and the man
> > page lists some parameters incorrectly and misses other entirely.
> >
> > I'm looking forward to getting this working.
> > -- 
> > AJ Lewis                                   Voice:  612-638-0500
> > Red Hat Inc.                               E-Mail: alewis@redhat.com
> > 720 Washington Ave. SE, Suite 200
> > Minneapolis, MN 55414
> >   
> > Current GPG fingerprint = D9F8 EDCE 4242 855F A03D  9B63 F50C 54A8 
> > 578C 8715
> > Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the
> > many keyservers out there...
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[Dave Wysochanski]

Andre Hedrick wrote:

>
> Hi Dave,
>
> If the NetAPP target is misconfigured, why would it even allow any 
> unknown
> initiator to go beyond discovery session?  It appears that a
> semi-successful lun scan has happened or am mis reading the report?
>
I think you're misreading the report and/or don't understand how our target
works.

> If I read deeper into your comments, does the target support ACL's?
> If yes, then how did any access to the target by an unauthorized 
> initiator
> happen?
>
The LUN mapping and security are technically different lists.  The
default  security policy (unless set otherwise) is AuthMethod=none.
By default though, you don't get any LUNs until you explicitly
map them.

> Also would you be kind enough to reveal what device class 31 is to 
> conform
> to under scsi?
>
I'll try to get more information on what's going on here.  I didn't actually
repro this, so I don't have a trace either - was just commenting based
on the mail.

> Cheers,
>
> Andre
>
> On Sun, 27 Feb 2005, Dave Wysochanski wrote:
>
> > AJ Lewis wrote:
> >
> > > "Nicholas A. Bellinger" <nick@pyxtechnologies.com> writes:
> > >
> > > > The following is the first public release of the PyX 
> Technologies iSCSI
> > > > Initiator Core Stack v1.6.1.20 for Linux 2.6.11-rc4.  This is a 
> full
> > > > featured iSCSI Initiator stack that is capable of mulitplexing 
> coast to
> > > > coast across multiple independant backbone providers using various
> > > > network transports _TODAY_.  This is the first release of the core
> > > stack
> > > > and assoicated userspace tools, the accompanying authentication 
> daemon
> > > > will be released shortly.
> > >
> > > Very cool - glad you're releasing this. I've gotten this setup and
> > > running, but I can't seem to talk to my iscsi target.  The initiator
> > > detects it, but it can't recognize the type:
> > >
> > > iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping
> > > SecurityNegotiation phase.
> > > iCHANNEL[0] - iSCSI login successful on CID: 0 to 
> 192.168.44.19:3260,2
> > > iCHANNEL[0] - Incremented iSCSI connection count to 1 to node:
> > > iqn.1992-08.com.netapp:sn.33615311
> > > iCHANNEL[0] - Established iSCSI session to node:
> > > iqn.1992-08.com.netapp:sn.33615311
> > > iSCSI Core Stack[1] - Incremented number of active iSCSI sessions 
> to 1.
> > > scsi: unknown device type 31
> > >   Vendor: NETAPP    Model: LUN               Rev: 0.2
> > >   Type:   Unknown                            ANSI SCSI revision: 04
> > >
> > > Any ideas on why this wouldn't be working?  Is there a configuration
> > > step I'm missing?
> > >
> > Make sure the initiator name is in the igroup on the filer for
> > any luns that you want mapped to the intiator.
> >
> > I think what you're seeing is not a real LUN, but the config LUN
> > or something like that, probably the result of the initiator name
> > not being in the igroup (see "igroup add" command on the filer).
> >
> > > Also, are you planning on updating the docs for the user tools at 
> all?
> > > There are several steps missing in the current README, and the man
> > > page lists some parameters incorrectly and misses other entirely.
> > >
> > > I'm looking forward to getting this working.
> > > --
> > > AJ Lewis                                   Voice:  612-638-0500
> > > Red Hat Inc.                               E-Mail: alewis@redhat.com
> > > 720 Washington Ave. SE, Suite 200
> > > Minneapolis, MN 55414
> > >  
> > > Current GPG fingerprint = D9F8 EDCE 4242 855F A03D  9B63 F50C 54A8
> > > 578C 8715
> > > Grab the key at: http://people.redhat.com/alewis/gpg.html or one 
> of the
> > > many keyservers out there...
> > >
> > > -
> > > To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > >
> > -
> > To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [ANNOUNCE] iSCSI Initiator Core Stack v1.6.1.20

[AJ Lewis]

[-- Attachment #1: Type: Text/Plain, Size: 2702 bytes --]

On Sun, 27 Feb 2005 13:46:00 -0500, Dave Wysochanski <davidw@netapp.com> wrote:
> AJ Lewis wrote:
> 
> > "Nicholas A. Bellinger" <nick@pyxtechnologies.com> writes:
> >
> > > The following is the first public release of the PyX Technologies iSCSI
> > > Initiator Core Stack v1.6.1.20 for Linux 2.6.11-rc4.  This is a full
> > > featured iSCSI Initiator stack that is capable of mulitplexing coast to
> > > coast across multiple independant backbone providers using various
> > > network transports _TODAY_.  This is the first release of the core 
> > stack
> > > and assoicated userspace tools, the accompanying authentication daemon
> > > will be released shortly.
> >
> > Very cool - glad you're releasing this. I've gotten this setup and
> > running, but I can't seem to talk to my iscsi target.  The initiator
> > detects it, but it can't recognize the type:
> >
> > iCHANNEL[0] - No defined iSCSI Authentication Methods, skipping 
> > SecurityNegotiation phase.
> > iCHANNEL[0] - iSCSI login successful on CID: 0 to 192.168.44.19:3260,2
> > iCHANNEL[0] - Incremented iSCSI connection count to 1 to node: 
> > iqn.1992-08.com.netapp:sn.33615311
> > iCHANNEL[0] - Established iSCSI session to node: 
> > iqn.1992-08.com.netapp:sn.33615311
> > iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
> > scsi: unknown device type 31
> >   Vendor: NETAPP    Model: LUN               Rev: 0.2
> >   Type:   Unknown                            ANSI SCSI revision: 04
> >
> > Any ideas on why this wouldn't be working?  Is there a configuration
> > step I'm missing?
> >
> Make sure the initiator name is in the igroup on the filer for
> any luns that you want mapped to the intiator.
> 
> I think what you're seeing is not a real LUN, but the config LUN
> or something like that, probably the result of the initiator name
> not being in the igroup (see "igroup add" command on the filer).

Umm...I doubt it.  The initiator name specified in
'/etc/sysconfig/initiator' is the same as the one specified in
/etc/initiatorname.iscsi (the one for the linux-iscsi initiator, which
works).  And this is the same setup I've had for several months now
with the netapp and the linux-iscsi initiator with now problems.  And
I actually pulled the initiator name from the igroup just to be
certain.

Any other ideas?

Regards,
--
AJ Lewis                                   Voice:  612-638-0500
Red Hat Inc.                               E-Mail: alewis@redhat.com
720 Washington Ave. SE, Suite 200
Minneapolis, MN 55414
   
Current GPG fingerprint = D9F8 EDCE 4242 855F A03D  9B63 F50C 54A8 578C 8715
Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the
many keyservers out there...


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]