Re: Re[2]: ata over ethernet question

Re: Re[2]: ata over ethernet question

[Guennadi Liakhovetski]

Hi

On Sat, 7 May 2005, Sander wrote:

> David Hollis wrote (ao):
> > There seem to be a few iSCSI implementations floating around for
> > Linux, hopefully one will be added to mainline soon. Most of those
> > implementations are for the client side though there is at least one
> > target implementation that allows you to provide local storage to
> > iSCSI clients. I don't remember the name of it or if it's still
> > maintained or not.
> 
> Quite active even:
> 
> http://sourceforge.net/projects/iscsitarget/
> 
> The "Quick Guide to iSCSI on Linux" is a good starting point btw.
> 
> Also check out http://www.open-iscsi.org/ (the client, aka 'initiator').

A follow up question - I recently used nbd to access a CD-ROM. It worked 
nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
had to stop the client, the server, then replace the CD, re-start the 
server, re-start the client... I thought about extending NBD to (better) 
support removable media, but then you start thinking about all those 
features that your local block device has that don't get exported over 
NBD...

Now, my understanding (sorry, without looking at any docs - yet) is, that 
iSCSI is (or at least should be) free from these limitations. So, does it 
make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
just kept simple as it is or would it be completely superseeded by iSCSI, 
or is there still something that NBD does that iSCSI wouldn't (easily) do?

Or am I completely misunderstanding what iSCSI target does?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: Re[2]: ata over ethernet question

[Dmitry Yusupov]

I'll redirect this question to iSCSI Target folks, so they will lead
some light on current IET limitations and advantages comparing to NBD.

On Wed, 2005-05-11 at 00:00 +0200, Guennadi Liakhovetski wrote:
> Hi
> 
> On Sat, 7 May 2005, Sander wrote:
> 
> > David Hollis wrote (ao):
> > > There seem to be a few iSCSI implementations floating around for
> > > Linux, hopefully one will be added to mainline soon. Most of those
> > > implementations are for the client side though there is at least one
> > > target implementation that allows you to provide local storage to
> > > iSCSI clients. I don't remember the name of it or if it's still
> > > maintained or not.
> > 
> > Quite active even:
> > 
> > http://sourceforge.net/projects/iscsitarget/
> > 
> > The "Quick Guide to iSCSI on Linux" is a good starting point btw.
> > 
> > Also check out http://www.open-iscsi.org/ (the client, aka 'initiator').
> 
> A follow up question - I recently used nbd to access a CD-ROM. It worked 
> nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> had to stop the client, the server, then replace the CD, re-start the 
> server, re-start the client... I thought about extending NBD to (better) 
> support removable media, but then you start thinking about all those 
> features that your local block device has that don't get exported over 
> NBD...
> 
> Now, my understanding (sorry, without looking at any docs - yet) is, that 
> iSCSI is (or at least should be) free from these limitations. So, does it 
> make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> just kept simple as it is or would it be completely superseeded by iSCSI, 
> or is there still something that NBD does that iSCSI wouldn't (easily) do?
> 
> Or am I completely misunderstanding what iSCSI target does?
> 
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Iscsitarget-devel] Re: Re[2]: ata over ethernet question

[FUJITA Tomonori]

From: Dmitry Yusupov <dmitry_yus@yahoo.com>
Subject: [Iscsitarget-devel] Re: Re[2]: ata over ethernet question
Date: Tue, 10 May 2005 16:14:35 -0700

> I'll redirect this question to iSCSI Target folks, so they will lead
> some light on current IET limitations and advantages comparing to NBD.

I'm not familiar with NBD, so I'll talk about only iSCSI and the
current IET implementation.


> > A follow up question - I recently used nbd to access a CD-ROM. It worked 
> > nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> > had to stop the client, the server, then replace the CD, re-start the 
> > server, re-start the client... I thought about extending NBD to (better) 
> > support removable media, but then you start thinking about all those 
> > features that your local block device has that don't get exported over 
> > NBD...
> > 
> > Now, my understanding (sorry, without looking at any docs - yet) is, that 
> > iSCSI is (or at least should be) free from these limitations. So, does it 
> > make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> > just kept simple as it is or would it be completely superseeded by iSCSI, 
> > or is there still something that NBD does that iSCSI wouldn't (easily) do?

The iSCSI protocol simply encapsulates the SCSI protocol into the
TCP/IP protocol, and carries packets over IP networks. You can handle
remote devices in the exact same way as local devices (which are
directly connected to your machine through SCSI cables). So iSCSI
enables you to access a remote SCSI CD-ROM withough the above trouble.

The current IET implementation can offer disk drives to
initiators. You can use files, that is, regular files, block devices,
LVM, or RAID.

The patch for SCSI tape and CD-ROM drive support is available. It
enables you to offer SCSI tape and CD-ROM drives connected the target
box to initiators. It works in most cases, however, it is not
finished. So it is not merged.

We've not started to work on virtual tape and CD-ROM support yet.

Re: ata over ethernet question

[Vladislav Bolkhovitin]

Guennadi Liakhovetski wrote:
> Hi
> 
> On Sat, 7 May 2005, Sander wrote:
> 
> 
>>David Hollis wrote (ao):
>>
>>>There seem to be a few iSCSI implementations floating around for
>>>Linux, hopefully one will be added to mainline soon. Most of those
>>>implementations are for the client side though there is at least one
>>>target implementation that allows you to provide local storage to
>>>iSCSI clients. I don't remember the name of it or if it's still
>>>maintained or not.
>>
>>Quite active even:
>>
>>http://sourceforge.net/projects/iscsitarget/
>>
>>The "Quick Guide to iSCSI on Linux" is a good starting point btw.
>>
>>Also check out http://www.open-iscsi.org/ (the client, aka 'initiator').
> 
> 
> A follow up question - I recently used nbd to access a CD-ROM. It worked 
> nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> had to stop the client, the server, then replace the CD, re-start the 
> server, re-start the client... I thought about extending NBD to (better) 
> support removable media, but then you start thinking about all those 
> features that your local block device has that don't get exported over 
> NBD...
> 
> Now, my understanding (sorry, without looking at any docs - yet) is, that 
> iSCSI is (or at least should be) free from these limitations. So, does it 
> make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> just kept simple as it is or would it be completely superseeded by iSCSI, 
> or is there still something that NBD does that iSCSI wouldn't (easily) do?
> 
> Or am I completely misunderstanding what iSCSI target does?

Actually, this is property not of iSCSI target itself, but of any SCSI 
target. So, we implemented it as part of our SCSI target mid-level 
(SCST, http://scst.sourceforge.net), therefore any target driver working 
over it will automatically benefit from this feature. Unfortunately, 
currently available only target drivers for Qlogic 2x00 cards and for 
poor UNH iSCSI target (that works not too reliable and only with very 
specific initiators). The published version supports only real SCSI 
CDROMs. CDROM FILEIO module, which allows exporting ISO images as SCSI 
CDROM devices, going to be available not later end of May.

Vlad

> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Re: several messages

[Guennadi Liakhovetski]

Hello and thanks for the replies

On Wed, 11 May 2005, FUJITA Tomonori wrote:
> The iSCSI protocol simply encapsulates the SCSI protocol into the
> TCP/IP protocol, and carries packets over IP networks. You can handle
...

On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> Actually, this is property not of iSCSI target itself, but of any SCSI target.
> So, we implemented it as part of our SCSI target mid-level (SCST,
> http://scst.sourceforge.net), therefore any target driver working over it will
> automatically benefit from this feature. Unfortunately, currently available
> only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
> works not too reliable and only with very specific initiators). The published
...

The above confirms basically my understanding apart from one "minor" 
confusion - I thought, that parallel to hardware solutions pure software 
implementations were possible / being developed, like a driver, that 
implements a SCSI LDD API on one side, and forwards packets to an IP 
stack, say, over an ethernet card - on the initiator side. And a counter 
part on the target side. Similarly to the USB mass-storage and storage 
gadget drivers?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: several messages

[Ming Zhang]

[-- Attachment #1: Type: text/plain, Size: 1703 bytes --]

iscsi is scsi over ip.
usb disk is scsi over usb.
so just a different transport.
u are rite. ;)

ming

On Wed, 2005-05-11 at 23:26 +0200, Guennadi Liakhovetski wrote:
> Hello and thanks for the replies
> 
> On Wed, 11 May 2005, FUJITA Tomonori wrote:
> > The iSCSI protocol simply encapsulates the SCSI protocol into the
> > TCP/IP protocol, and carries packets over IP networks. You can handle
> ...
> 
> On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> > Actually, this is property not of iSCSI target itself, but of any SCSI target.
> > So, we implemented it as part of our SCSI target mid-level (SCST,
> > http://scst.sourceforge.net), therefore any target driver working over it will
> > automatically benefit from this feature. Unfortunately, currently available
> > only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
> > works not too reliable and only with very specific initiators). The published
> ...
> 
> The above confirms basically my understanding apart from one "minor" 
> confusion - I thought, that parallel to hardware solutions pure software 
> implementations were possible / being developed, like a driver, that 
> implements a SCSI LDD API on one side, and forwards packets to an IP 
> stack, say, over an ethernet card - on the initiator side. And a counter 
> part on the target side. Similarly to the USB mass-storage and storage 
> gadget drivers?
> 
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: several messages

[Vladislav Bolkhovitin]

Guennadi Liakhovetski wrote:
> Hello and thanks for the replies
> 
> On Wed, 11 May 2005, FUJITA Tomonori wrote:
> 
>>The iSCSI protocol simply encapsulates the SCSI protocol into the
>>TCP/IP protocol, and carries packets over IP networks. You can handle
> 
> ...
> 
> On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> 
>>Actually, this is property not of iSCSI target itself, but of any SCSI target.
>>So, we implemented it as part of our SCSI target mid-level (SCST,
>>http://scst.sourceforge.net), therefore any target driver working over it will
>>automatically benefit from this feature. Unfortunately, currently available
>>only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
>>works not too reliable and only with very specific initiators). The published
> 
> ...
> 
> The above confirms basically my understanding apart from one "minor" 
> confusion - I thought, that parallel to hardware solutions pure software 
> implementations were possible / being developed, like a driver, that 
> implements a SCSI LDD API on one side, and forwards packets to an IP 
> stack, say, over an ethernet card - on the initiator side. And a counter 
> part on the target side. Similarly to the USB mass-storage and storage 
> gadget drivers?

There is some confusion in the SCSI world between SCSI as a transport 
and SCSI as a commands set and software communication protocol, which 
works above the transport. So, you can implement SCSI transport at any 
software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
etc.) way, but if the SCSI message passing protocol is used overall 
system remains SCSI with all protocol obligations like task management. 
So, pure software SCSI solution is possible. BTW, there are pure 
hardware iSCSI implementations as well.

Vlad

> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Re: several messages

[Dmitry Yusupov]

On Wed, 2005-05-11 at 22:16 -0400, Ming Zhang wrote:
> iscsi is scsi over ip.

correction. iSCSI today has RFC at least for two transports - TCP/IP and
iSER/RDMA(in finalized progress) with RDMA over Infiniband or RNIC. And
I think people start writing initial draft for SCTP/IP transport...

>From this perspective, iSCSI certainly more advanced and matured
comparing to NBD variations. 

> usb disk is scsi over usb.
> so just a different transport.
> u are rite. ;)
> 
> ming
> 
> On Wed, 2005-05-11 at 23:26 +0200, Guennadi Liakhovetski wrote:
> > Hello and thanks for the replies
> > 
> > On Wed, 11 May 2005, FUJITA Tomonori wrote:
> > > The iSCSI protocol simply encapsulates the SCSI protocol into the
> > > TCP/IP protocol, and carries packets over IP networks. You can handle
> > ...
> > 
> > On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> > > Actually, this is property not of iSCSI target itself, but of any SCSI target.
> > > So, we implemented it as part of our SCSI target mid-level (SCST,
> > > http://scst.sourceforge.net), therefore any target driver working over it will
> > > automatically benefit from this feature. Unfortunately, currently available
> > > only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
> > > works not too reliable and only with very specific initiators). The published
> > ...
> > 
> > The above confirms basically my understanding apart from one "minor" 
> > confusion - I thought, that parallel to hardware solutions pure software 
> > implementations were possible / being developed, like a driver, that 
> > implements a SCSI LDD API on one side, and forwards packets to an IP 
> > stack, say, over an ethernet card - on the initiator side. And a counter 
> > part on the target side. Similarly to the USB mass-storage and storage 
> > gadget drivers?
> > 
> > Thanks
> > Guennadi
> > ---
> > Guennadi Liakhovetski
> > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Re[2]: ata over ethernet question

[James Bottomley]

On Wed, 2005-05-11 at 00:00 +0200, Guennadi Liakhovetski wrote:
> A follow up question - I recently used nbd to access a CD-ROM. It worked 
> nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> had to stop the client, the server, then replace the CD, re-start the 
> server, re-start the client... I thought about extending NBD to (better) 
> support removable media, but then you start thinking about all those 
> features that your local block device has that don't get exported over 
> NBD...

That's correct; NBD is basically just a remote data pipe type block
device.  It doesn't understand arbitrary packet commands.

> Now, my understanding (sorry, without looking at any docs - yet) is, that 
> iSCSI is (or at least should be) free from these limitations. So, does it 
> make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> just kept simple as it is or would it be completely superseeded by iSCSI, 
> or is there still something that NBD does that iSCSI wouldn't (easily) do?

Caveat: I've done quite a bit of work on nbd, so I'm biased.  However,
for what it does, nbd is extremely small, simple and efficient, so I
think we'd want a hole in our head to replace it with something as
complex and bloated as iSCSI---remember we'd need both a target and an
initiator to do what nbd does today.

However, there is room for improvement in nbd, notably the handling of
packet commands, which looks to be eminently doable in the current
infrastructure (this would basically make nbd a replicator for the linux
block system, and would probably necessitate some client side changes to
achieve).  If you have any thoughts in this direction, you could drop an
email to the maintainer.

James

Re: Re[2]: ata over ethernet question

[Dmitry Yusupov]

On Thu, 2005-05-12 at 14:52 -0400, James Bottomley wrote:
> On Wed, 2005-05-11 at 00:00 +0200, Guennadi Liakhovetski wrote:
> > A follow up question - I recently used nbd to access a CD-ROM. It worked 
> > nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> > had to stop the client, the server, then replace the CD, re-start the 
> > server, re-start the client... I thought about extending NBD to (better) 
> > support removable media, but then you start thinking about all those 
> > features that your local block device has that don't get exported over 
> > NBD...
> 
> That's correct; NBD is basically just a remote data pipe type block
> device.  It doesn't understand arbitrary packet commands.
> 
> > Now, my understanding (sorry, without looking at any docs - yet) is, that 
> > iSCSI is (or at least should be) free from these limitations. So, does it 
> > make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> > just kept simple as it is or would it be completely superseeded by iSCSI, 
> > or is there still something that NBD does that iSCSI wouldn't (easily) do?
> 
> Caveat: I've done quite a bit of work on nbd, so I'm biased.  However,
> for what it does, nbd is extremely small, simple and efficient, so I
> think we'd want a hole in our head to replace it with something as
> complex and bloated as iSCSI---remember we'd need both a target and an
> initiator to do what nbd does today.

oh, please! don't compare nbd and iSCSI this way...
iSCSI is an emerging SAN technology, and the only technology to compare
is FC.

> However, there is room for improvement in nbd, notably the handling of
> packet commands, which looks to be eminently doable in the current
> infrastructure (this would basically make nbd a replicator for the linux
> block system, and would probably necessitate some client side changes to
> achieve).  If you have any thoughts in this direction, you could drop an
> email to the maintainer.
> 
> James
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Re[2]: ata over ethernet question

[James Bottomley]

On Thu, 2005-05-12 at 12:05 -0700, Dmitry Yusupov wrote:
> oh, please! don't compare nbd and iSCSI this way...
> iSCSI is an emerging SAN technology, and the only technology to compare
> is FC.

Well, the question was whether iSCSI could replace nbd; It's rather
difficult to answer that question by comparing iSCSI to FC ...

But even projecting to iSCSI being totally mature, the amount of code
required to conform to the iSCSI standard is easily going to put it 10x
over the amount of code we have in nbd, principally because they're
aimed at solving different problems and nbd achieves a lot of
streamlining by being tied to the linux block subsystem instead of
trying to be a generic transport.

James

Re: SCSI/ISCSI, hardware/software

[Bryan Henderson]

>There is some confusion in the SCSI world between SCSI as a transport 
>and SCSI as a commands set and software communication protocol, which 
>works above the transport. So, you can implement SCSI transport at any 
>software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
>etc.) way, but if the SCSI message passing protocol is used overall 
>system remains SCSI with all protocol obligations like task management. 

The above doesn't really resolve the confusion, since it uses some 
ambiguous terms and constructions.  I'm not sure what it's supposed to 
say, but let me try to state in the terminology of the SCSI standards what 
SCSI is:

SCSI is a family of separate specifications.  Some are specifications of 
transports, and others are specifications of command sets (a layer above 
the transports).  A SCSI device must implement a SCSI transport spec and a 
SCSI command set spec -- and also contain a piece that actually does the 
work (e.g. a disk drive), the details of which aren't specified by SCSI.

Examples of SCSI transport specification are (I'm paraphrasing the names) 
parallel SCSI, Fibre Channel, and ISCSI.  Examples of command sets are the 
disk device command set and the tape device command set.

>So, pure software SCSI solution is possible. BTW, there are pure 
>hardware iSCSI implementations as well.

I don't think it's even meaningful to talk about whether an implementation 
is hardware or software.  The "pure hardware" implementations contain 
megabytes of software, which was written in languages like C, contains 
operating systems like Linux, and can be transmitted across a network and 
updated easily.  The "pure software" implementation involve kilograms of 
hardware in every SCSI command -- CPUs, power supplies, etc.

Not only that, but the "all hardware" ISCSI initiators people talk about, 
which are PCI cards with Ethernet jacks, are not complete initiators.  The 
computer you plug the card into, on which you run Linux and some 
application programs, is the initiator.  The card is just the 
ISCSI-specific core of it.

There's really two distinctions people mean to make when talking about 
hardware vs software:

1) Is it preassembled?  Can you lift it out of box whole, or do you have 
to acquire some special software and some more generic parts separately 
and manage their combination?

2) Does it involve a general purpose computing system, particularly one 
that you share with some other computing, or a faster special purpose 
dedicated one?

In the context of a Linux SCSI discussion, I'd just talk about how much of 
the implementation is in or above the Linux kernel, and how much is below. 
 And then we can say that ISCSI-specific function (initiator or target) 
can be implemented 1) entirely above the Linux kernel; 2) entirely inside 
the Linux kernel; 3) entirely below the Linux kernel; or 4) a combination 
of these.

Re: Re[2]: ata over ethernet question

[Dmitry Yusupov]

On Thu, 2005-05-12 at 15:15 -0400, James Bottomley wrote:
> On Thu, 2005-05-12 at 12:05 -0700, Dmitry Yusupov wrote:
> > oh, please! don't compare nbd and iSCSI this way...
> > iSCSI is an emerging SAN technology, and the only technology to compare
> > is FC.
> 
> Well, the question was whether iSCSI could replace nbd; It's rather
> difficult to answer that question by comparing iSCSI to FC ...

ok.
i'm just reacting on "bloated" wording. It really depends on
implementation and design. If you were talking about amount of code in
the kernel, than take a look on open-iscsi(just one file iscsi_tcp.c)
and IET where we doing a lot of management stuff in user-space. It is
not that much code in the kernel, really, but it is doing x10 times more
useful things comparing to nbd and yet compliant with RFC.

> But even projecting to iSCSI being totally mature, the amount of code
> required to conform to the iSCSI standard is easily going to put it 10x
> over the amount of code we have in nbd, principally because they're
> aimed at solving different problems and nbd achieves a lot of
> streamlining by being tied to the linux block subsystem instead of
> trying to be a generic transport.

yeah, generic transport, recovery levels, direct data placement for HW
HBAs, etc, etc... it is all *must* features for enterprise's SAN
deployment. So, yes, there is a price as usual.

Dmitry.

> James
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SCSI/ISCSI, hardware/software

[Douglas Gilbert]

Bryan Henderson wrote:
>>There is some confusion in the SCSI world between SCSI as a transport 
>>and SCSI as a commands set and software communication protocol, which 
>>works above the transport. So, you can implement SCSI transport at any 
>>software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
>>etc.) way, but if the SCSI message passing protocol is used overall 
>>system remains SCSI with all protocol obligations like task management. 
> 
> 
> The above doesn't really resolve the confusion, since it uses some 
> ambiguous terms and constructions.  I'm not sure what it's supposed to 
> say, but let me try to state in the terminology of the SCSI standards what 
> SCSI is:
> 
> SCSI is a family of separate specifications.  Some are specifications of 
> transports, and others are specifications of command sets (a layer above 
> the transports).  A SCSI device must implement a SCSI transport spec and a 
> SCSI command set spec -- and also contain a piece that actually does the 
> work (e.g. a disk drive), the details of which aren't specified by SCSI.
> 
> Examples of SCSI transport specification are (I'm paraphrasing the names) 
> parallel SCSI, Fibre Channel, and ISCSI.  Examples of command sets are the 
> disk device command set and the tape device command set.

Bryan,
This url might help to illustrate things:

http://t10.org/scsi-3.htm

Transports are below the yellow line, SCSI command sets
are above it.

Doug Gilbert

>>So, pure software SCSI solution is possible. BTW, there are pure 
>>hardware iSCSI implementations as well.
> 
> 
> I don't think it's even meaningful to talk about whether an implementation 
> is hardware or software.  The "pure hardware" implementations contain 
> megabytes of software, which was written in languages like C, contains 
> operating systems like Linux, and can be transmitted across a network and 
> updated easily.  The "pure software" implementation involve kilograms of 
> hardware in every SCSI command -- CPUs, power supplies, etc.
> 
> Not only that, but the "all hardware" ISCSI initiators people talk about, 
> which are PCI cards with Ethernet jacks, are not complete initiators.  The 
> computer you plug the card into, on which you run Linux and some 
> application programs, is the initiator.  The card is just the 
> ISCSI-specific core of it.
> 
> There's really two distinctions people mean to make when talking about 
> hardware vs software:
> 
> 1) Is it preassembled?  Can you lift it out of box whole, or do you have 
> to acquire some special software and some more generic parts separately 
> and manage their combination?
> 
> 2) Does it involve a general purpose computing system, particularly one 
> that you share with some other computing, or a faster special purpose 
> dedicated one?
> 
> In the context of a Linux SCSI discussion, I'd just talk about how much of 
> the implementation is in or above the Linux kernel, and how much is below. 
>  And then we can say that ISCSI-specific function (initiator or target) 
> can be implemented 1) entirely above the Linux kernel; 2) entirely inside 
> the Linux kernel; 3) entirely below the Linux kernel; or 4) a combination 
> of these.

Re: several messages

[Christoph Hellwig]

On Thu, May 12, 2005 at 11:32:12AM -0700, Dmitry Yusupov wrote:
> On Wed, 2005-05-11 at 22:16 -0400, Ming Zhang wrote:
> > iscsi is scsi over ip.
> 
> correction. iSCSI today has RFC at least for two transports - TCP/IP and
> iSER/RDMA(in finalized progress) with RDMA over Infiniband or RNIC. And
> I think people start writing initial draft for SCTP/IP transport...
> 
> >From this perspective, iSCSI certainly more advanced and matured
> comparing to NBD variations. 

It's for certainly much more complicated (in marketing speak that's usually
called advanced) but far less mature.

If you want network storage to just work use nbd.

Re: Re[2]: ata over ethernet question

[Christoph Hellwig]

On Thu, May 12, 2005 at 12:05:47PM -0700, Dmitry Yusupov wrote:
> oh, please! don't compare nbd and iSCSI this way...
> iSCSI is an emerging SAN technology, and the only technology to compare
> is FC.

who cares whether A is an emergeing <insert buzzword here> technology
when B is a lot simpler and just works for the customer.

I think you're a little too marketing driven here :)

Re: Re[2]: ata over ethernet question

[Christoph Hellwig]

On Thu, May 12, 2005 at 12:44:18PM -0700, Dmitry Yusupov wrote:
> i'm just reacting on "bloated" wording. It really depends on
> implementation and design. If you were talking about amount of code in
> the kernel, than take a look on open-iscsi(just one file iscsi_tcp.c)
> and IET where we doing a lot of management stuff in user-space. It is
> not that much code in the kernel, really, but it is doing x10 times more
> useful things comparing to nbd and yet compliant with RFC.

Keeping code out of the kernel is really nice, but that doesn't meant it
isn't bloat - the bloat is just in userland.

> yeah, generic transport, recovery levels, direct data placement for HW
> HBAs, etc, etc... it is all *must* features for enterprise's SAN
> deployment. So, yes, there is a price as usual.

I'm sure your marketing department can use all these buzzwords to sell
NICs to CTOs and CEOs, but else..

Re: SCSI/ISCSI, hardware/software

[Vladislav Bolkhovitin]

Bryan Henderson wrote:
>>There is some confusion in the SCSI world between SCSI as a transport 
>>and SCSI as a commands set and software communication protocol, which 
>>works above the transport. So, you can implement SCSI transport at any 
>>software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
>>etc.) way, but if the SCSI message passing protocol is used overall 
>>system remains SCSI with all protocol obligations like task management. 
> 
> 
> The above doesn't really resolve the confusion, since it uses some 
> ambiguous terms and constructions.  I'm not sure what it's supposed to 
> say, but let me try to state in the terminology of the SCSI standards what 
> SCSI is:
> 
> SCSI is a family of separate specifications.  Some are specifications of 
> transports, and others are specifications of command sets (a layer above 
> the transports).  A SCSI device must implement a SCSI transport spec and a 
> SCSI command set spec -- and also contain a piece that actually does the 
> work (e.g. a disk drive), the details of which aren't specified by SCSI.
> 
> Examples of SCSI transport specification are (I'm paraphrasing the names) 
> parallel SCSI, Fibre Channel, and ISCSI.  Examples of command sets are the 
> disk device command set and the tape device command set.

This is exactly what I wanted to say. Thanks for clarification

>>So, pure software SCSI solution is possible. BTW, there are pure 
>>hardware iSCSI implementations as well.
> 
> 
> I don't think it's even meaningful to talk about whether an implementation 
> is hardware or software.  The "pure hardware" implementations contain 
> megabytes of software, which was written in languages like C, contains 
> operating systems like Linux, and can be transmitted across a network and 
> updated easily.  The "pure software" implementation involve kilograms of 
> hardware in every SCSI command -- CPUs, power supplies, etc.
> 
> Not only that, but the "all hardware" ISCSI initiators people talk about, 
> which are PCI cards with Ethernet jacks, are not complete initiators.  The 
> computer you plug the card into, on which you run Linux and some 
> application programs, is the initiator.  The card is just the 
> ISCSI-specific core of it.

Such iSCSI card from a user point of view as well as for system running 
on a computer with it is just another SCSI card, not matter which 
transport it uses and how much software it runs onboard, so for they it 
doesn't differ from FC or parallel SCSI one, which I think you would not 
call a software unit. As usually, you only need appropriate driver for 
_SCSI_ subsystem.

> There's really two distinctions people mean to make when talking about 
> hardware vs software:
> 
> 1) Is it preassembled?  Can you lift it out of box whole, or do you have 
> to acquire some special software and some more generic parts separately 
> and manage their combination?
> 
> 2) Does it involve a general purpose computing system, particularly one 
> that you share with some other computing, or a faster special purpose 
> dedicated one?
> 
> In the context of a Linux SCSI discussion, I'd just talk about how much of 
> the implementation is in or above the Linux kernel, and how much is below. 
>  And then we can say that ISCSI-specific function (initiator or target) 
> can be implemented 1) entirely above the Linux kernel; 2) entirely inside 
> the Linux kernel; 3) entirely below the Linux kernel; or 4) a combination 
> of these.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: several messages

[Dmitry Yusupov]

On Fri, 2005-05-13 at 09:12 +0100, Christoph Hellwig wrote:
> On Thu, May 12, 2005 at 11:32:12AM -0700, Dmitry Yusupov wrote:
> > On Wed, 2005-05-11 at 22:16 -0400, Ming Zhang wrote:
> > > iscsi is scsi over ip.
> > 
> > correction. iSCSI today has RFC at least for two transports - TCP/IP and
> > iSER/RDMA(in finalized progress) with RDMA over Infiniband or RNIC. And
> > I think people start writing initial draft for SCTP/IP transport...
> > 
> > >From this perspective, iSCSI certainly more advanced and matured
> > comparing to NBD variations. 
> 
> It's for certainly much more complicated (in marketing speak that's usually
> called advanced) but far less mature.
> 
> If you want network storage to just work use nbd.

You could tell this to school's computer class teacher... Serious SAN
deployment will always be based either on FC or iSCSI for the reasons I
explained before.

I do not disagree, nbd is nice and simple and for sure has its own
deployment space.

Dmitry

Re: several messages

[Christoph Hellwig]

On Fri, May 13, 2005 at 08:04:16AM -0700, Dmitry Yusupov wrote:
> You could tell this to school's computer class teacher... Serious SAN
> deployment will always be based either on FC or iSCSI for the reasons I
> explained before.

Just FYI Steeleye ships a very successful clustering product that builds
on nbd.

Re: several messages

[Dmitry Yusupov]

On Fri, 2005-05-13 at 16:07 +0100, Christoph Hellwig wrote:
> On Fri, May 13, 2005 at 08:04:16AM -0700, Dmitry Yusupov wrote:
> > You could tell this to school's computer class teacher... Serious SAN
> > deployment will always be based either on FC or iSCSI for the reasons I
> > explained before.
> 
> Just FYI Steeleye ships a very successful clustering product that builds
> on nbd.

AFAIK, it is used for Data Replication purposes only. Correct me if I'm
wrong...

Re: Re[2]: ata over ethernet question

[Dmitry Yusupov]

On Fri, 2005-05-13 at 09:16 +0100, Christoph Hellwig wrote:
> On Thu, May 12, 2005 at 12:44:18PM -0700, Dmitry Yusupov wrote:
> > i'm just reacting on "bloated" wording. It really depends on
> > implementation and design. If you were talking about amount of code in
> > the kernel, than take a look on open-iscsi(just one file iscsi_tcp.c)
> > and IET where we doing a lot of management stuff in user-space. It is
> > not that much code in the kernel, really, but it is doing x10 times more
> > useful things comparing to nbd and yet compliant with RFC.
> 
> Keeping code out of the kernel is really nice, but that doesn't meant it
> isn't bloat - the bloat is just in userland.

well, "userland" == "bloatland" anyways... Multiple discovery methods,
configuration database, bunch of security protocols, etc... all this of
course will make it "slightly" :) bigger than nbd. But again, for a good
reason and better usefulness.

Dmitry

iSCSI vs. NBD (was Re: ata over ethernet question)

[Guennadi Liakhovetski]

On Thu, 12 May 2005, James Bottomley wrote:

> However, there is room for improvement in nbd, notably the handling of
> packet commands, which looks to be eminently doable in the current
> infrastructure (this would basically make nbd a replicator for the linux
> block system, and would probably necessitate some client side changes to
> achieve).  If you have any thoughts in this direction, you could drop an
> email to the maintainer.

Thanks, James

I'll try to get some (thoughts):-) BTW, who is the maintainer of nbd? No 
one in MAINTAINERS, in nbd.c only
 * Copyright 1997-2000 Pavel Machek <pavel@ucw.cz>
 * Parts copyright 2001 Steven Whitehouse <steve@chygwyn.com>
Is it Pavel then?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: iSCSI vs. NBD (was Re: ata over ethernet question)

[James Bottomley]

On Fri, 2005-05-13 at 20:50 +0200, Guennadi Liakhovetski wrote:
> I'll try to get some (thoughts):-) BTW, who is the maintainer of nbd? No 
> one in MAINTAINERS, in nbd.c only
>  * Copyright 1997-2000 Pavel Machek <pavel@ucw.cz>
>  * Parts copyright 2001 Steven Whitehouse <steve@chygwyn.com>
> Is it Pavel then?

Well, my copy of the MAINTAINERS file has this:

NETWORK BLOCK DEVICE
P:      Paul Clements
M:      Paul.Clements@steeleye.com
S:      Maintained

James

Re: iSCSI vs. NBD (was Re: ata over ethernet question)

[Guennadi Liakhovetski]

On Fri, 13 May 2005, James Bottomley wrote:

> Well, my copy of the MAINTAINERS file has this:
> 
> NETWORK BLOCK DEVICE
> P:      Paul Clements
> M:      Paul.Clements@steeleye.com
> S:      Maintained

Auch, `grep -i nbd MAINTAINERS`:-)))

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: SCSI/ISCSI, hardware/software

[Bryan Henderson]

>Such iSCSI card from a user point of view as well as for system running 
>on a computer with it is just another SCSI card, not matter which 
>transport it uses and how much software it runs onboard, so for they it 
>doesn't differ from FC or parallel SCSI one, which I think you would not 
>call a software unit.  As usually, you only need appropriate driver for 
>_SCSI_ subsystem.

The point I'd like to make is that _I_ would not call it a software unit 
or a hardware unit, because I don't think in most contexts (including that 
of this thread), it makes any difference which technology is used in the 
implementation.  What _does_ matter is 1) this card comes preassembled.  I 
don't have to find and load independently produced software onto it, or 
worry about interoperability.  2) It's below the Linux kernel, which means 
I won't need to mess with Linux applications or kernels except to load a 
low level SCSI driver.  It also means it doesn't place any load on my main 
CPU and probably goes faster than something implemented in or above my 
Linux kernel would.

And there's the separate point that it would be a misnomer to say that 
this card is an ISCSI initiator (it's only part of one); so even if the 
card itself can be called hardware, that still doesn't mean you can say 
you have a hardware ISCSI initiator.  Same is true of a parallel SCSI host 
adapter card.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click

Re: NBD (vs. iSCSI vs. EATA vs...)

[Guennadi Liakhovetski]

(Dropped most recepients of the original "ata over ethernet question" 
discussion, including lkml, from CC)

Hi

On Thu, 12 May 2005, James Bottomley wrote:
> However, there is room for improvement in nbd, notably the handling of
> packet commands, which looks to be eminently doable in the current
> infrastructure (this would basically make nbd a replicator for the linux
> block system, and would probably necessitate some client side changes to
> achieve).  If you have any thoughts in this direction, you could drop an
> email to the maintainer.

Ok, I came to a stage, when I start a server and a client, and then on the 
client side I can do open / close cycles, which would be respectively 
passed to the server, thus not keeping the server file(s) busy all the 
time. This allows, e.g., to export a CD-ROM, mount / umount it, eject it 
(not yet over nbd), insert a new one, mount, etc. Also, open errors are 
passed back, so, you get a nice "no medium" error on the client side 
without a CD.

IIUC, the suggestion from James to implement ATAPI translates in 
user-space terms to implementing respective ioctl's, because this is what 
nbd gets, and this is what nbd-server has to reproduce.

So, the work is not finished, at least some ioctl's are needed. I had to 
modify the kernel driver, and both the server and the client apps. No 
backward compatibility, sorry:-( What I'd like to do now is to ask - is 
there an interest in such changes to be integrated in the kernel / 
nbd-package. If there is one - what are your wishes? If we get that far, 
what should I post here - only the kernel patch, and user-space separately 
to sf (I think?), or both?

(see also my another post today to linux-scsi)

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: NBD (vs. iSCSI vs. EATA vs...)

[James Bottomley]

On Thu, 2005-05-26 at 01:41 +0200, Guennadi Liakhovetski wrote:
> IIUC, the suggestion from James to implement ATAPI translates in 
> user-space terms to implementing respective ioctl's, because this is what 
> nbd gets, and this is what nbd-server has to reproduce.

Actually, no, that's not what I was thinking.

nbd currently only understands the block layer REQ_CMD (which are basic
READ/WRITE commands) if it understood REQ_BLOCK_PC it would probably be
sufficient a transport for sending the packet commands that are
necessary to drive a remote device.

The idea being that you could then do

cdrom - nbd <--\/\-->remote - ide/scsi - CD-ROM device

to drive an actual cdrom.  There are some pieces missing in this vision
(like how to attach cdrom to nbd) but it should be doable in principle.

James

Re: NBD (vs. iSCSI vs. EATA vs...)

[Guennadi Liakhovetski]

(I wonder why my emails still haven't appeared on linux-scsi?)

On Wed, 25 May 2005, James Bottomley wrote:

> On Thu, 2005-05-26 at 01:41 +0200, Guennadi Liakhovetski wrote:
> > IIUC, the suggestion from James to implement ATAPI translates in 
> > user-space terms to implementing respective ioctl's, because this is what 
> > nbd gets, and this is what nbd-server has to reproduce.
> 
> Actually, no, that's not what I was thinking.
> 
> nbd currently only understands the block layer REQ_CMD (which are basic
> READ/WRITE commands) if it understood REQ_BLOCK_PC it would probably be
> sufficient a transport for sending the packet commands that are
> necessary to drive a remote device.

Aha, I see...

> The idea being that you could then do
> 
> cdrom - nbd <--\/\-->remote - ide/scsi - CD-ROM device
> 
> to drive an actual cdrom.  There are some pieces missing in this vision
> (like how to attach cdrom to nbd) but it should be doable in principle.

You need the cdrom driver to convert user requests to cdrom commands, 
which with your scheme would be collected by nbd from the block queue, 
passed to nbd-server to be translated back to user-space requests...

Well, doesn't this mean, that you would have to look, if the server 
exports a SCSI cdrom, nbd would have to add a SCSI bus to the system and a 
cdrom on it, respectively for ide. So, you would just get a new srX / sdX 
/ hdX / loopX? Or you would have to write a new cdrom driver for the "nbd 
bus", or teach one of existing to work over nbd (ide or scsi?). Wouldn't 
either of these 3 possibilities be too complicated? The third one could be 
not too difficult, but you would have to choose a victim - one of ide or 
scsi cdrom drivers. And what do you do with discs?...

OTOH, if you just pass all user-space requests to the server and just pass 
back return code, you wouldn't need high-level drivers on the host side. 
You would miss anything, comming not from userspace - any non REQ_CMD 
requests from filesystems. But those we could just add separately - for 
that you don't need cdrom / sd / hd drivers, right?

So, just pass ioctl()'s directly and convert the rest (filesystem) block 
requests separately - wouldn't it be easier and avoid any modifications to 
other drivers?

Thanks
Guennadi
---
Guennadi Liakhovetski