Re: [PATCH 2.6.25.1] Add scsi_execute_async_fifo()

Re: [PATCH 2.6.25.1] Add scsi_execute_async_fifo()

[Vladislav Bolkhovitin]

James Bottomley wrote:
> On Fri, 2008-05-02 at 18:06 +0200, Bart Van Assche wrote:
>> On Fri, May 2, 2008 at 5:55 PM, Christoph Hellwig <hch@infradead.org> wrote:
>>> On Fri, May 02, 2008 at 05:53:22PM +0200, Bart Van Assche wrote:
>>>> Regarding out-of-tree modules: this is just a preparatory step before
>>>> submitting SCST for inclusion in the mainstream kernel.
>>> And what crackpipe did you smoke to thing we'd put duplicated target
>>> framework in?
>> Why are you so aggressive ? I didn't insult you in any way.
>>
>> Regarding inclusion of SCSI target code in the mainline, this subject
>> has already been discussed extensively in the past
>> (http://lkml.org/lkml/2008/1/23/134). The conclusion was clear: SCST
>> is faster than any other existing iSCSI target for Linux (IET, STGT,
>> LIO), stable, well maintained and the most standards compliant target.
>> Why do you want to reopen this discussion ?
> 
> That's an interesting rewrite of history.  The evidence you presented
> showed fairly identical results apart from on one contrived IB benchmark
> that couldn't directly compare the two.
> 
> I'm also on record in the thread saying that was insufficient proof for
> me to justify throwing STGT out and replacing it with SCST.

James, why do you keep ignoring important points, written by me in that 
e-mail: http://lkml.org/lkml/2008/1/30/178?

Namely:

1. Solid architecture of SCST is inherently more simple than distributed 
user/kernel space processing, when kernel behaves under control of user 
space, used in STGT, and allows to get better results with less effort. 
Better in all aspects: simplicity (hence, maintainability), reliability 
and performance. Linux once made step away from microkernel based design 
and that was for really good reasons.

2. Zero-copy operations with page cache will halve processing latency on 
high speed links, like InfiniBand, and it is impossible to implement 
that in a sane way with STGT approach, while for SCST it can be 
implemented simply and naturally.

Vlad

SCSI target subsystem

[Vladislav Bolkhovitin]

[-- Attachment #1: Type: text/plain, Size: 3728 bytes --]

James Bottomley wrote:
> On Fri, 2008-05-02 at 18:06 +0200, Bart Van Assche wrote:
>> James Bottomley clearly expressed in that thread that he doesn't want
>> to maintain two SCSI target frameworks. So what I propose is that SCST
>> is included in the mainline and afterwards that it is evaluated
>> whether or not it is desirable to keep other target code in the
>> mainline kernel.
> 
> That's hardly sufficient.  STGT is already in use.  Their either has to
> be a migration path or, the preferred option, take the pieces of SCST
> that are actual improvements and embed them in STGT.

Actually, between SCSI initiator and target subsystems there is almost 
*nothing* in common. This claim, at first glance, looks pretty wrong, 
because both serve SCSI, so they must have a lot common. But look deeper 
and you quickly find out that the majority of functionality as well as 
data they use are dedicated for each subsystem, not shared.

Just look at SCST/qla2x00t/(changes done in the initiator qla2xxx driver 
to support target mode, patch attached): 90% of changes is adding 
callbacks for external target add-on, the rest is support for older, 
than 2.6.17, kernels and sysfs magic. Note, no data are common between 
initiator and target parts in the meaning that they both use them.

Then look at SCST (http://scst.sf.net). It implements complete 
pass-through SCSI support and look how it interacts with initiator SCSI 
subsystem. It calls only 2 functions: FIFO version of 
scsi_execute_async() (original scsi_execute_async() provides 
unacceptable LIFO commands order) and scsi_reset_provider() for task 
management. And there is only one common variable: struct scsi_device. 
That's all! In other storage modes (FILEIO/BLOCKIO) there is nothing 
common with SCSI initiator subsystem at all.

Finally, try to find out in SCST any duplicated functionality.

Now, let's look how SCSI target/initiator integration currently done in 
the kernel. For me it looks pretty artificial. For example, if I make a 
general purpose kernel, for which 1% of users would run target mode, I 
would have to enable as module "SCSI target support" as well as SCSI 
target support for transport attributes. Now 99% of users of my kernel, 
who don't need SCSI target, but need SCSI initiator drivers, would have 
to have scsi_tgt loaded, because transport attribute drivers would 
depend on it:

# lsmod
Module                  Size  Used by
qla2xxx               130844  0
firmware_class          8064  1 qla2xxx
scsi_transport_fc      40900  1 qla2xxx
scsi_tgt               12196  1 scsi_transport_fc
brd                     6924  0
xfs                   511280  1
dm_mirror              24368  0
dm_mod                 51148  1 dm_mirror
uhci_hcd               21400  0
sg                     31784  0
e1000                 114536  0
pcspkr                  3328  0

No target functionality needed, but target mode subsystem is needed. Is 
it a good design?

I wrote all above to support my at first glance shocking conclusion that 
SCSI target subsystem is completely new subsystem of the kernel and it 
should live on its own with its own maintainer! This is the same as with 
the current interaction between SCSI and block subsystems in the kernel: 
SCSI uses block's functionality, but that doesn't mean that block and 
SCSI are the same subsystem.

Thus, how IMHO initiator and target drivers should be written:

  - All initiator drivers should live in the SCSI initiator subsystem 
(aka current SCSI subsystem) only, the same as today.

  - All target drivers should live in the SCSI target subsystem only and 
be either add-ons to initiator drivers, like, e.g., qla2x00t, or be a 
complete driver, like, e.g., iSCSI-SCST.

Vlad


[-- Attachment #2: qla2x00t.diff.gz --]
[-- Type: application/x-gzip, Size: 17294 bytes --]

Re: SCSI target subsystem

[Bart Van Assche]

On Fri, May 2, 2008 at 8:17 PM, Vladislav Bolkhovitin <vst@vlnb.net> wrote:
> I wrote all above to support my at first glance shocking conclusion that
> SCSI target subsystem is completely new subsystem of the kernel and it
> should live on its own with its own maintainer! This is the same as with the
> current interaction between SCSI and block subsystems in the kernel: SCSI
> uses block's functionality, but that doesn't mean that block and SCSI are
> the same subsystem.

Hello Tomo,

Due to the IET and STGT projects you have a lot of experience with
implementing SCSI target frameworks. What is your opinion about how a
kernel space SCSI target framework should fit in the Linux kernel ?

Bart.

Re: SCSI target subsystem

[Matthew Wilcox]

On Sat, May 03, 2008 at 11:41:41AM +0200, Bart Van Assche wrote:
> On Fri, May 2, 2008 at 8:17 PM, Vladislav Bolkhovitin <vst@vlnb.net> wrote:
> > I wrote all above to support my at first glance shocking conclusion that
> > SCSI target subsystem is completely new subsystem of the kernel and it
> > should live on its own with its own maintainer! This is the same as with the
> > current interaction between SCSI and block subsystems in the kernel: SCSI
> > uses block's functionality, but that doesn't mean that block and SCSI are
> > the same subsystem.
> 
> Hello Tomo,
> 
> Due to the IET and STGT projects you have a lot of experience with
> implementing SCSI target frameworks. What is your opinion about how a
> kernel space SCSI target framework should fit in the Linux kernel ?

Bart, what is your role in the SCST project?  You don't seem to have
contributed any code to it (going by the SVN logs on sourceforge), and
your questions and suggestions seem to be those of someone not familiar
with the code.  If you're not a developer, it might be more helpful for
you to step back and let Vladislav handle this.

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: SCSI target subsystem

[Bart Van Assche]

On Sat, May 3, 2008 at 11:53 AM, Matthew Wilcox <matthew@wil.cx> wrote:
> Bart, what is your role in the SCST project?  You don't seem to have
> contributed any code to it (going by the SVN logs on sourceforge), and
> your questions and suggestions seem to be those of someone not familiar
> with the code.  If you're not a developer, it might be more helpful for
> you to step back and let Vladislav handle this.

I'd like to see this discussion focus on the technical issues. Why are
people constantly throwing up political arguments in this discussion ?

To answer your question: as you noticed, I did not yet contribute any
code to the SCST project. I'm an iSCSI/iSER/SRP user. I'd like to have
a SCSI target framework available on Linux that is as fast as
possible, as reliable as possible, as standards compliant as possible,
well maintained and for which all kernel code is integrated in the
mainstream Linux kernel. I'm probably sharing this desire with all
STGT, IET, SCST and LIO users. Do you consider the opinion of SCSI
target framework users irrelevant in this discussion ?

Bart.

Re: SCSI target subsystem

[Matthew Wilcox]

On Sat, May 03, 2008 at 12:39:19PM +0200, Bart Van Assche wrote:
> To answer your question: as you noticed, I did not yet contribute any
> code to the SCST project. I'm an iSCSI/iSER/SRP user. I'd like to have
> a SCSI target framework available on Linux that is as fast as
> possible, as reliable as possible, as standards compliant as possible,
> well maintained and for which all kernel code is integrated in the
> mainstream Linux kernel. I'm probably sharing this desire with all
> STGT, IET, SCST and LIO users. Do you consider the opinion of SCSI
> target framework users irrelevant in this discussion ?

Yes.

Once Vladislav is ready for his project to be merged, let him approach
us and ask.  I don't know if he considers his project ready for merging.
Did he ask you for his help?  It's generally considered rather rude to
merge someone else's code without their consent.  The way you tried to
get it merged was unhelpful and, if Vladislav were less skillful a
communicator, could have actively hindered getting it merged.

The correct way to have done this would have been to start a thread on
the scst mailing list asking whether the developers felt it ready to
merge.  Then *leave it to them*.

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: SCSI target subsystem

[Bart Van Assche]

On Sat, May 3, 2008 at 3:28 PM, Matthew Wilcox <matthew@wil.cx> wrote:
> Once Vladislav is ready for his project to be merged, let him approach
> us and ask.  I don't know if he considers his project ready for merging.
> Did he ask you for his help?  It's generally considered rather rude to
> merge someone else's code without their consent.  The way you tried to
> get it merged was unhelpful and, if Vladislav were less skillful a
> communicator, could have actively hindered getting it merged.

You'd better verify your assumptions before you take them for truth. I
asked Vladislav for permission by private e-mail before I posted the
scsi_execute_async_fifo() patch. Did try I to merge this code without
Vladislav's consent ? No. Anyone can see that I kept Vladislav and the
scst-devel mailing list in CC.

Let us return to the actual topic of this thread. Does the above mean
that you agree with inclusion of SCST in the mainline kernel ? It
still has to be decided who will maintain the mainline kernel SCSI
target code. In the past James wrote that he does not really have the
time for this. Until now the only one who volunteered to maintain such
a subsystem is Vladislav.

Bart.

Re: SCSI target subsystem

[Vladislav Bolkhovitin]

Matthew Wilcox wrote:
> On Sat, May 03, 2008 at 11:41:41AM +0200, Bart Van Assche wrote:
>> On Fri, May 2, 2008 at 8:17 PM, Vladislav Bolkhovitin <vst@vlnb.net> wrote:
>>> I wrote all above to support my at first glance shocking conclusion that
>>> SCSI target subsystem is completely new subsystem of the kernel and it
>>> should live on its own with its own maintainer! This is the same as with the
>>> current interaction between SCSI and block subsystems in the kernel: SCSI
>>> uses block's functionality, but that doesn't mean that block and SCSI are
>>> the same subsystem.
>> Hello Tomo,
>>
>> Due to the IET and STGT projects you have a lot of experience with
>> implementing SCSI target frameworks. What is your opinion about how a
>> kernel space SCSI target framework should fit in the Linux kernel ?
> 
> Bart, what is your role in the SCST project?  You don't seem to have
> contributed any code to it (going by the SVN logs on sourceforge), and
> your questions and suggestions seem to be those of someone not familiar
> with the code.  If you're not a developer, it might be more helpful for
> you to step back and let Vladislav handle this.

Consider Bart as a member of SCST team at the moment responsible for 
Linux kernel inclusion.

Vlad

Re: SCSI target subsystem

[Vladislav Bolkhovitin]

Vladislav Bolkhovitin wrote:
> James Bottomley wrote:
>> On Fri, 2008-05-02 at 18:06 +0200, Bart Van Assche wrote:
>>> James Bottomley clearly expressed in that thread that he doesn't want
>>> to maintain two SCSI target frameworks. So what I propose is that SCST
>>> is included in the mainline and afterwards that it is evaluated
>>> whether or not it is desirable to keep other target code in the
>>> mainline kernel.
>> That's hardly sufficient.  STGT is already in use.  Their either has to
>> be a migration path or, the preferred option, take the pieces of SCST
>> that are actual improvements and embed them in STGT.
> 
> Actually, between SCSI initiator and target subsystems there is almost 
> *nothing* in common. This claim, at first glance, looks pretty wrong, 
> because both serve SCSI, so they must have a lot common. But look deeper 
> and you quickly find out that the majority of functionality as well as 
> data they use are dedicated for each subsystem, not shared.
> 
> Just look at SCST/qla2x00t/(changes done in the initiator qla2xxx driver 
> to support target mode, patch attached): 90% of changes is adding 
> callbacks for external target add-on, the rest is support for older, 
> than 2.6.17, kernels and sysfs magic. Note, no data are common between 
> initiator and target parts in the meaning that they both use them.

Perhaps, I should elaborate more on this to eliminate possible 
misunderstanding. Of course, both main initiator driver and target 
add-on driver directly use the same hardware, so they share all internal 
hardware-related data, e.g. hardware_lock, but this doesn't matter for 
our topic, because this sharing is on the different level. All such data 
are hardware specific, hence different hardware have different sets of 
such data, hence it is impractical to find something common in them to 
expose as a common interface, which all initiator drivers should expose 
to its target add-ons: the interface would be more complicated than 
direct implementation in each particular case.

> Then look at SCST (http://scst.sf.net). It implements complete 
> pass-through SCSI support and look how it interacts with initiator SCSI 
> subsystem. It calls only 2 functions: FIFO version of 
> scsi_execute_async() (original scsi_execute_async() provides 
> unacceptable LIFO commands order) and scsi_reset_provider() for task 
> management. And there is only one common variable: struct scsi_device. 
> That's all! In other storage modes (FILEIO/BLOCKIO) there is nothing 
> common with SCSI initiator subsystem at all.
> 
> Finally, try to find out in SCST any duplicated functionality.
> 
> Now, let's look how SCSI target/initiator integration currently done in 
> the kernel. For me it looks pretty artificial. For example, if I make a 
> general purpose kernel, for which 1% of users would run target mode, I 
> would have to enable as module "SCSI target support" as well as SCSI 
> target support for transport attributes. Now 99% of users of my kernel, 
> who don't need SCSI target, but need SCSI initiator drivers, would have 
> to have scsi_tgt loaded, because transport attribute drivers would 
> depend on it:
> 
> # lsmod
> Module                  Size  Used by
> qla2xxx               130844  0
> firmware_class          8064  1 qla2xxx
> scsi_transport_fc      40900  1 qla2xxx
> scsi_tgt               12196  1 scsi_transport_fc
> brd                     6924  0
> xfs                   511280  1
> dm_mirror              24368  0
> dm_mod                 51148  1 dm_mirror
> uhci_hcd               21400  0
> sg                     31784  0
> e1000                 114536  0
> pcspkr                  3328  0
> 
> No target functionality needed, but target mode subsystem is needed. Is 
> it a good design?
> 
> I wrote all above to support my at first glance shocking conclusion that 
> SCSI target subsystem is completely new subsystem of the kernel and it 
> should live on its own with its own maintainer! This is the same as with 
> the current interaction between SCSI and block subsystems in the kernel: 
> SCSI uses block's functionality, but that doesn't mean that block and 
> SCSI are the same subsystem.
> 
> Thus, how IMHO initiator and target drivers should be written:
> 
>   - All initiator drivers should live in the SCSI initiator subsystem 
> (aka current SCSI subsystem) only, the same as today.
> 
>   - All target drivers should live in the SCSI target subsystem only and 
> be either add-ons to initiator drivers, like, e.g., qla2x00t, or be a 
> complete driver, like, e.g., iSCSI-SCST.
> 
> Vlad
> 

Re: SCSI target subsystem

[Bart Van Assche]

On Sat, May 3, 2008 at 12:39 PM, Bart Van Assche
<bart.vanassche@gmail.com> wrote:
> To answer your question: as you noticed, I did not yet contribute any
> code to the SCST project. I'm an iSCSI/iSER/SRP user. I'd like to have
> a SCSI target framework available on Linux that is as fast as
> possible, as reliable as possible, as standards compliant as possible,
> well maintained and for which all kernel code is integrated in the
> mainstream Linux kernel. I'm probably sharing this desire with all
> STGT, IET, SCST and LIO users.

Hello Matthew,

To be complete I should have mentioned that there already exists an
FCoE target implementation for SCST. I'm still learning about FCoE.
What I read about it is that FCoE has more to offer with regard to
storage management than iSCSI/iSER/SRP ?

See also: http://www.open-fcoe.org/openfc/wiki/index.php/Quickstart

Bart.