ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Andrew Patterson wrote:
> I believe there is a common understanding that IOCTL's are bad and
> should be avoided. See:
> 
> http://lkml.org/lkml/2001/5/20/81

Yep.  Linus's rant here reflects not only his opinion, but general 
consensus, I feel.


> Yes, CSMI should have had more Linux input when it was developed.  The
> no-new IOCTL policy certainly came as a surprise to the authors. Still,
> there doesn't seem to be any other usable cross-platform interface that
> is acceptable to the linux community (or at least to Christoph)'s corner

Beyond Linus's rant, ioctls have a practical limitation, too:  because 
they are untyped, we have to deal with stuff like the 32<->64 compat 
ioctl thunking.

Consider what an ioctl is, overall:  a domain-specific "do this 
operation" interface.  Which, further, is nothing but a wrapping of a 
"send message" + "receive response" interface.  There are several ways 
to do this in Linux:

* block driver.  a block driver is nothing but a message queue.  This is 
why James has suggested implementing SMP as a block driver.  People get 
stuck into thinking "block driver == block device", which is wrong.  The 
Linux block layer is nothing but a message queueing interface.

* netlink.  You connect to <an object> and send/receive messages.  Not 
strictly limited to networking, as this is used in some areas of the 
kernel now for generic event delivery.

* char driver.  Poor man's netlink.  Unless its done right, it suffers 
from the same binary problems as ioctls.  I don't recommend this path.

* sysfs.  This has no inherent message/queue properties by itself, and 
is less structured than blockdrvr or netlink, so dealing with more than 
one outstanding operation at a time requires some coding.

sysfs's attractiveness is in its ease of use.  It works with standard 
Unix filesystem tools.  You don't need to use a library to read 
information, cat(1) often suffices.  sysfs, since its normal ASCII 
(UTF8), also has none of the annoying 32<->64 compatibility issues that 
ioctls suffer from.

Which is best?  I don't have a good answer.  Largely depends on the 
situation, particularly queueing needs.  Networking and storage are 
rapidly converging into "messaging", so the situation is highly fluid in 
any case.  Coming from a networking background, I sorta lean towards the 
solution noone has attempted yet:  netlink.


> of it).  My personal preference is to hide this stuff in a library, so
> the kernel implementation is hidden. But even a library needs an
> underlying kernel implementation.

Strongly agree here.  libc shelters userspace from [most] kernel 
changes, by exporting syscalls in a standard manner.  alsa-lib was 
created to shelter userspace from current and future changes in the 
kernel audio interface.  libsdi is quite reasonable.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Mike Christie]

Jeff Garzik wrote:
> 
> Which is best?  I don't have a good answer.  Largely depends on the 
> situation, particularly queueing needs.  Networking and storage are 
> rapidly converging into "messaging", so the situation is highly fluid in 
> any case.  Coming from a networking background, I sorta lean towards the 
> solution noone has attempted yet:  netlink.

Dimitry and Alex did netlink for scsi_tranport_iscsi.c in scsi-misc. 
Which reminds me of some of the problems they discovered. See here
http://marc.theaimsgroup.com/?l=linux-netdev&m=111273099708516&w=2

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Douglas Gilbert]

Jeff Garzik wrote:
> Andrew Patterson wrote:
> 
>> I believe there is a common understanding that IOCTL's are bad and
>> should be avoided. See:
>>
>> http://lkml.org/lkml/2001/5/20/81
> 
> 
> Yep.  Linus's rant here reflects not only his opinion, but general
> consensus, I feel.

Jeff,
ioctls represent the most direct, unimpeded (by OS
policy) route between the user space and a driver,
potentially a few levels down a driver stack.
I see it as a control issue, in one corner there are
Microsoft and Linux while in the other there are
the hardware vendors. OS vendors come out with
ioctl replacements but can't resist enforcing policy.

As for type safety in linux, I stopped taking that
seriously when practicality of having C++ code
in the kernel was killed by "struct class".

>> Yes, CSMI should have had more Linux input when it was developed.  The
>> no-new IOCTL policy certainly came as a surprise to the authors. Still,
>> there doesn't seem to be any other usable cross-platform interface that
>> is acceptable to the linux community (or at least to Christoph)'s corner
> 
> 
> Beyond Linus's rant, ioctls have a practical limitation, too:  because
> they are untyped, we have to deal with stuff like the 32<->64 compat
> ioctl thunking.

Do you know where there are some clear guidelines
on the use of pointers in a structure passed to an
ioctl to lessen (or bypass) 32<->64 compat ioctl
thunking?

> Consider what an ioctl is, overall:  a domain-specific "do this
> operation" interface.  Which, further, is nothing but a wrapping of a
> "send message" + "receive response" interface.  There are several ways
> to do this in Linux:
> 
> * block driver.  a block driver is nothing but a message queue.  This is
> why James has suggested implementing SMP as a block driver.  People get
> stuck into thinking "block driver == block device", which is wrong.  The
> Linux block layer is nothing but a message queueing interface.
> 
> * netlink.  You connect to <an object> and send/receive messages.  Not
> strictly limited to networking, as this is used in some areas of the
> kernel now for generic event delivery.
> 
> * char driver.  Poor man's netlink.  Unless its done right, it suffers
> from the same binary problems as ioctls.  I don't recommend this path.
> 
> * sysfs.  This has no inherent message/queue properties by itself, and
> is less structured than blockdrvr or netlink, so dealing with more than
> one outstanding operation at a time requires some coding.
> 
> sysfs's attractiveness is in its ease of use.  It works with standard
> Unix filesystem tools.  You don't need to use a library to read
> information, cat(1) often suffices.  sysfs, since its normal ASCII
> (UTF8), also has none of the annoying 32<->64 compatibility issues that
> ioctls suffer from.

... and on the other side for sysfs are the loss of
layered error reporting, inability to supply auxiliary
attributes such as a request timeout and the possibility
that write()s will either be limited in size or segmented.

> Which is best?  I don't have a good answer.  Largely depends on the
> situation, particularly queueing needs.  Networking and storage are
> rapidly converging into "messaging", so the situation is highly fluid in
> any case.  Coming from a networking background, I sorta lean towards the
> solution noone has attempted yet:  netlink.

damn, we agree :-)

All in all, this is a good summary of the options available.

>> of it).  My personal preference is to hide this stuff in a library, so
>> the kernel implementation is hidden. But even a library needs an
>> underlying kernel implementation.
> 
> 
> Strongly agree here.  libc shelters userspace from [most] kernel
> changes, by exporting syscalls in a standard manner.  alsa-lib was
> created to shelter userspace from current and future changes in the
> kernel audio interface.  libsdi is quite reasonable.

Doug Gilbert

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Douglas Gilbert wrote:
> ioctls represent the most direct, unimpeded (by OS
> policy) route between the user space and a driver,
> potentially a few levels down a driver stack.

With the inevitable result that most ioctl code is poorly written, and 
causes bugs and special case synchronization problems :)  Driver writers 
love to stuff way too much stuff into ioctls, without thinking about 
overall design.


> I see it as a control issue, in one corner there are
> Microsoft and Linux while in the other there are
> the hardware vendors. OS vendors come out with
> ioctl replacements but can't resist enforcing policy.
> 
> As for type safety in linux, I stopped taking that
> seriously when practicality of having C++ code
> in the kernel was killed by "struct class".

C++ was never ever meant to work with the kernel.


>>>Yes, CSMI should have had more Linux input when it was developed.  The
>>>no-new IOCTL policy certainly came as a surprise to the authors. Still,
>>>there doesn't seem to be any other usable cross-platform interface that
>>>is acceptable to the linux community (or at least to Christoph)'s corner
>>
>>
>>Beyond Linus's rant, ioctls have a practical limitation, too:  because
>>they are untyped, we have to deal with stuff like the 32<->64 compat
>>ioctl thunking.
> 
> 
> Do you know where there are some clear guidelines
> on the use of pointers in a structure passed to an
> ioctl to lessen (or bypass) 32<->64 compat ioctl
> thunking?

Its impossible.  If you pass pointers, you need to thunk.  (Not even 
passing pointers via sysfs can avoid this.)  Consider running a 32-bit 
app (with 32-bit pointers and 32-bit ABI) on a 64-bit kernel.


>>Consider what an ioctl is, overall:  a domain-specific "do this
>>operation" interface.  Which, further, is nothing but a wrapping of a
>>"send message" + "receive response" interface.  There are several ways
>>to do this in Linux:
>>
>>* block driver.  a block driver is nothing but a message queue.  This is
>>why James has suggested implementing SMP as a block driver.  People get
>>stuck into thinking "block driver == block device", which is wrong.  The
>>Linux block layer is nothing but a message queueing interface.
>>
>>* netlink.  You connect to <an object> and send/receive messages.  Not
>>strictly limited to networking, as this is used in some areas of the
>>kernel now for generic event delivery.
>>
>>* char driver.  Poor man's netlink.  Unless its done right, it suffers
>>from the same binary problems as ioctls.  I don't recommend this path.
>>
>>* sysfs.  This has no inherent message/queue properties by itself, and
>>is less structured than blockdrvr or netlink, so dealing with more than
>>one outstanding operation at a time requires some coding.
>>
>>sysfs's attractiveness is in its ease of use.  It works with standard
>>Unix filesystem tools.  You don't need to use a library to read
>>information, cat(1) often suffices.  sysfs, since its normal ASCII
>>(UTF8), also has none of the annoying 32<->64 compatibility issues that
>>ioctls suffer from.
> 
> 
> ... and on the other side for sysfs are the loss of
> layered error reporting,

Only for the most simple interfaces.  sysfs is for exporting kernel data 
structures to userspace, using read(2) and write(2).  You can quite 
literally accomplish anything with that.  sysfs could export an event 
directory entry, and a response directory entry.  The response dirent 
could provide all the error reporting you could ever want.  That's just 
a 2-second off-the-cuff example.


> inability to supply auxiliary
> attributes such as a request timeout and the possibility
> that write()s will either be limited in size or segmented.

This entirely depends on the interface you define.  These problems are 
all surmountable.

Note, I'm _not_ suggesting this is the best route for an SMP interface, 
just stating sysfs generalities.  sysfs is flexible enough that it could 
completely replace SG_IO (though that would not be a wise idea).

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 02:19, Jeff Garzik wrote:
> With the inevitable result that most ioctl code is poorly written, and 
> causes bugs and special case synchronization problems :)  Driver writers 
> love to stuff way too much stuff into ioctls, without thinking about 
> overall design.

Could you please use adjectives, like "some" or "all" before
"Device writers"?  Could you also use a qualifier like, "excluding me",
etc.  Thanks.

> Only for the most simple interfaces.  sysfs is for exporting kernel data 
> structures to userspace, using read(2) and write(2).  You can quite 
> literally accomplish anything with that.  sysfs could export an event 
> directory entry, and a response directory entry.  The response dirent 
> could provide all the error reporting you could ever want.  That's just 
> a 2-second off-the-cuff example.

For the 3-second "off-the-cuff example" see how the "smp_portal" sysfs
binary attribute works in drivers/scsi/sas/sas_expander.c at the bottom
of the file, whereby user space first writes and then reads as much data
it is expecting.

(Thus there is _no_ hardcoding of the amount of data SMP request can
return (as is done in SDI).)

Here are the tiny functions for refresh:

/* ---------- SMP portal ---------- */

static ssize_t smp_portal_write(struct kobject *kobj, char *buf, loff_t offs,
				size_t size)
{
	struct domain_device *dev = to_dom_device(kobj);
	struct expander_device *ex = &dev->ex_dev;

	if (offs != 0)
		return -EFBIG;
	else if (size == 0)
		return 0;

	down_interruptible(&ex->smp_sema);
	if (ex->smp_req)
		kfree(ex->smp_req);
	ex->smp_req = kzalloc(size, GFP_USER);
	if (!ex->smp_req) {
		up(&ex->smp_sema);
		return -ENOMEM;
	}
	memcpy(ex->smp_req, buf, size);
	ex->smp_req_size = size;
	ex->smp_portal_pid = current->pid;
	up(&ex->smp_sema);

	return size;
}

static ssize_t smp_portal_read(struct kobject *kobj, char *buf, loff_t offs,
			       size_t size)
{
	struct domain_device *dev = to_dom_device(kobj);
	struct expander_device *ex = &dev->ex_dev;
	u8 *smp_resp;
	int res = -EINVAL;

	down_interruptible(&ex->smp_sema);
	if (!ex->smp_req || ex->smp_portal_pid != current->pid ||
	    offs != ex->smp_req_size)
		goto out;

	res = 0;
	if (size == 0)
		goto out;

	res = -ENOMEM;
	smp_resp = alloc_smp_resp(size);
	if (!smp_resp)
		goto out;
	res = smp_execute_task(dev, ex->smp_req, ex->smp_req_size,
			       smp_resp, size);
	if (!res) {
		memcpy(buf, smp_resp, size);
		res = size;
	}

	kfree(smp_resp);
out:
	kfree(ex->smp_req);
	ex->smp_req = NULL;
	ex->smp_req_size = 0;
	ex->smp_portal_pid = -1;
	up(&ex->smp_sema);
	return res;
}

> Note, I'm _not_ suggesting this is the best route for an SMP interface, 
> just stating sysfs generalities.  sysfs is flexible enough that it could 
> completely replace SG_IO (though that would not be a wise idea).

Indeed, sysfs _is_ very flexible, and with a single process open(2) it could
satisfy the transaction needed functionality.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/20/05 20:46, Jeff Garzik wrote:
> Consider what an ioctl is, overall:  a domain-specific "do this 
> operation" interface.  Which, further, is nothing but a wrapping of a 
> "send message" + "receive response" interface.  There are several ways 
> to do this in Linux:
> 
> * block driver.  a block driver is nothing but a message queue.  This is 

Not quite.  This maybe the way it operates, but it is called "block"
for a reason.

> why James has suggested implementing SMP as a block driver.  People get 
> stuck into thinking "block driver == block device", which is wrong.  The 
> Linux block layer is nothing but a message queueing interface.

Now, just because James suggested implementing the SMP service as a block
device you think this is the right thing to do?

How about this: Why not as a char device?

At least MS isn't suffering from the "no to specs" syndrome which
the Linux community seems to be suffering...

SMP as a Block device????

You need to see the history of SMP and its intended use.

> * netlink.  You connect to <an object> and send/receive messages.  Not 
> strictly limited to networking, as this is used in some areas of the 
> kernel now for generic event delivery.

This is very similar to the binary attr file "smp_portal" in
drivers/scsi/sas/sas_expander.c, bottom of file.

> * char driver.  Poor man's netlink.  Unless its done right, it suffers 
> from the same binary problems as ioctls.  I don't recommend this path.
> 
> * sysfs.  This has no inherent message/queue properties by itself, and 
> is less structured than blockdrvr or netlink, so dealing with more than 
> one outstanding operation at a time requires some coding.

Exactly! SMP. (binary attr file)

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Christoph Hellwig]

On Fri, Oct 21, 2005 at 01:48:45PM -0400, Luben Tuikov wrote:
> > why James has suggested implementing SMP as a block driver.  People get 
> > stuck into thinking "block driver == block device", which is wrong.  The 

                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> > Linux block layer is nothing but a message queueing interface.
> 
> Now, just because James suggested implementing the SMP service as a block
> device you think this is the right thing to do?
> 
> How about this: Why not as a char device?

you can implement a char device using the block layer.  See
drivers/scsi/{ch.c,osst.c,sg.c,st.c} for examples.

That beeing said I tried this approach.  It looks pretty cool when you
think about it, but the block layer is quite a bit too heavyweight for
queueing up a few SMP requests, and we need to carry too much useless
code around for it.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:04, Christoph Hellwig wrote:
>>How about this: Why not as a char device?
> 
> you can implement a char device using the block layer.  See
> drivers/scsi/{ch.c,osst.c,sg.c,st.c} for examples.

Christoph, you failed to see that my question was _rhetorical_.

> That beeing said I tried this approach.  It looks pretty cool when you
> think about it, but the block layer is quite a bit too heavyweight for
> queueing up a few SMP requests, and we need to carry too much useless
> code around for it.

That's the last reason not to implement SMP as a block device.
But this is good that you tried it and it "flopped".  This way
people will stop repeating "SMP... block device".

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Matthew Wilcox]

On Fri, Oct 21, 2005 at 02:12:49PM -0400, Luben Tuikov wrote:
> > That beeing said I tried this approach.  It looks pretty cool when you
> > think about it, but the block layer is quite a bit too heavyweight for
> > queueing up a few SMP requests, and we need to carry too much useless
> > code around for it.
> 
> That's the last reason not to implement SMP as a block device.
> But this is good that you tried it and it "flopped".  This way
> people will stop repeating "SMP... block device".

Block layer != Block device.

Nobody wants to implement SMP as a block device.

The question is whether the SMP interface should be implemented as part
of the block layer.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Douglas Gilbert]

Matthew Wilcox wrote:
> On Fri, Oct 21, 2005 at 02:12:49PM -0400, Luben Tuikov wrote:
> 
>>>That beeing said I tried this approach.  It looks pretty cool when you
>>>think about it, but the block layer is quite a bit too heavyweight for
>>>queueing up a few SMP requests, and we need to carry too much useless
>>>code around for it.
>>
>>That's the last reason not to implement SMP as a block device.
>>But this is good that you tried it and it "flopped".  This way
>>people will stop repeating "SMP... block device".
> 
> 
> Block layer != Block device.
> 
> Nobody wants to implement SMP as a block device.
> 
> The question is whether the SMP interface should be implemented as part
> of the block layer.

However, the block layer is used in the context of a
block device (and in some cases a char device).
If SAS domain discovery is done from the user space, and
the root file system is the far side of a SAS expander,
there are no suitable devices, just the SAS initiator
(HBA) which currently we cannot address via the block layer.


Doug Gilbert

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Douglas Gilbert wrote:
> However, the block layer is used in the context of a
> block device (and in some cases a char device).
> If SAS domain discovery is done from the user space, and
> the root file system is the far side of a SAS expander,
> there are no suitable devices, just the SAS initiator
> (HBA) which currently we cannot address via the block layer.

Invalid example.  All of the methods listed -- request_queue, netlink, 
chrdev, sysfs, ioctl -- will work just fine when the root filesystem is 
on the far side of a SAS expander.  These are just methods of 
communication, nothing more.

In your example -- userspace discovery required before root filesystem 
can be found -- a program running from initrd/initramfs would create an 
SMP device node, open it, and then proceed with the discovery and 
configuration process, which in turn creates the device nodes necessary 
to mount the root filesystem.

A request_queue is just a queue.  You are in complete control of who are 
the producer(s) of requests, and who are consumer(s).

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Jeff Garzik wrote:
> Douglas Gilbert wrote:
>> However, the block layer is used in the context of a
>> block device (and in some cases a char device).
>> If SAS domain discovery is done from the user space, and
>> the root file system is the far side of a SAS expander,
>> there are no suitable devices, just the SAS initiator
>> (HBA) which currently we cannot address via the block layer.

> Invalid example.  All of the methods listed -- request_queue, netlink, 
> chrdev, sysfs, ioctl -- will work just fine when the root filesystem is 
> on the far side of a SAS expander.  These are just methods of 
> communication, nothing more.
> 
> In your example -- userspace discovery required before root filesystem 
> can be found -- a program running from initrd/initramfs would create an 
> SMP device node, open it, and then proceed with the discovery and 
> configuration process, which in turn creates the device nodes necessary 
> to mount the root filesystem.
> 
> A request_queue is just a queue.  You are in complete control of who are 
> the producer(s) of requests, and who are consumer(s).


Since people are having such a tough time grasping the use of 
request_queue without an associated block device, here is a concrete 
example:  drivers/block/sx8.c.

sx8 creates a queue (grep for 'oob_q') specifically for handling 
discovery and configuration requests.  The only requests sent to this 
queue are I2O-ish management commands, never reads or writes.  Since 
they are management commands, these requests are NEVER associated with a 
block device.  Further, when sx8 discovery begins, sx8 block devices 
(and associated request_queues) simply don't exist.

Although sx8 management is entirely in-kernel, one could easily imagine 
how a userland interface (chrdev?) submits userspace commands into this 
queue.  Further, one can see how a host adapter could register one or 
more queues specifically for the transit of SMP commands.

NOTE:  THIS IS NOT AN ENDORSEMENT OF REQUEST QUEUES FOR SMP.  I merely 
wish to clear up misunderstandings about the block layer found in this 
thread.

It remains an open question whether the _complexity_ of this approach is 
more than is warranted for SMP.  But we've departed from that question, 
in this sub-thread :)

I merely illustrate that the block layer is being used _today_ for 
management commands.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Jeff Garzik <jgarzik@pobox.com> wrote:
> Since people are having such a tough time grasping the use of 

Please drop the generalizaion and FUD.

You are missing the point, and the point is that using the
block layer for SMP is so, so heavyweight, that no sane OS engineer
would do that.

Ready to see example:
 - see "smp_portal" in the complete and working SAS Transport Layer/Stack
from the 1st link of my signature.  See that the overhead of sending an SMP
task from user space to the domain device is _zero_, zero overhead.  As soon
as the user process does a read(2), the SMP task it wrote(2), is immediately
sent to the domain device, using the interconnect.  No bloat, no going around
in circles.

SDI does exactly the same things as it binds to the controller, the PCI device
implementing the domain interconnect functions (exported via SDI).

> It remains an open question whether the _complexity_ of this approach is 
> more than is warranted for SMP.  But we've departed from that question, 
> in this sub-thread :)

Let everyone see what is happening:
1. The industry submits a driver which clearly shows that SMP is not a block
device and there is no technical merit to do it this way.  The industry shows
a proper implementation of SMP access to the domain.
2. Then the Linux SCSI community decides, on their own, without ANY TECHNICAL
MERIT, or knowlege of the technology that they should try to implement SMP as
a block device.  Then they see that this isn't viable since it adds bloat and
SMP doesn't warrant it, and then we're back to 1.

The parent says: here is how you do it.  The child though, ignores the parent's
advice and does it its own way only to find out that the parent was right
all along.

My question is: at what point is this process going to stop?

   Luben


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Francois Romieu]

Luben Tuikov <ltuikov@yahoo.com> :
[...]
> Let everyone see what is happening:

Is there any chance that you can put this kind of content on a different
medium, say blob, rss feed, whatever instead of loading l-k ?

--
Ueimor

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Jeff Garzik <jgarzik@pobox.com> wrote:
> Invalid example.  All of the methods listed -- request_queue, netlink, 
> chrdev, sysfs, ioctl -- will work just fine when the root filesystem is 
> on the far side of a SAS expander.  These are just methods of 
> communication, nothing more.

Jeff, why don't you listen from time to time to people who work with
the technology on a daily basis who have experience with it, who
have _insight_ of the technology?  Such insight gives them great
intuition when it comes to design, among other things.

> In your example -- userspace discovery required before root filesystem 
> can be found -- a program running from initrd/initramfs would create an 
> SMP device node, open it, and then proceed with the discovery and 

SMP is part of the protocol, of what the device (PCI) implements.  It is always
there, just like phys.  You do not need to create it from user space. It
will be there for a user process to use, via say SDI.  SDI provides this as
part of the controller.  Read the SDI spec.

Insight of the whole architecture is irreplacable to create good design.

> configuration process, which in turn creates the device nodes necessary 
> to mount the root filesystem.

Also note that everyone does domain discovery in the kernel/FW and not
only for SAS but for other domains (even non-SCSI).  While domain
discovery is in the kernel/FW, _control_ of the domain is given to
user space, via say SDI -- everyone agrees on this.

   Luben

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/20/05 20:46, Jeff Garzik wrote:
> 
>>Consider what an ioctl is, overall:  a domain-specific "do this 
>>operation" interface.  Which, further, is nothing but a wrapping of a 
>>"send message" + "receive response" interface.  There are several ways 
>>to do this in Linux:
>>
>>* block driver.  a block driver is nothing but a message queue.  This is 
> 
> 
> Not quite.  This maybe the way it operates, but it is called "block"
> for a reason.

This illustrates you fundamentally don't understand a lot of Linux, and 
SCSI too.

Several non-blkdev device classes (Christoph listed them) use block 
layer request_queue for command transit, as does SG_IO and /dev/sg.


>>why James has suggested implementing SMP as a block driver.  People get 
>>stuck into thinking "block driver == block device", which is wrong.  The 
>>Linux block layer is nothing but a message queueing interface.
> 
> 
> Now, just because James suggested implementing the SMP service as a block
> device you think this is the right thing to do?

I very clearly said I don't know the best answer.  Perhaps you need to 
re-read the quoted email?


> How about this: Why not as a char device?

I covered that in the quoted email.


> At least MS isn't suffering from the "no to specs" syndrome which
> the Linux community seems to be suffering...

We have plenty of specs.  It's called source code.

You don't understand the Linux development process (think its more 
political than technical) and you don't understand even what a block 
driver is, and you wonder why you have difficulty getting code into the 
kernel?

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:18, Jeff Garzik wrote:
> This illustrates you fundamentally don't understand a lot of Linux, and 
> SCSI too.
> 
> Several non-blkdev device classes (Christoph listed them) use block 
> layer request_queue for command transit, as does SG_IO and /dev/sg.

When people start getting personal you know that they're losing it.

> We have plenty of specs.  It's called source code.
> 
> You don't understand the Linux development process (think its more 
> political than technical) and you don't understand even what a block 
> driver is, and you wonder why you have difficulty getting code into the 
> kernel?

Again, when people start getting personal, you know that they are losing it.

Thank you for spreading FUD -- I'm sure you've impressed your managament,
how great of a Linux programmer you are and how I don't know anything.
I'd suggest you keep pushing the politics _behind_ the scences.

Have a good day,
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 14:18, Jeff Garzik wrote:
>>We have plenty of specs.  It's called source code.
>>
>>You don't understand the Linux development process (think its more 
>>political than technical) and you don't understand even what a block 
>>driver is, and you wonder why you have difficulty getting code into the 
>>kernel?
> 
> 
> Again, when people start getting personal, you know that they are losing it.
> 
> Thank you for spreading FUD -- I'm sure you've impressed your managament,
> how great of a Linux programmer you are and how I don't know anything.
> I'd suggest you keep pushing the politics _behind_ the scences.


I'm trying to tell it like it is, in the hopes that you will eventually 
learn the process, and be a good upstream maintainer we can all work with.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:54, Jeff Garzik wrote:
>>Thank you for spreading FUD -- I'm sure you've impressed your managament,
>>how great of a Linux programmer you are and how I don't know anything.
>>I'd suggest you keep pushing the politics _behind_ the scences.
> 
> I'm trying to tell it like it is, in the hopes that you will eventually 
> learn the process, and be a good upstream maintainer we can all work with.

You're again spreading FUD and playing this political game hoping
your management is reading this and merely because of this they'll award you.
I cannot blame you -- how many of them read C code or specs...  I mean, even
"the community" has said "no" to specs.  All you're doing is just playing
your political game, spreading FUD.

And what "process" are you talking about?  Do you have any idea the
_amazement_ people get when learing that Linux is going to implement
showing all phys on the domain in sysfs...?  Do you have any idea
the effort in keeping customers happy when the whims of "the community"
change on an a hourly basis (as the community is learning)?  Shall I give
examples?

As to your reply above:
Jeff, not so according to people I work with on a daily basis from all
over the world.

When you said in your email:

On Oct 21, 2005, Jeff Garzik <jgarzik@pobox> wrote:
>
> We have plenty of specs.  It's called source code.

That pretty much seals it.

Have a nice day,
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 14:54, Jeff Garzik wrote:
> 
>>>Thank you for spreading FUD -- I'm sure you've impressed your managament,
>>>how great of a Linux programmer you are and how I don't know anything.
>>>I'd suggest you keep pushing the politics _behind_ the scences.
>>
>>I'm trying to tell it like it is, in the hopes that you will eventually 
>>learn the process, and be a good upstream maintainer we can all work with.
> 
> 
> You're again spreading FUD and playing this political game hoping
> your management is reading this and merely because of this they'll award you.
> I cannot blame you -- how many of them read C code or specs...  I mean, even
> "the community" has said "no" to specs.  All you're doing is just playing
> your political game, spreading FUD.

My only response is from SNL:  hear me now and believe me later.

I can only repeat the truth, and eventually (hope) you'll get it.


> And what "process" are you talking about?  Do you have any idea the
> _amazement_ people get when learing that Linux is going to implement
> showing all phys on the domain in sysfs...?  Do you have any idea
> the effort in keeping customers happy when the whims of "the community"
> change on an a hourly basis (as the community is learning)?  Shall I give
> examples?
> 
> As to your reply above:
> Jeff, not so according to people I work with on a daily basis from all
> over the world.
> 
> When you said in your email:
> 
> On Oct 21, 2005, Jeff Garzik <jgarzik@pobox> wrote:
> 
>>We have plenty of specs.  It's called source code.
> 
> 
> That pretty much seals it.

Shockingly, through all this stuff that annoys you, Linux is a wild and 
unqualified success.  ;-)

Linux is organic.  Its not cleanly laid out.  Its not specified in some 
document.  Linux is result of evolution as "nature" as dictated.  This 
drives some people nuts, but at the end of the day, it _works_.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Jeff Garzik wrote:
> Linux is result of evolution as "nature" as dictated.  This 
> drives some people nuts, but at the end of the day, it _works_.

The last two words sound a bit general.
-- 
Stefan Richter
-=====-=-=-= =-=- =-=-=
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:54, Jeff Garzik wrote:
>
> I'm trying to tell it like it is, in the hopes that you will eventually 
> learn the process, and be a good upstream maintainer we can all work with.

Look how you're using "all", generalizing left and right.  You just want to
create this FUD and spread this FUD that I don't work well with anyone.  Yep,
this is a pretty low blow.  While in fact did you talk to everyone I work with?

I say, when people are losing it on the technical front, they try to attack
personally and on political basis.

Have a nice day,
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 14:54, Jeff Garzik wrote:
> 
>>I'm trying to tell it like it is, in the hopes that you will eventually 
>>learn the process, and be a good upstream maintainer we can all work with.
> 
> 
> Look how you're using "all", generalizing left and right.  You just want to
> create this FUD and spread this FUD that I don't work well with anyone.  Yep,
> this is a pretty low blow.  While in fact did you talk to everyone I work with?
> 
> I say, when people are losing it on the technical front, they try to attack
> personally and on political basis.


The technical stuff got covered long ago.  Here are the basic basics:

* aic94xx needs to have the scsi-host-template in the LLDD, to fix 
improper layering.
* SAS generic code needs to use SAS transport class, which calls 
scsi_scan_target(), to avoid code duplication.
* other stuff I listed in my "analysis" email, including updating libata 
to support SAS+SATA hardware.

This is the stuff that I have been working on (nothing pushed to sas-2.6 
yet, as it doesn't yet boot locally).

If you were willing to do this stuff, _working with others_, then I 
would be off in happy happy SATA land right now, and you would have been 
nominated to be the Linux SAS maintainer.

Call it FUD, politics, personal attacks, wanking off to please 
manglement, whatever.  My goal has always been to (a) help Linux users 
by getting aic94xx+SAS upstream, and (b) try to help you understand why 
your code didn't go upstream verbatim, long after others have given up 
trying to do that.

	Jeff, he of infinite patience

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:39, Jeff Garzik wrote:
> The technical stuff got covered long ago.  Here are the basic basics:
> 
> * aic94xx needs to have the scsi-host-template in the LLDD, to fix 
> improper layering.
> * SAS generic code needs to use SAS transport class, which calls 
> scsi_scan_target(), to avoid code duplication.
> * other stuff I listed in my "analysis" email, including updating libata 
> to support SAS+SATA hardware.

All this bastardizes the code and the layering infrastructure.

If you're saying that there is "improper layering" you must be
smoking something really strong.

I'd suggest you talk to storage engineers with years of storage
experience about the design of the Linux SAS Stack.
I'd suggest you take a look at the USB and SBP code.
I'd suggest you take an overview of all of SAM and see how the protocols
fit together.
I'd suggest you take a look at the hardware design and see how and _where_
it fits and where it does not and _why_.
I'd suggest you take a look at the event management interface
(drivers/scsi/sas/sas_event.c) what it is, how it provides _all_
communication from LLDD to SAS Layer, filled in by the SAS Stack.
I'd suggest you take a look at the Execute Command SCSI RPC and the TMFs,
declared in the struct sas_ha_struct, filled in by the LLDD,
how they provide all communication from the SAS Stack to the LLDD (inteconnect).

I'm even ready to do a presentation in front of people explaining
the design of the Linux SAS Stack, why and how it works.  Specify
time and venue, please.

(I will put something up on the web site -- html and figures
to show the layering, function stubs, event management, etc.)

> This is the stuff that I have been working on (nothing pushed to sas-2.6 
> yet, as it doesn't yet boot locally).

I'm sorry to hear that.  Maybe you shouldn't have to bastardize the code.
Implement SDI and then everyone will be happy.  Look, I've alredy twice
posted code and templates, just grab it and put your name there.

Think about it: there is no reason to mess with the code.  It is
present at the link below, it works and people are using it.  It is also
sound and spec complient.  There are other subsystems which use the exact
same layering (namely USB Storage and SBP) which is consistent with the
SCSI Architecture Model all new protocols abide by.

> If you were willing to do this stuff, _working with others_, then I 
> would be off in happy happy SATA land right now, and you would have been 
> nominated to be the Linux SAS maintainer.

You seem to be traveling a path which I've already traveled.

Jeff, if you want an Adaptec SAS driver which has the host template
_in_ the driver, you can use "adp94xx" submitted earlier this year
by Adaptec to "the community".

"The community" rejected it. Why?  Because "common SAS tasks should
be in a common layer".  Well there you have the Linux SAS Stack and
aic94xx at the link below (my sig), which does separate common SAS
tasks and the interconnect _as per architecture and spec_.  Apparently
this still isn't good enough for "the community".

You see how this going around in circles just never seems to end.

One day one thing, another day the opposite, etc.

> Call it FUD, politics, personal attacks, wanking off to please 
> manglement, whatever.  My goal has always been to (a) help Linux users 
> by getting aic94xx+SAS upstream, and (b) try to help you understand why 
> your code didn't go upstream verbatim, long after others have given up 
> trying to do that.

(a) There are Linux users using "aic94xx+SAS" right now.  They've
downloaded a working Linux code from the link at my signature.

(b) I think it was because Linux says "no to specs" and you say that
"specs are the source code".

> 	Jeff, he of infinite patience

Everyone is very pleased with you.  You will go places.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 15:39, Jeff Garzik wrote:
> 
>>The technical stuff got covered long ago.  Here are the basic basics:
>>
>>* aic94xx needs to have the scsi-host-template in the LLDD, to fix 
>>improper layering.
>>* SAS generic code needs to use SAS transport class, which calls 
>>scsi_scan_target(), to avoid code duplication.
>>* other stuff I listed in my "analysis" email, including updating libata 
>>to support SAS+SATA hardware.
> 
> 
> All this bastardizes the code and the layering infrastructure.
> 
> If you're saying that there is "improper layering" you must be
> smoking something really strong.

I already described why.  Examples are DMA boundary and s/g limit, among 
others.  When confronted with this, you proposed an additional hardware 
information struct which duplicates Scsi_Host_Template.

Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
driver is implemented.  The host template is in the low level driver, 
while most of the code is common code, implemented elsewhere.



>>If you were willing to do this stuff, _working with others_, then I 
>>would be off in happy happy SATA land right now, and you would have been 
>>nominated to be the Linux SAS maintainer.
> 
> 
> You seem to be traveling a path which I've already traveled.
> 
> Jeff, if you want an Adaptec SAS driver which has the host template
> _in_ the driver, you can use "adp94xx" submitted earlier this year
> by Adaptec to "the community".
> 
> "The community" rejected it. Why?  Because "common SAS tasks should
> be in a common layer".  Well there you have the Linux SAS Stack and
> aic94xx at the link below (my sig), which does separate common SAS
> tasks and the interconnect _as per architecture and spec_.  Apparently
> this still isn't good enough for "the community".
> 
> You see how this going around in circles just never seems to end.
> 
> One day one thing, another day the opposite, etc.

Your current code is much closer to the desired end result, as it 
separates out SAS common code.  That's why its being used as a base.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 17:12, Jeff Garzik wrote:
> 
> I already described why.  Examples are DMA boundary and s/g limit, among 
> others.  When confronted with this, you proposed an additional hardware 
> information struct which duplicates Scsi_Host_Template.

I told you -- I have this in the struct asd_ha_struct and it was merely
a downplay that I didn't include the same thing in struct sas_ha_struct.

> Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 

No, this is the solution which would turn everything upside down.
The easiest and smallest solution is to just include this tiny struct
and end this.  It would have 0 impact on code.  In fact I'll
implement it now and push it to the git tree. ;-)

The host template _mixes_ hw, scsi core, and protocol knowlege into
one ugly blob.  I've given this argument before, several times.

> driver is implemented.  The host template is in the low level driver, 
> while most of the code is common code, implemented elsewhere.

libata isn't without architectural problems.  What strikes me is
that you think that libata-scsi is SATL.

You are so much better off renaming it to satl.c and given
the knowlege you've gained over the years and the backing you have
from engineers from companies, start with it at device level.  I, as
I'm sure other (not to name names) will be more than happy to contribute
if you started this.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 17:12, Jeff Garzik wrote:
> 
>>I already described why.  Examples are DMA boundary and s/g limit, among 
>>others.  When confronted with this, you proposed an additional hardware 
>>information struct which duplicates Scsi_Host_Template.
> 
> 
> I told you -- I have this in the struct asd_ha_struct and it was merely
> a downplay that I didn't include the same thing in struct sas_ha_struct.
> 
> 
>>Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
> 
> 
> No, this is the solution which would turn everything upside down.
> The easiest and smallest solution is to just include this tiny struct
> and end this.  It would have 0 impact on code.  In fact I'll
> implement it now and push it to the git tree. ;-)
> 
> The host template _mixes_ hw, scsi core, and protocol knowlege into
> one ugly blob.

True.

If you do not like the current situation, evolve the SCSI core (and all 
drivers) to where you think they should be.

The correct answer is NOT to duplicate information between 
Scsi_Host_Template and Lubens_Hardware_Struct.


>>driver is implemented.  The host template is in the low level driver, 
>>while most of the code is common code, implemented elsewhere.
> 
> 
> libata isn't without architectural problems.  What strikes me is
> that you think that libata-scsi is SATL.

The only things that matter are (a) what the code is now, and (b) what 
changes are needed to get where we need to be.

Thus, regardless of whether or not libata-scsi meets the needs of 
SAS+SATA hardware, libata-scsi is where all SCSI<->ATA translation 
should occur.  If you are dissatisfied, evolve the code to where it 
needs to be.


> You are so much better off renaming it to satl.c and given

Naming is completely irrelevant.  Just modify the code.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 17:41, Jeff Garzik wrote:
>>>I already described why.  Examples are DMA boundary and s/g limit, among 
>>>others.  When confronted with this, you proposed an additional hardware 
>>>information struct which duplicates Scsi_Host_Template.
>>
>>
>>I told you -- I have this in the struct asd_ha_struct and it was merely
>>a downplay that I didn't include the same thing in struct sas_ha_struct.

Here is the commit in question:
http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a

>>>Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
>>
>>
>>No, this is the solution which would turn everything upside down.
>>The easiest and smallest solution is to just include this tiny struct
>>and end this.  It would have 0 impact on code.  In fact I'll
>>implement it now and push it to the git tree. ;-)
>>
>>The host template _mixes_ hw, scsi core, and protocol knowlege into
>>one ugly blob.
> 
> 
> True.
> 
> If you do not like the current situation, evolve the SCSI core (and all 
> drivers) to where you think they should be.

While the architecture in my mind is clear, I cannot do this myself
(and for all drivers).  Such a change would be gradual, involving more
than one developer, for more than one (new) driver, etc.

First we need SDI to make everyone happy.  This way HP/LSI/Adaptec
can move quickest to the customers with minimal pain to the customers
and to the companies which would have to change software (minimal change).

> Thus, regardless of whether or not libata-scsi meets the needs of 
> SAS+SATA hardware, libata-scsi is where all SCSI<->ATA translation 
> should occur.  If you are dissatisfied, evolve the code to where it 
> needs to be.

Ok.

> Naming is completely irrelevant.  Just modify the code.

Ok.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 17:41, Jeff Garzik wrote:
> 
>>>>I already described why.  Examples are DMA boundary and s/g limit, among 
>>>>others.  When confronted with this, you proposed an additional hardware 
>>>>information struct which duplicates Scsi_Host_Template.
>>>
>>>
>>>I told you -- I have this in the struct asd_ha_struct and it was merely
>>>a downplay that I didn't include the same thing in struct sas_ha_struct.
> 
> 
> Here is the commit in question:
> http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a

This effectively illustrates the wrong thing to do:  duplicating 
information that's already in Scsi_Host_Template.

Just use Scsi_Host_Template in the LLDD and see where that goes.


>>>>Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
>>>
>>>
>>>No, this is the solution which would turn everything upside down.
>>>The easiest and smallest solution is to just include this tiny struct
>>>and end this.  It would have 0 impact on code.  In fact I'll
>>>implement it now and push it to the git tree. ;-)
>>>
>>>The host template _mixes_ hw, scsi core, and protocol knowlege into
>>>one ugly blob.
>>
>>
>>True.
>>
>>If you do not like the current situation, evolve the SCSI core (and all 
>>drivers) to where you think they should be.
> 
> 
> While the architecture in my mind is clear, I cannot do this myself
> (and for all drivers).  Such a change would be gradual, involving more
> than one developer, for more than one (new) driver, etc.

Correct.  That's why there is resistance to aic94xx's approach of 
creating a totally new "strict SAM" path, existing in parallel with the 
traditional SCSI core.  You need to evolve the existing code to get 
there.  Such changes are gradual, involving more than one developer, etc.

We don't need one small set of SCSI drivers behaving differently from 
the vast majority of existing SCSI drivers.

Hear me now, and believe me later:  we all largely agree on the points 
you've raised about legacy crapola in the SCSI core.  James, Christoph, 
myself, and several others disagree with your assertion that the old 
SCSI core should exist in parallel with your new SCSI core.

We differ on the path, not the goal.  As a thought experiment, you could 
try simply implementing the changes requested, and see where that goes.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Jeff Garzik wrote:
> Luben Tuikov wrote:
>>>>> Examples are DMA boundary and s/g limit, 
>>>>> among others.  When confronted with this, you proposed an 
>>>>> additional hardware information struct which duplicates 
>>>>> Scsi_Host_Template.
>>>>
>>>> I told you -- I have this in the struct asd_ha_struct and it was merely
>>>> a downplay that I didn't include the same thing in struct 
>>>> sas_ha_struct.
>>
>> Here is the commit in question:
>> http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a 
> 
> This effectively illustrates the wrong thing to do:  duplicating 
> information that's already in Scsi_Host_Template.
> 
> Just use Scsi_Host_Template in the LLDD and see where that goes.

Will cmd_per_lun, sg_tablesize, max_sectors, dma_boundary, 
use_clustering ever have to be adjusted specifically for a SAS hardware?

Obviuosly none of this is required _at the moment_. IOW neither the 
introduction of a sas_ha_hw_profile nor a pass-through of 
scsi_host_template down to SAS interconnect drivers is required right 
now. So why do one or the other now? Isn't it a sensible rule to not 
solve problems now which do not exist yet?

(I guess Luben only introduced sas_ha_hw_profile to demonstrate that 
there will never be an absolute requirement for scsi_host_template --- 
in its present form --- to be visible in a SAS transport layer <-> SAS 
interconnect driver interface. And there are certainly more alternatives 
to these two approaches.)
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:
> 
> Will cmd_per_lun, sg_tablesize, max_sectors, dma_boundary, 
> use_clustering ever have to be adjusted specifically for a SAS hardware?

No hardware SAS chip I know of needs any of those legacy limitations.
Neither BCM8603 nor Fusion MPT.

Those limitations are purely Parallel SCSI.

I just included it, to show proof of concept -- when the architecure and
layering is correct, how easy it is to do it.  But you're right, it is
not needed.

> Obviuosly none of this is required _at the moment_. IOW neither the 
> introduction of a sas_ha_hw_profile nor a pass-through of 
> scsi_host_template down to SAS interconnect drivers is required right 
> now. So why do one or the other now? Isn't it a sensible rule to not 
> solve problems now which do not exist yet?

This is exactly the rule I followed when developing the SAS Transport
Layer for Linux.  Furthermore, _that_ rule, to never overengineer, I learned
from Linux.  Sadly the politics are too deep and that rule applies only
to what is convenient, at least in Linux SCSI.

   Luben


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Jeff Garzik wrote:
> Luben Tuikov wrote:
>>>> The host template _mixes_ hw, scsi core, and protocol knowlege into
>>>> one ugly blob.
>>>
>>> True.
>>>
>>> If you do not like the current situation, evolve the SCSI core (and 
>>> all drivers) to where you think they should be.
>>
>> While the architecture in my mind is clear, I cannot do this myself
>> (and for all drivers).  Such a change would be gradual, involving more
>> than one developer, for more than one (new) driver, etc.
> 
> Correct.  That's why there is resistance to aic94xx's approach of 
> creating a totally new "strict SAM" path, existing in parallel with the 
> traditional SCSI core.  You need to evolve the existing code to get 
> there.  Such changes are gradual, involving more than one developer, etc.
> 
> We don't need one small set of SCSI drivers behaving differently from 
> the vast majority of existing SCSI drivers.
> 
> Hear me now, and believe me later:  we all largely agree on the points 
> you've raised about legacy crapola in the SCSI core.  James, Christoph, 
> myself, and several others disagree with your assertion that the old 
> SCSI core should exist in parallel with your new SCSI core.
> 
> We differ on the path, not the goal.  As a thought experiment, you could 
> try simply implementing the changes requested, and see where that goes.

I am not familiar with most parts of the SCSI subsystem. However from 
what I understood, some existing concepts in the core need to be removed 
from SCSI core entirely, others need to be cleaned up WRT what they mean 
and how they represent it. It seems to me that a practical path to go 
would be:

A. Post mock-ups and pseudo code about how to change the core, discuss.
B. Set up a scsi-cleanup tree. In this tree,
      1. renovate the core (thereby break all command set drivers and
         all transport subsystems),
      2. update ~2 command set drivers and ~2 transport subsystems
      3. validate the renovated core,
      4. fix the conceptual errors of the renovated core (as well as
         first few discovered bugs in the implementation),
      5. update all other command set drivers,
      6. update all transport subsystems where resources to do so are
         available,
      7. test all command set drivers as far as hardware is accessible,
      8. test the updated transport subsystems as far as hardware is
         accessible,
      9. fix prominent bugs.
C. In mainline,
      i. mark all drivers which cannot be updated in the mid term as
         scheduled for temporary feature removal (i.e. they will be
         broken for an undetermined period),
     ii. mark all drivers which have been updated in scsi-cleanup tree,
         but were not thoroughly tested, as scheduled for temporary
         feature regression (i.e. they will be experimental for an
         undetermined period).
D. Push scsi-cleanup tree to -mm and shake out bugs.
E. Push to mainline.
F. Fix remaining drivers as time goes by, remove drivers which remain
    broken for too long.

Most steps may overlap, some steps may repeat. Step A is fortunately 
already going on for quite some time.

Step 1.-4. could involve much dispute, thus taking much more time than 
technically necessary -or- (and that would be less fortunate) being 
dragged out into later stages when a conceptually stabilized core is 
desirable. Much of that may be prevented in step A.

I doubt that the desired cleanup of the SCSI core could be done 
on-the-go, i.e. without temporary breakage of larger parts of the 
subsystem (out of mainline). But then again, I don't know much of the 
subsystem, so what am I talking about here?

Also, long-term breakage of smaller parts of the SCSI subsystem in 
mainline is to be expected; breakage which is to be announced and scheduled.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Christoph Hellwig]

On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
> A. Post mock-ups and pseudo code about how to change the core, discuss.
> B. Set up a scsi-cleanup tree. In this tree,
>      1. renovate the core (thereby break all command set drivers and
>         all transport subsystems),

No way.  Doing things from scatch is a really bad idea.  See how far we came
with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break the
world.  Please submit changes to fix _one_ thing at a time and fix all users.
Repeat until done or you don't care anymore.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Sergey Panov]

On Sat, 2005-10-22 at 11:58 +0100, Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
> > A. Post mock-ups and pseudo code about how to change the core, discuss.
> > B. Set up a scsi-cleanup tree. In this tree,
> >      1. renovate the core (thereby break all command set drivers and
> >         all transport subsystems),
> 
> No way.  Doing things from scatch is a really bad idea.  See how far we came
> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break the
> world.  Please submit changes to fix _one_ thing at a time and fix all users.
> Repeat until done or you don't care anymore.

 It is a mistake to think that you can not do a big rework and keep SCSI
sub-system stable. You just have to make sure the OLD way is supported
for as log as it is needed.

 E.g. before moving in the RIGHT direction and weeding out parallel SCSI
atavism from the common SCSI layer (or should I say SAM, to please
Luben) you can insert "generic" transport layer at the bottom (just like
Luben inserted his SAS) and use it to keep all currently supported
drivers stable. SCSI mid-layer can be reworked in any number of steps,
even in one step, because is not a big thing, it is just 11k lines plus
4k of recently added transport modules.

> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Christoph Hellwig]

On Sat, Oct 22, 2005 at 11:28:30AM -0400, Sergey Panov wrote:
>  It is a mistake to think that you can not do a big rework and keep SCSI
> sub-system stable. You just have to make sure the OLD way is supported
> for as log as it is needed.

No.  Rewriting something from scratch is horrible engineering practice.
It's impossible to very huge changes, small incremental changes OTOH
allow easier planning, easier calculation of the risks and cost and most
import better test coverage.  There's nothing specific to scsi or linux
kernel code about it.  It'd suggest you read:

 http://www.joelonsoftware.com/articles/fog0000000069.html

or various similar articles.  Full scale rewrites almost never work
out.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Sergey Panov]

On Sat, 2005-10-22 at 18:19 +0100, Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 11:28:30AM -0400, Sergey Panov wrote:
> >  It is a mistake to think that you can not do a big rework and keep SCSI
> > sub-system stable. You just have to make sure the OLD way is supported
> > for as log as it is needed.
> 
> No.  Rewriting something from scratch is horrible engineering practice.

Most of the time. Besides "rework" is not necessarily "rewrite from
scratch", most of the time it means "modification" of the existing
system.

> It's impossible to very huge changes, small incremental changes OTOH
> allow easier planning, easier calculation of the risks and cost and most
> import better test coverage.  There's nothing specific to scsi or linux
> kernel code about it.  It'd suggest you read:
> 
>  http://www.joelonsoftware.com/articles/fog0000000069.html

Bad example -- just count number of lines in drivers/scsi/scsi*.c
and in Netscape 4.0 and you'll see why.
That does not mean I advocate throughing out current SCSI mid layer and
writing a new one. As I can tell, no one on that list is proposing the
"rewrite from scratch" approach.
  I just was trying to point out that Luben's transport "layers" in
place of transport "modules-appendages" simplifies that
migration/evolution.   

> or various similar articles.  Full scale rewrites almost never work
> out.


Sergey Panov

===========================================

I expressed my personal opinion and I am not speaking for anyone else.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/22/05 13:38, Sergey Panov wrote:
>   I just was trying to point out that Luben's transport "layers" in
> place of transport "modules-appendages" simplifies that
> migration/evolution.

True. "modules-appendages" just makes everything messy.  There is
so much SPI in the host template... Take for example recovery,
or as Linux SCSI calls it Error Handling (EH).

EH in Linux SCSI is SPI.  This is a fact, and anyone can look at the
code to convince themselves.

In order to properly get rid ourselves of HCIL, recovery should
be offloaded to the transport _layer_ which communicates with
the transport/interconnect to get things going, as shown in the SAS
Transport Stack at the link in my signature, USB* or SBP subsystems.

* I've been asking Alan to properly implement USB recovery
in the USB Storage subsystem...  Maybe one day...

That is, the layer above knows nothing of the layer below, but
each layer provides well defined functionality, and they all
work in tandem.  This is exactly what allows for quick growth
and rich future (of a design).

In such a design, SCSI Core would be very small and the task
paths would be very straightforward as shown in the SAS Transport
Stack in the link in my signature, USB or SBP subsystems.

BTW, "modules-appendages" very truly describes the current state
of Linux SCSI and this can be seen from, for example the scsi host
structure, where those "modules-appendages" are "hooked".

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Alan Cox]

On Sad, 2005-10-22 at 18:19 +0100, Christoph Hellwig wrote:
> No.  Rewriting something from scratch is horrible engineering practice.

There I must disagree strongly. There are times when it is the right
thing to do because the original is unsalvagable (see drivers/ide). 

OTOH I do not believe the scsi layer is in this state. 


Luben, it isn't about layering. Linux does layering and does it strongly
but it does not do unneccessary layering. If the layer above is wrong
then adapt it step by step without breaking existing stuff (or making
sure the existing changes to drivers are clean and testable).

That process works. There is very little left of the original "Linux"
and most of what is left is the stuff that most needs maintenance (eg
floppy.c, tty layer). In time they too will change.

Don't go around the scsi layer for something generic, go through it.
Going around things or adding ugly hacks is fine (and justified) for a
specific single piece of hardware that is somehow brain damaged, its not
the right approach for something likely to be mainstream and generic.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/22/05 13:19, Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 11:28:30AM -0400, Sergey Panov wrote:
> 
>> It is a mistake to think that you can not do a big rework and keep SCSI
>>sub-system stable. You just have to make sure the OLD way is supported
>>for as log as it is needed.
> 
> 
> No.  Rewriting something from scratch is horrible engineering practice.

Off Topic:
	Discourse on Christoph's "Rewriting something from scratch
		is horrible engineering practice"

Christoph,

Imagine a for ( ) { } loop spanning 5000 lines.  Imagine never using
functions to separate things.  But imagine that a _factory_ is using
this code on its production line and that code, although badly written,
does work and keeps the production line going and thousands of
people working.

They hire an engineer to make it _managable_ and supportable -- this
warrants understanding the production line, what the code does, what it
controls and how.  Understanding how the factory workers use it and what
they expect.  Understanding the code (which may not be as easy).  Then it
is rewritten so that it can be easily supported and maintained.

This is real life example.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Alan Cox]

On Llu, 2005-10-24 at 09:51 -0400, Luben Tuikov wrote:
> controls and how.  Understanding how the factory workers use it and what
> they expect.  Understanding the code (which may not be as easy).  Then it
> is rewritten so that it can be easily supported and maintained.

Very very rarely, because it means down time and supporting two systems
at once. Take a look at the australian customs fiasco or the british
passport office disaster to see why (actually almost any large
government IT project where politics dictated 'write new stuff so I can
announce it in parliament').

The smart factory update would occur piece by piece. Starting with the
most pressing problems (ie fastest ROI) and working to a plan that ends
up with the system modular and clean.

You don't turn a steel plant off for a software upgrade.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/24/05 11:41, Alan Cox wrote:
> On Llu, 2005-10-24 at 09:51 -0400, Luben Tuikov wrote:
> 
>>controls and how.  Understanding how the factory workers use it and what
>>they expect.  Understanding the code (which may not be as easy).  Then it
>>is rewritten so that it can be easily supported and maintained.
> 
> 
> Very very rarely, because it means down time and supporting two systems
> at once. Take a look at the australian customs fiasco or the british
> passport office disaster to see why (actually almost any large
> government IT project where politics dictated 'write new stuff so I can
> announce it in parliament').
> 
> The smart factory update would occur piece by piece. Starting with the
> most pressing problems (ie fastest ROI) and working to a plan that ends
> up with the system modular and clean.
> 
> You don't turn a steel plant off for a software upgrade.

There was 0 (zero) effective downtime to the factory.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Regala]

Luben Tuikov <luben_tuikov@adaptec.com> disait dernièrement que :

> On 10/24/05 11:41, Alan Cox wrote:
>> On Llu, 2005-10-24 at 09:51 -0400, Luben Tuikov wrote:
>> 
>>>controls and how.  Understanding how the factory workers use it and what
>>>they expect.  Understanding the code (which may not be as easy).  Then it
>>>is rewritten so that it can be easily supported and maintained.
>> 
>> 
>> Very very rarely, because it means down time and supporting two systems
>> at once. Take a look at the australian customs fiasco or the british
>> passport office disaster to see why (actually almost any large
>> government IT project where politics dictated 'write new stuff so I can
>> announce it in parliament').
>> 
>> The smart factory update would occur piece by piece. Starting with the
>> most pressing problems (ie fastest ROI) and working to a plan that ends
>> up with the system modular and clean.
>> 
>> You don't turn a steel plant off for a software upgrade.
>
> There was 0 (zero) effective downtime to the factory.

but refactoring can be done in incremental pieces, can't be ?
rewriting it from scratch is, in this very case, really for the sake
of self-pride and brain-masturbation.
Bravo

This is not a really convincing example...

-- 
Mathieu Segaud

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/24/05 11:59, Regala wrote:
> but refactoring can be done in incremental pieces, can't be ?
> rewriting it from scratch is, in this very case, really for the sake
> of self-pride and brain-masturbation.
> Bravo
> 
> This is not a really convincing example...
> 
> -- 
> Mathieu Segaud

They, the factory, wanted it rewritten from scratch,
so that they can maintain and support it themselves.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Christoph Hellwig <hch@infradead.org> wrote:
> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
> > A. Post mock-ups and pseudo code about how to change the core, discuss.
> > B. Set up a scsi-cleanup tree. In this tree,
> >      1. renovate the core (thereby break all command set drivers and
> >         all transport subsystems),
> 
> No way.  Doing things from scatch is a really bad idea.  See how far we came
> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break
> the
> world.  Please submit changes to fix _one_ thing at a time and fix all users.
> Repeat until done or you don't care anymore.

No offence Christoph, but who are you again?

There is a clear reason why you among others do not want new architecture.
And that reason is (people) obsoletion.

Such political stance cannot go on forever -- just look at History.
Sooner or later things change and they change radically.  The question
is How prepared are you/we to cope with this (inevitable) change?

Either way, obsoletion or adoption -- think about it, it doesn't only apply
to computer and OS design, it applies to everything.

    Luben




-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> No offence Christoph, but who are you again?
> 
> There is a clear reason why you among others do not want new architecture.
> And that reason is (people) obsoletion.
> 
> Such political stance cannot go on forever -- just look at History.
> Sooner or later things change and they change radically.  The question
> is How prepared are you/we to cope with this (inevitable) change?


THIS HAS FUCK-ALL TO DO WITH POLITICS.

Its about working with the SCSI layer, not around it.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
>>A. Post mock-ups and pseudo code about how to change the core, discuss.
>>B. Set up a scsi-cleanup tree. In this tree,
>>     1. renovate the core (thereby break all command set drivers and
>>        all transport subsystems),
> 
> No way.  Doing things from scatch is a really bad idea.  See how far we came
> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break the
> world.  Please submit changes to fix _one_ thing at a time and fix all users.
> Repeat until done or you don't care anymore.

I agree with you. Alas my wording was misunderstandable und obviously 
carried a wrong tone.

I did not say "replace the core" in step 1. Also, the breakage which I 
refer to in step 1 would have to be immediately corrected in step 2 
(although not for the whole subsystem at once, to allow for a fast cycle 
of validation of what happened in step 1). Furthermore I specifically 
said that most steps may (let me add: and should) overlap.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[David Lang]

On Sat, 22 Oct 2005, Stefan Richter wrote:

> Christoph Hellwig wrote:
>> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
>>> A. Post mock-ups and pseudo code about how to change the core, discuss.
>>> B. Set up a scsi-cleanup tree. In this tree,
>>>     1. renovate the core (thereby break all command set drivers and
>>>        all transport subsystems),
>> 
>> No way.  Doing things from scatch is a really bad idea.  See how far we 
>> came
>> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break 
>> the
>> world.  Please submit changes to fix _one_ thing at a time and fix all 
>> users.
>> Repeat until done or you don't care anymore.
>
> I agree with you. Alas my wording was misunderstandable und obviously carried 
> a wrong tone.
>
> I did not say "replace the core" in step 1. Also, the breakage which I refer 
> to in step 1 would have to be immediately corrected in step 2 (although not 
> for the whole subsystem at once, to allow for a fast cycle of validation of 
> what happened in step 1). Furthermore I specifically said that most steps may 
> (let me add: and should) overlap.

Stefan,
   we are supposed to be on a 2-month release cycle, with all major changes 
going in in the first two weeks of that cycle. This timeframe doesn't 
leave you any noticable time to implement your steps seperatly (and zero 
testing between them). as a result, in practice your proposal amounts to a 
big-bang approach, and/or results in releases that are known-broken.

and while you suggest putting this in -mm, remember that the -mm kernel 
needs to be useable so that people can test it, and it is on the same 
schedule as the main kernel so again you can't have known-broken things 
(of this scale) there either.

David Lang

-- 
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
  -- C.A.R. Hoare

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Stefan Richter]

David Lang wrote:
> Stefan,
>   we are supposed to be on a 2-month release cycle, with all major 
> changes going in in the first two weeks of that cycle. This timeframe 
> doesn't leave you any noticable time to implement your steps seperatly 
> (and zero testing between them). as a result, in practice your proposal 
> amounts to a big-bang approach, and/or results in releases that are 
> known-broken.

Experimental branches of subsystems usually cannot (nor need to) be
bound to a release cycle.

> and while you suggest putting this in -mm, remember that the -mm kernel 
> needs to be useable so that people can test it, and it is on the same 
> schedule as the main kernel so again you can't have known-broken things 
> (of this scale) there either.

I assumed there would be a few unmaintained bits (of marginal "scale")
left which could not be updated, but it seems now that won't be the
case.
-- 
Stefan Richter
-=====-=-=-= =-=- ==--=
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[David Lang]

On Sat, 22 Oct 2005, Stefan Richter wrote:

> Jeff Garzik wrote:
>> 
>> We differ on the path, not the goal.  As a thought experiment, you could 
>> try simply implementing the changes requested, and see where that goes.
>
> I doubt that the desired cleanup of the SCSI core could be done on-the-go, 
> i.e. without temporary breakage of larger parts of the subsystem (out of 
> mainline). But then again, I don't know much of the subsystem, so what am I 
> talking about here?
>
> Also, long-term breakage of smaller parts of the SCSI subsystem in mainline 
> is to be expected; breakage which is to be announced and scheduled.

Stefan, what you and Luben are missing is that big-bang changes like you 
are proposing are simply not acceptable anymore.

a few years ago when the 2.5 kernel series opened a similar big-bang 
approach was attempted for the IDE drivers. the instability that resulted 
(and rumors of the instability being worse then it was) eliminated a lot 
of people from testing things. things finally got bad enough that the 
entire system was reverted, and then a developer (one who had previously 
stated that such drastic changes were impossible) sat down and produced a 
long series of patches, each of which did a small amount of changes, each 
of which left the kernel in a working state (and each of which provided an 
advantage that could be identified at the time of the patch, either a 
better abstraction or a code cleanup). over a very few months (especially 
relative to the time spent working on the big-bang patches) the entire 
system was re-written.

This is what Jeff is trying to tell you. you can't just produce an 
entirely new SCSI subsystem and drop it into the kernel one day, you can 
all agree on a goal (this has almost been done, but nto quite), but that's 
only the first step. After you have some idea of the goal you then have to 
look at how to move to that goal without breaking things in the meantime. 
This requires that each step along the way keeps things working and is 
relativly straightforward in and of itself.

this definantly sounds a LOT harder then the 'throw it out and replace it 
all' approach, and it is (from the point of view of the programmer), 
however the result ends up being far more reliable as the process forces 
better examination of all the details, and allows more people to 
understand what's happening each step of the way. And since there are no 
releases that are made unuseable for people deliberatly, you also get 
extensive testing of all the steps along the way. This not only finds bugs 
sooner, it also makes it far more obvious where performance issues show up

so please accept that you aren't going to be able to replace any 
significant system in one massive change and instead start looking for 
ways to move the existing design towards where you want it to be and you 
will receive a lot of assistance in the process instead of banging heads 
with everyone.

David Lang

-- 
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
  -- C.A.R. Hoare

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Luben Tuikov]

--- David Lang <david.lang@digitalinsight.com> wrote:
> Stefan, what you and Luben are missing is that big-bang changes like you 
> are proposing are simply not acceptable anymore.

Take a look at the link below, working SAS code with the current
kernel.  No one is trying to produce big-bang changes.  This is what
Jeff want's you to believe that I'm trying to do -- this is part of his
political game and FUD and apparently he's succeeding.

I repeat: no on is trying to produce a new "big-bang changes" as you
call them. See the working code in the link at my sig.  This is
_current_ Linus's tree with SAS Stack in it.

Don't yield to the FUD.

> This is what Jeff is trying to tell you. you can't just produce an 
> entirely new SCSI subsystem and drop it into the kernel one day, you can 

No, this is what he wants you to believe that I'm trying to do.  This is what
people start doing when they loose on technological ground.  Do not
yield to the FUD and politics.  See the working code of current Linus's tree
with SAS in it in my sig link.

I repeat, no one is trying to replace anything or throw anything out.
This is just FUD and politics which Jeff is spreading because he's got
an agenda in convincing certain people certain things.

See the working SAS code in the link below -- current Linus' tree with
SAS in it.  Soon enough it would contain SDI back-end and then HP and LSI
would be able to plug right in, and I think there's an engineer out there
who's already implemented a front end.

    Luben


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Stefan Richter]

David Lang wrote:
> On Sat, 22 Oct 2005, Stefan Richter wrote:
>> I doubt that the desired cleanup of the SCSI core could be done 
>> on-the-go, i.e. without temporary breakage of larger parts of the 
>> subsystem (out of mainline). But then again, I don't know much of the 
>> subsystem, so what am I talking about here?
>>
>> Also, long-term breakage of smaller parts of the SCSI subsystem in 
>> mainline is to be expected; breakage which is to be announced and 
>> scheduled.
> 
> Stefan, what you and Luben are missing is that big-bang changes like you 
> are proposing are simply not acceptable anymore.

I agree with you that big-bang changes are not acceptable. This wasn't 
however what I had in mind.

> a few years ago when the 2.5 kernel series opened a similar big-bang 
> approach was attempted for the IDE drivers. the instability that 
> resulted (and rumors of the instability being worse then it was) 
> eliminated a lot of people from testing things. things finally got bad 
> enough that the entire system was reverted, and then a developer (one 
> who had previously stated that such drastic changes were impossible) sat 
> down and produced a long series of patches, each of which did a small 
> amount of changes, each of which left the kernel in a working state (and 
> each of which provided an advantage that could be identified at the time 
> of the patch, either a better abstraction or a code cleanup). over a 
> very few months (especially relative to the time spent working on the 
> big-bang patches) the entire system was re-written.

What I proposed was to "renovate" a _part_ of the SCSI subsystem (the 
core and the interfaces of the rest of the subsystem to the core) with 
the 2.6 kernel as a basis. This is a huge difference to 2.5. Changes 
took place everywhere in 2.5, many of them drastic.

I agree with you that the incremental approach is preferrable whenever 
possible. I even believe that this method could be applied to the SCSI 
core cleanup. However, concerns were voiced that this method would 
effectively lead to a double SCSI core: The old one, and a new one in 
parallel to it which doesn't share much code with the old one. (At least 
for some time, perhaps for a too long time.)

It has been said multiple times that this would not be desirable for 
reasons of /a/ more kernel bloat (while the goal was to remove existing 
bloat and lay foundations to avoid future bloat), and /b/ massive 
maintenance burden of two parallel infrastructures.

So that's why I said that *short-term* breakage right on top and right 
below the core should be accepted.

Again, the huge difference to the 2.5 times would be that all this would 
happen on top of a relatively stable kernel. (Stable in two senses.)

> This is what Jeff is trying to tell you. you can't just produce an 
> entirely new SCSI subsystem and drop it into the kernel one day, you can 

What I was referring to was to clean up a _part_ of the subsystem (the 
core), not to replace the subsystem. I admit though that my wording left 
much room for misunderstanding.

Furthermore, note that the "scsi-cleanup tree" which I referred to is 
not meant to be a fork. It should merely be another working stage before 
the -mm stage. And let me add that this stage should be left as soon as 
possible.

> all agree on a goal (this has almost been done, but nto quite), but 
> that's only the first step. After you have some idea of the goal you 
> then have to look at how to move to that goal without breaking things in 
> the meantime. This requires that each step along the way keeps things 
> working and is relativly straightforward in and of itself.
> 
> this definantly sounds a LOT harder then the 'throw it out and replace 
> it all' approach, and it is (from the point of view of the programmer), 
> however the result ends up being far more reliable as the process forces 
> better examination of all the details,

I agree with you on that. Again, although my post may have sounded like 
it, I did not want to advocate the "throw it out and replace it all" 
approach.

> and allows more people to 
> understand what's happening each step of the way.

Absolutely.

However I don't agree with you that _every_ little step must keep 
everything working. I believe that this may actually make the transition 
less easy to follow.

> And since there are no 
> releases that are made unuseable for people deliberatly, you also get 

I did not suggest to make unuseable releases.

> extensive testing of all the steps along the way. This not only finds 
> bugs sooner, it also makes it far more obvious where performance issues 
> show up

I agree with you on these advantages of the "(try to) keep everything 
working after each patch". Although the monitoring of performance is 
less important during the initial stage of the cleanup of the core.

> so please accept that you aren't going to be able to replace any 
> significant system in one massive change

Agreed. This is absolutely not what should be done.

> and instead start looking for 
> ways to move the existing design towards where you want it to be and you 
> will receive a lot of assistance in the process instead of banging heads 
> with everyone.

OK.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Christoph Hellwig]

On Sat, Oct 22, 2005 at 07:41:08PM +0200, Stefan Richter wrote:
> What I was referring to was to clean up a _part_ of the subsystem (the 
> core), not to replace the subsystem. I admit though that my wording left 
> much room for misunderstanding.
> 
> Furthermore, note that the "scsi-cleanup tree" which I referred to is 
> not meant to be a fork. It should merely be another working stage before 
> the -mm stage. And let me add that this stage should be left as soon as 
> possible.

gosh, could you please shut up and code now?

There's been various TODO items posted:

 (1) my TODO list for making the core HCIL idependent
 (2) finishing the transition to remove struct scsi_request and only
     send down S/G list
 (3) get rid of legacy host probing

and we'd all be happy if you added to the list.  Talking doesn't change
anything, please submit patches and help moving things forward.  Thanks.

And please start a new thread for such suggestions, this thread has finally
managed to get into my killfile.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Stefan Richter]

Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 07:41:08PM +0200, Stefan Richter wrote:
[obviously something annoying]
> 
> gosh, could you please shut up and code now?

I am shutting up right now, am in the process of learning more about 
SCSI and Linux' implementation of it, and will submit patches as soon as 
I am able to. This will take much time though as I am new here and 
cannot devote as much time to it as I liked, as it is not my profession.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Sergey Panov]

On Sat, 2005-10-22 at 18:51 +0100, Christoph Hellwig wrote:
> There's been various TODO items posted:

I thought "been posted" warrants at least a reference to the post. I am,
for one, have not seen that TODO list.

Sergey

===================================================================
I am expressing my personal opinion and not speaking for anyone else.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

I wrote:
>      1. renovate the core (thereby break all command set drivers and
>         all transport subsystems),
>      2. update ~2 command set drivers and ~2 transport subsystems
>      3. validate the renovated core,
>      4. fix the conceptual errors of the renovated core (as well as
>         first few discovered bugs in the implementation),
>      5. update all other command set drivers,
>      6. update all transport subsystems where resources to do so are
>         available,

Step 6 probably involves the creation of a SPI transport layer. It 
implements former SPI related functionality of the core and may receive 
former code of the core. BTW, it may be a good idea to really defer this 
to step 6 instead of doing so in step 2.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Jeff Garzik <jgarzik@pobox.com> wrote:
>
http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a
> 
> This effectively illustrates the wrong thing to do:  duplicating 
> information that's already in Scsi_Host_Template.

No, it doesn't.  It shows you that accomodating of this is so very easy. Easy
and straightforward, just as the comming shows.

> Just use Scsi_Host_Template in the LLDD and see where that goes.

We've done this already: adp94xx submitted earlier this year, rejected by
the community, because "common SAS tasks should go to a common layer".  Now
you _have_ that common layer, the SAS Transport Stack/Layer.  But the
requirements of the community seem to change on a daily basis.

It looks like "the community" got more than it bargained for, and since
it doesn't understand it, it wants to bastardize the code and the layering.

> > While the architecture in my mind is clear, I cannot do this myself
> > (and for all drivers).  Such a change would be gradual, involving more
> > than one developer, for more than one (new) driver, etc.
> 
> Correct.  That's why there is resistance to aic94xx's approach of 
> creating a totally new "strict SAM" path, existing in parallel with the 

No, it is not correct -- you complety turn around anything I say to make
yourself look right.

Here is what I meant: the resistance is because a few people of Linux SCSI
want things done _their_ way, using _their_ code, reinventing code and concepts
done already by engineers who use the technology every day.

Why else is there a COMPLETLY BROKEN and WRONG "SAS transport attributes"?
Politics and influence, thats why.  So that you can say "change already
existing code".  So in effect politically "correct" people can submit
COMPLETY WRONG AND BROKEN CODE with their name on it, and when there is
a proper code wating to go in, they can BLOCK IT and ask that
current BROKEN code be changed (beyond recognition) code which made it
in, NOT because of technological merit but because of political influence.

Linux SCSI people should start _listening_ to technology engineers who
work with this technology every day.  If they showed this open attitude,
I can tell you that _a lot more_ engineers would get involved to _improve_
further Linux SCSI, who are holding out now just because of this "my way
or no way" attitude of a few folks at Linux SCSI.

Furthermore, "aic94xx" is NOT CREATING ANYTHING NEW.  This is how
USB Storage and SBP already work.  There is clearly separated transport
layer, independent of the interconnect.  That layer depends on the type
of device connected.  Using such an infrastructure you can layer more
varied architectures, as opposed to dictating that aic94xx needs the
host template in it, short-circuting proper layring.

I repeat again: apd94xx already has the Host Template IN IT.
I repeat again: nothing in the Hardware of AIC-94XX Host Adapter has
_anything_ to do with Host Templates.  Host Templates are a SCSI thing.
The hardware implements an access point to the _interconnect_: Execute
Command SCSI RPC and TMFs.  It is completely backwards to force something
at a place where it does not belong.
I repeat again: USB Storage and SBP _already_ implement such an architecture
as shown by SAS.

Had we had multiple Transport Layers back when SCSI Core was written (imported
from BSD?) I can tell you that there'd be such a layering disposistion as
shown in USB/SBP/SAS _already_ in the kernel -- just because IT MAKES SENSE.

> traditional SCSI core.  You need to evolve the existing code to get 
> there.  Such changes are gradual, involving more than one developer, etc.

Thank you for the mockery.  What I meant is that I don't mind working towards
this goal, with storage engineers who _know_ and _understand_ SCSI Technology,
not necessarily SAS, but SCSI in general.  There is only a few folks
like that here at lsml who are active and at least one whom I know of
who is not active.

What actually seems to be happening at Linux SCSI is not "working together"
but "If you do not do it _my_ way, I'll spin your words and say that
you are not willing to ``work together''." which is _exactly_ what you
do.

Is it possible for Linux SCSI to learn that they are doing something wrong?
Is it possible for Linux SCSI to learn that they need to listen and learn
from company engineers, as opposed to doing things _their_ way?
Apparently not.

> We don't need one small set of SCSI drivers behaving differently from 
> the vast majority of existing SCSI drivers.

USB Storage and SBP are already like that.

Then again why are you saying this?  Would you loose control?
Cannot you see that such changes are necessary to go forward towards
the future?  Do you think that such changes would be happening only
in Linux SCSI?

If Linux overall has the attitude you are showing, Linux would go extinct
in just a few years.  Just enough time for such antics to get to paying
customers (who not necessarily have technological know-how, but it would
eventually get to them).

> Hear me now, and believe me later:  we all largely agree on the points

You are a very good politican -- just keep repeating this.
 
> you've raised about legacy crapola in the SCSI core.  James, Christoph, 
> myself, and several others disagree with your assertion that the old 
> SCSI core should exist in parallel with your new SCSI core.

This isn't my new SCSI Core.  I don't have a new SCSI Core.
But I'm flattered that you feel this way.

> We differ on the path, not the goal.  As a thought experiment, you could 
> try simply implementing the changes requested, and see where that goes.

Been there, done that: adp94xx, rejected by none other but "the community".
Here is now the Linux SAS Stack and aic94xx, working completely fine,
full source code, git tree, patch file, whole working tree (Linus' as of
Friday evening + SAS Stack + aic94xx) present at the link in my sig.

   Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Matthew Wilcox]

On Fri, Oct 21, 2005 at 03:22:09PM -0400, Luben Tuikov wrote:
> On 10/21/05 14:54, Jeff Garzik wrote:
> >
> > I'm trying to tell it like it is, in the hopes that you will eventually 
> > learn the process, and be a good upstream maintainer we can all work with.
> 
> Look how you're using "all", generalizing left and right.  You just want to
> create this FUD and spread this FUD that I don't work well with anyone.  Yep,
> this is a pretty low blow.  While in fact did you talk to everyone I work with?

Since you asked, I found you quite impossible to deal with at the OLS
BOF you held, and I wasn't the only one.  I was hoping you'd learn, but
you seem unwilling to listen to other people's ideas.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:41, Matthew Wilcox wrote:
> 
> Since you asked, I found you quite impossible to deal with at the OLS
> BOF you held, and I wasn't the only one.  I was hoping you'd learn, but
> you seem unwilling to listen to other people's ideas.

Be specific, be technical.  All I did was merely present SAS.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Matthew Wilcox]

On Fri, Oct 21, 2005 at 03:48:04PM -0400, Luben Tuikov wrote:
> On 10/21/05 15:41, Matthew Wilcox wrote:
> > 
> > Since you asked, I found you quite impossible to deal with at the OLS
> > BOF you held, and I wasn't the only one.  I was hoping you'd learn, but
> > you seem unwilling to listen to other people's ideas.
> 
> Be specific, be technical.  All I did was merely present SAS.

No.  What was advertised was a SCSI BOF which you then took over and
spent the entire time talking about the Adaptec SAS driver.  You weren't
interested in discussing wider SCSI issues.  You weren't interested in
talking about how other vendors implemented SAS.  You weren't interested
in discussing how we could get the best possible SAS interface in Linux.
You shut down other people when they tried to discuss these things.
It was a complete waste of time.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:54, Matthew Wilcox wrote:
> 
> No.  What was advertised was a SCSI BOF which you then took over and

It was advertised as "SAS BOF" -- the person who wrote that on the message
board (reading this list currently) can verify that.

> spent the entire time talking about the Adaptec SAS driver.  You weren't
> interested in discussing wider SCSI issues.  You weren't interested in
> talking about how other vendors implemented SAS.  You weren't interested
> in discussing how we could get the best possible SAS interface in Linux.

I still am.  What everyone now wants is SDI. And as you can see I've posted
several times _code_ and templates as to how to do a backend which would
work as per spec and a front end which would be adjustable to the whims of
"the community", sg/sysfs/whatever1/whatever2.

I think SDI will completely satisfy everyone's needs, independently of
the fact whether the the protocol is hidden in the FW or not.

In fact Fusion MPT is very cool: you only add a few PCI IDs and your
hw works with the same driver!  And if you care about protocol
specifics: use SDI.

But the community wanted involvement so then you say: "No! Give us
your hardware, we'll do it for you." and then you get into this
never-ending goose chase, implementing the wrong thing, the wrong way,
as opposed to _listening_ to what is actually wanted.

> You shut down other people when they tried to discuss these things.
> It was a complete waste of time.

Sorry you feel this way.  I don't remember you saying anything about
SAS.
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Arjan van de Ven]

> I say, when people are losing it on the technical front, they try to attack
> personally and on political basis.

Luben,

to be honest, a lot of people got exactly the impression that you've
been doing that from the start in this saga, by attacking Christophs and
James' integrity early on etc. Jeff has been just about the only person
who ignored that and kept talking to you, and now you're attacking him
too... I'm not sure what you're trying to achieve. But I'm pretty sure
this way isn't the best or easiest way to achieve that. 

Greetings,
   Arjan van de Ven

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:46, Arjan van de Ven wrote:
>>I say, when people are losing it on the technical front, they try to attack
>>personally and on political basis.
> 
> 
> Luben,
> 
> to be honest, a lot of people got exactly the impression that you've
> been doing that from the start in this saga, by attacking Christophs and
> James' integrity early on etc. Jeff has been just about the only person
> who ignored that and kept talking to you, and now you're attacking him
> too... I'm not sure what you're trying to achieve. But I'm pretty sure
> this way isn't the best or easiest way to achieve that. 

No, look back on this thread -- he started a personal attack first.
Here is the email:
http://marc.theaimsgroup.com/?l=linux-scsi&m=112991877716308&w=2

This is where the "you don't understand" personal attacks from Jeff
start.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

RE: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Salyzyn, Mark]

Luben sez:
>On 10/21/05 15:54, Matthew Wilcox wrote:
>> No.  What was advertised was a SCSI BOF which you then took over and
>It was advertised as "SAS BOF" -- the person who wrote that on the
message board (reading this list >currently) can verify that.

Yup. "SAS BOF"

Matthew, I am sorry your expectations for the BOF did not match. Heck,
*I* did not know where the
BOF would take us(!). Christoph voiced a concern that the principals
needed to get into the same
room together, and I rolled the ball ...

I do not believe it is productive to do a 'he sez, she sez', it
distracts us from the task(s) at hand.

-- Mark