RE: [PATCH 1/4] sas: add flag for locally attached PHYs

RE: [PATCH 1/4] sas: add flag for locally attached PHYs

[Moore, Eric Dean]

On Wednesday, October 19, 2005 3:00 PM, Luben Tuikov wrote: 

> 
> On 10/19/05 16:08, Moore, Eric Dean wrote:
> > Luben:  He is adding support for linkerrors and link/phy reset;
> > these for for CSMI/SDI API.
> > 
> > Can you in your driver retreive link errors for the links 
> on the expanders?
> 
> Yes. (offloaded to user space)
> 

I had a very good discussion with Luben over the 
phone last night.

Luben clarified this for me in regards to expanders:

http://www.t10.org/drafts.htm#sas2 
The linkerrors can be done by issuing a SMP passthru
sending SMP_REPORT_ERROR_LOG. Section 10.4.3.6

The link/phy reset can be done by issuing a SMP passthru
sending SMP_PHY_CONTROL. Section 10.4.3.10

The mpt fusion f/w supports smp passthru, and we could for
these attributes do SMP passthru if we wanted to return
this data for expanders.  Our f/w only provides a easy special 
wrapper config page for hba phys. For expanders, we
are going to have to do smp passthru.

So Christoph what should we do in driver?  Ignore returning
data for expanders?  

Luben suggestion(correct me if I'm wrong) is this should be 
done thru user space, instead of exporting this in the 
/sys/class/sas_phy attributes.  Thus we need to export a 
SMP_PASSTHRU mechanism.  I believe there is discussion
by Christoph ( and I believe Luben backs this idea) of doing 
all this thru the /dev/sg interface.


> 
> > What and where can I get this?
> 
> http://linux.adaptec.com/sas/
> Also see my sig.

Thanks. I will check it out.

Eric Moore

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 11:25, Moore, Eric Dean wrote:
> 
> I had a very good discussion with Luben over the 
> phone last night.
> 
> Luben clarified this for me in regards to expanders:
> 
> http://www.t10.org/drafts.htm#sas2 
> The linkerrors can be done by issuing a SMP passthru
> sending SMP_REPORT_ERROR_LOG. Section 10.4.3.6

Example of it being used is shown in this user space program
drivers/scsi/sas/expanders_conf.c:343:
static int report_phy_error_log(char *smp_portal_name, int phy_id)

> The link/phy reset can be done by issuing a SMP passthru
> sending SMP_PHY_CONTROL. Section 10.4.3.10

Example of it being use is shown in
drivers/scsi/sas/sas_expander.c:484:
static int smp_phy_control(struct domain_device *dev, int phy_id,
			   enum phy_func phy_func)

> The mpt fusion f/w supports smp passthru, and we could for
> these attributes do SMP passthru if we wanted to return
> this data for expanders.

> Our f/w only provides a easy special 
> wrapper config page for hba phys.

And this is correct in in line with SDI/SAS.

This method can be used by drivers/scsi/sdi.c.
And it fits both yours and our simplementation.

Dispatching to your implementation would go to FW, whose
interface you'll have to implement.  Remember you have
2 tasks: drivers/scsi/sdi.c and the FW dependent way
of dispatching this to the FW for LSI.

As to ours, it would go to

	/* Phy management */
	int (*lldd_control_phy)(struct sas_phy *, enum phy_func);

function stub in the of the sas_ha_struct structure as I showed
yesterday in this thread.

> For expanders, we
> are going to have to do smp passthru.

Makes sense.

> So Christoph what should we do in driver?  Ignore returning
> data for expanders?  
> 
> Luben suggestion(correct me if I'm wrong) is this should be 
> done thru user space, instead of exporting this in the 
> /sys/class/sas_phy attributes.  Thus we need to export a 
> SMP_PASSTHRU mechanism.  I believe there is discussion
> by Christoph ( and I believe Luben backs this idea) of doing 
> all this thru the /dev/sg interface.

sg would be a good choice if the ioctl headers can be "translated"
easily into sg headers -- this would make the whole task
so much easier and straightforward.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

On Thu, Oct 20, 2005 at 09:25:54AM -0600, Moore, Eric Dean wrote:
> So Christoph what should we do in driver?  Ignore returning
> data for expanders?  

As you mentioned doing link reset and reporting stats on expanders
is done using SMP, and thus totally independent of the driver used.
It should be either implemented in the transport class or in userland.
Doing it in the transport class has the advantage of a unified interface
for hba phys and expander phys, doing it in userland has the advantage
of less kernel bloat.

> Luben suggestion(correct me if I'm wrong) is this should be 
> done thru user space, instead of exporting this in the 
> /sys/class/sas_phy attributes.  Thus we need to export a 
> SMP_PASSTHRU mechanism.  I believe there is discussion
> by Christoph ( and I believe Luben backs this idea) of doing 
> all this thru the /dev/sg interface.

I've done a prototype using an SG_IO interface, but I'm not totally
happy about.  I've also seen lubens interface and I'm not totally
happy about it either.  Getting the interface right for SMP passthru
is very hard.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 12:01, Christoph Hellwig wrote:
> 
> As you mentioned doing link reset and reporting stats on expanders
> is done using SMP, and thus totally independent of the driver used.

Not quite. "Dispatching to" would be driver dependent.  If you implement
sdi_register_provider(struct ...); then you can make everyone happy
in the _shortest_ amount of time.

I'd suggest a reading of the SDI, SAS and SAM specs for an insight
of how it all ties together.

> It should be either implemented in the transport class or in userland.
> Doing it in the transport class has the advantage of a unified interface
> for hba phys and expander phys, doing it in userland has the advantage
> of less kernel bloat.

You already know the answer to that.

"customers" and "customer satisfaction" is a foreign
concept with the community, being the nature of Linux as it is,
but what you should want to is:
	- make less applications (which you may not know of)
	  break,
	- make less assumptions already taken break.

Those applications are already using SDI and only the interface
would change ("/dev/sg") if you were to implement
drivers/scsi/sdi.c.

> I've done a prototype using an SG_IO interface, but I'm not totally
> happy about.

Care to say why?

> I've also seen lubens interface and I'm not totally
> happy about it either.

Care to say why?

> Getting the interface right for SMP passthru is very hard.

1) You have the SDI spec which _tells you_ how to do it.
2) There are (HP) applications which already use that (1).

You have to stay focused on _what is required_.

Why are you trying to reinvent "SMP passthrough"?
Why not just make the customers happy?

If you implement drivers/scsi/sdi.c, as:

int  sdi_register_provider(struct sdi_provider_struct *sdi_prov);
void sdi_unregister_provider(struct sdi_provider_struct *sdi_prov);

everyone will be _more than happy_ to plug right into it.
And those are the only 2 functions you need to EXPORT_SYMBOL_GPL.

Everything else (SDI_ functions) would be in the struct as stubs
and the provider would set them if they implement them.

LSI/HP can plug right into it with *minimum of overhead*,
testing etc, both from their application side and from kernel
side.  And this is important to both customers and companies.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

On Thu, Oct 20, 2005 at 12:51:15PM -0400, Luben Tuikov wrote:
> On 10/20/05 12:01, Christoph Hellwig wrote:
> > 
> > As you mentioned doing link reset and reporting stats on expanders
> > is done using SMP, and thus totally independent of the driver used.
> 
> Not quite. "Dispatching to" would be driver dependent.  If you implement
> sdi_register_provider(struct ...); then you can make everyone happy
> in the _shortest_ amount of time.
> 
> I'd suggest a reading of the SDI, SAS and SAM specs for an insight
> of how it all ties together.

Thanks Luben.  You might not be able to imagine it, but there are people
other than you who are able to read specs, and despite having read them
they can actually talk like a normal human without referencing them in
every second sentence.  That beeing said SDI is not a final
specification.  It's a draft that's probablt not going to make it as a
t10 standard because the main pushers have backed of a bit again.

We are not going to implement SDI in the kernel.  Long before SDI or
it's predecessor, HP CSMI were designed we made it clear we're not going
to accept wide ioctl-APIs, especially when they're even bad for old
ioctl API standards.  The CSMI spec has been passed around in an early
phase and been totally rejected, I think even publically on linux-scsi.
HP decided to move ahead despite that and did a huge mis-services to
their customers.  It's not my problem if big companies can't listen and
do things their customers have to pay for, and it's certainly not our
job to work around their idiocy.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

> We are not going to implement SDI in the kernel.  Long before SDI or
> it's predecessor, HP CSMI were designed we made it clear we're not going
> to accept wide ioctl-APIs, especially when they're even bad for old
> ioctl API standards.  The CSMI spec has been passed around in an early
> phase and been totally rejected, 

Just to say AYE on this:

the problem with these things is that they define an IOCTL interface,
while a library level interface would have been a LOT better. By far.
That way the spec doesn't define the implementation, but only a real
INTERFACE while allowing flexibility and compatibility. IOCTLs are just
very very much the wrong layer to put such an interface. (just as
posix/sus specify the (g)libc interface and not the kernel interface,
although of course they impact it somewhat and glibc can for several
things do a straight pass-through to the raw kernel interface).

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 13:22, Arjan van de Ven wrote:
>>We are not going to implement SDI in the kernel.  Long before SDI or
>>it's predecessor, HP CSMI were designed we made it clear we're not going
>>to accept wide ioctl-APIs, especially when they're even bad for old
>>ioctl API standards.  The CSMI spec has been passed around in an early
>>phase and been totally rejected, 
> 
> 
> Just to say AYE on this:
> 
> the problem with these things is that they define an IOCTL interface,
> while a library level interface would have been a LOT better. By far.
> That way the spec doesn't define the implementation, but only a real
> INTERFACE while allowing flexibility and compatibility. IOCTLs are just
> very very much the wrong layer to put such an interface. (just as
> posix/sus specify the (g)libc interface and not the kernel interface,
> although of course they impact it somewhat and glibc can for several
> things do a straight pass-through to the raw kernel interface).

Hi Arjan, how are you?

As I mentioned already, you do not have to take section 4 of SDI
literally.  If you implement section 5 of SDI, the back end, then
the front end is ajustable: sg/sysfs/whathaveyou1/whathaveyou2, etc.

Here is an excerpt sent by me to another storage engineer:
--cut--
Example of include/scsi/sdi/sdi.h (back-end):

/* This struct describes section 5 of the SDI spec */
struct sdi_functions_struct {
	int (*sdi_get_driver_info)(struct sdi_header *header,
				   struct sdi_driver_info *info);
	int (*sdi_get_controller_config)(struct sdi_header *header,
					 struct sdi_controller_config *config);
	/* the rest of the stub SDI functions defined here */
	...
};
	
/* This struct describes the provider */
struct sdi_provider_struct {
	char *provider_name; /* some name, whatever, may not be necessary */
	...
	struct sdi_functions_struct sdi_functions;   /* see above */
	...
};

And then the _calling_ of those functions is either via sg or sysfs
or whatever makes people happy (done in drivers/scsi/sdi.c).

Dispatching of those is driver/fw/hw/whatever dependent depending on
the provider and what they registered.

This will live in drivers/scsi/sdi.c and include/scsi/sdi/sdi.h
--cut--

You see?  You don't need IOCTLs, yet you can be fully compliant
to the spec so that already existing user space management programs
can transition with minimal change to their code (over to the front
end: sg/whatever).  This cuts down production costs in testing,
verification/etc.  Makes things cheaper to deploy and customers
are happy.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

On Thu, 2005-10-20 at 16:10 -0400, Luben Tuikov wrote:
> You see?  You don't need IOCTLs, yet you can be fully compliant
> to the spec so that already existing user space management programs
> can transition with minimal change to their code (over to the front
> end: sg/whatever). 

this is EXACTLY my point.  A proper API would NOT need the userland apps
to change if the underlying mechanism changes. Or in other words: today
it may be SG, tomorrow BSG or SysFS, and when we open 2.7 the mechanism
may be WokkieWokkie. The implementation of the interface library may
need to change for this, but the management app should not. 

There is a parallel in the glibc/posix/sus world: With the 2.6 kernel
there is a new syscall mechanism (sysenter) on x86 kernels available.
NONE of the applications needed changing, fopen() remained fopen(), not
just as API but also as ABI. All that needed some adjustments were glibc
internals.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 13:03, Christoph Hellwig wrote:
>>I'd suggest a reading of the SDI, SAS and SAM specs for an insight
>>of how it all ties together.
> 
> 
> Thanks Luben.  You might not be able to imagine it, but there are people
> other than you who are able to read specs, and despite having read them
> they can actually talk like a normal human without referencing them in
> every second sentence.  That beeing said SDI is not a final
> specification.  It's a draft that's probablt not going to make it as a
> t10 standard because the main pushers have backed of a bit again.

The whole point is that LSI/HP/Adaptec are _using_ SDI in one form
or another and would like to have it officially in the kernel.

> We are not going to implement SDI in the kernel.  Long before SDI or

Who is "we"?

> it's predecessor, HP CSMI were designed we made it clear we're not going
> to accept wide ioctl-APIs, especially when they're even bad for old

You do not have to take section 4 of SDI literally, but what everyone
wants is the _functionality_ of section 5 of SDI.  That can be implemented
as a backend, see my next message to Arjan on this thread, and the front
end can be anything at all (sg/sysfs/whatever1/whatever2).

> ioctl API standards.  The CSMI spec has been passed around in an early
> phase and been totally rejected, I think even publically on linux-scsi.

Rejected by whom?  "The community" or by you?

Let me understand this properly: LSI/HP/Adaptec are using SDI in one
form or another and would like to see something like it in the kernel,
but "the community", the "we" you refer to above, rejects it?

> HP decided to move ahead despite that and did a huge mis-services to
> their customers.

Yes, because there is a _need_ for it.

> It's not my problem if big companies can't listen and do things their
> customers have to pay for, and it's certainly not our job to work
> around their idiocy.

Bold statment.

Who should "big companies" listen to?  You?  "The community?"
Are you saying "big companies" whould listen to Linux which
says "no to specs" among other things?

Often enough what "big companies" agree on and use and deploy is
what Linux (you?) should _listen_ to, try to understand and maybe
get out of the way.

It is all about customer satisfaction, which is completely
foreign to Linux, simply because of the _nature_ of Linux.

Why is there a need to _reinvent_ things?
Aren't there engineers at those "big companies" who know
OS design and programming?

Why do you feel the need to antiquate engineers from
companies submitting drivers/patches/solutions to the
Linux kernel?

Surely this isn't good for the Linux kernel in the long term.
Or is it?

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Andrew Patterson]

On Thu, 2005-10-20 at 16:02 -0400, Luben Tuikov wrote:
> On 10/20/05 13:03, Christoph Hellwig wrote:
> >>I'd suggest a reading of the SDI, SAS and SAM specs for an insight
> >>of how it all ties together.
> > 
> > 
> > Thanks Luben.  You might not be able to imagine it, but there are people
> > other than you who are able to read specs, and despite having read them
> > they can actually talk like a normal human without referencing them in
> > every second sentence.  That beeing said SDI is not a final
> > specification.  It's a draft that's probablt not going to make it as a
> > t10 standard because the main pushers have backed of a bit again.
> 
> The whole point is that LSI/HP/Adaptec are _using_ SDI in one form
> or another and would like to have it officially in the kernel.

I would hope that other users/vendors might find a use for this as well.

> 
> > We are not going to implement SDI in the kernel.  Long before SDI or
> 
> Who is "we"?
> 
> > it's predecessor, HP CSMI were designed we made it clear we're not going
> > to accept wide ioctl-APIs, especially when they're even bad for old
> 

CSMI is a semi-proprietary spec.  SDI is an attempt to turn this into an
open spec.  One of the reasons it has not been made into a standard is
the whole IOCTL issue.  Other OS's, Windows, HP-UX, Solaris allow them,
Linux has recently banned them (at least in the SCSI sub-system). 

> You do not have to take section 4 of SDI literally, but what everyone
> wants is the _functionality_ of section 5 of SDI. 

My guess is that what current app writers want is to use IOCTL's so they
don't have to special-case Linux.  Next best thing would be something
that closely approaches it, to avoid re-writing a lot of code.  

>  That can be implemented
> as a backend, see my next message to Arjan on this thread, and the front
> end can be anything at all (sg/sysfs/whatever1/whatever2).
> 
> > ioctl API standards.  The CSMI spec has been passed around in an early
> > phase and been totally rejected, I think even publically on linux-scsi.

The LSI driver attempted go in with CSMI. One of the reasons it was
rejected was because it used IOCTL's to implement CSMI.  I don't think
CSMI itself was rejected (other than the fact that is required IOCTL's
of course).  The SDI authors are willing to use another type of
interface, but have had no help in suggesting alternatives.

> 
> Rejected by whom?  "The community" or by you?

I believe there is a common understanding that IOCTL's are bad and
should be avoided. See:

http://lkml.org/lkml/2001/5/20/81

> 
> Let me understand this properly: LSI/HP/Adaptec are using SDI in one
> form or another and would like to see something like it in the kernel,
> but "the community", the "we" you refer to above, rejects it?
> 
> > HP decided to move ahead despite that and did a huge mis-services to
> > their customers.

Perhaps.  Given that there seems to be no alternative, we don't have
much of a choice.

> 
> Yes, because there is a _need_ for it.
> 
> > It's not my problem if big companies can't listen and do things their
> > customers have to pay for, and it's certainly not our job to work
> > around their idiocy.

Yes, CSMI should have had more Linux input when it was developed.  The
no-new IOCTL policy certainly came as a surprise to the authors. Still,
there doesn't seem to be any other usable cross-platform interface that
is acceptable to the linux community (or at least to Christoph)'s corner
of it).  My personal preference is to hide this stuff in a library, so
the kernel implementation is hidden. But even a library needs an
underlying kernel implementation.

> 
> Bold statment.
> 
> Who should "big companies" listen to?  You?  "The community?"
> Are you saying "big companies" whould listen to Linux which
> says "no to specs" among other things?
> 
> Often enough what "big companies" agree on and use and deploy is
> what Linux (you?) should _listen_ to, try to understand and maybe
> get out of the way.
> 

Big companies often want to do things in a proprietary fashion.  I
personally would prefer to see a standard's-based approach.

> It is all about customer satisfaction, which is completely
> foreign to Linux, simply because of the _nature_ of Linux.

More bold statements? ;-)

> 
> Why is there a need to _reinvent_ things?
> Aren't there engineers at those "big companies" who know
> OS design and programming?
> 
> Why do you feel the need to antiquate engineers from
> companies submitting drivers/patches/solutions to the
> Linux kernel?
> 
> Surely this isn't good for the Linux kernel in the long term.
> Or is it?
> 
> 	Luben

ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Andrew Patterson wrote:
> I believe there is a common understanding that IOCTL's are bad and
> should be avoided. See:
> 
> http://lkml.org/lkml/2001/5/20/81

Yep.  Linus's rant here reflects not only his opinion, but general 
consensus, I feel.


> Yes, CSMI should have had more Linux input when it was developed.  The
> no-new IOCTL policy certainly came as a surprise to the authors. Still,
> there doesn't seem to be any other usable cross-platform interface that
> is acceptable to the linux community (or at least to Christoph)'s corner

Beyond Linus's rant, ioctls have a practical limitation, too:  because 
they are untyped, we have to deal with stuff like the 32<->64 compat 
ioctl thunking.

Consider what an ioctl is, overall:  a domain-specific "do this 
operation" interface.  Which, further, is nothing but a wrapping of a 
"send message" + "receive response" interface.  There are several ways 
to do this in Linux:

* block driver.  a block driver is nothing but a message queue.  This is 
why James has suggested implementing SMP as a block driver.  People get 
stuck into thinking "block driver == block device", which is wrong.  The 
Linux block layer is nothing but a message queueing interface.

* netlink.  You connect to <an object> and send/receive messages.  Not 
strictly limited to networking, as this is used in some areas of the 
kernel now for generic event delivery.

* char driver.  Poor man's netlink.  Unless its done right, it suffers 
from the same binary problems as ioctls.  I don't recommend this path.

* sysfs.  This has no inherent message/queue properties by itself, and 
is less structured than blockdrvr or netlink, so dealing with more than 
one outstanding operation at a time requires some coding.

sysfs's attractiveness is in its ease of use.  It works with standard 
Unix filesystem tools.  You don't need to use a library to read 
information, cat(1) often suffices.  sysfs, since its normal ASCII 
(UTF8), also has none of the annoying 32<->64 compatibility issues that 
ioctls suffer from.

Which is best?  I don't have a good answer.  Largely depends on the 
situation, particularly queueing needs.  Networking and storage are 
rapidly converging into "messaging", so the situation is highly fluid in 
any case.  Coming from a networking background, I sorta lean towards the 
solution noone has attempted yet:  netlink.


> of it).  My personal preference is to hide this stuff in a library, so
> the kernel implementation is hidden. But even a library needs an
> underlying kernel implementation.

Strongly agree here.  libc shelters userspace from [most] kernel 
changes, by exporting syscalls in a standard manner.  alsa-lib was 
created to shelter userspace from current and future changes in the 
kernel audio interface.  libsdi is quite reasonable.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Mike Christie]

Jeff Garzik wrote:
> 
> Which is best?  I don't have a good answer.  Largely depends on the 
> situation, particularly queueing needs.  Networking and storage are 
> rapidly converging into "messaging", so the situation is highly fluid in 
> any case.  Coming from a networking background, I sorta lean towards the 
> solution noone has attempted yet:  netlink.

Dimitry and Alex did netlink for scsi_tranport_iscsi.c in scsi-misc. 
Which reminds me of some of the problems they discovered. See here
http://marc.theaimsgroup.com/?l=linux-netdev&m=111273099708516&w=2

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Douglas Gilbert]

Jeff Garzik wrote:
> Andrew Patterson wrote:
> 
>> I believe there is a common understanding that IOCTL's are bad and
>> should be avoided. See:
>>
>> http://lkml.org/lkml/2001/5/20/81
> 
> 
> Yep.  Linus's rant here reflects not only his opinion, but general
> consensus, I feel.

Jeff,
ioctls represent the most direct, unimpeded (by OS
policy) route between the user space and a driver,
potentially a few levels down a driver stack.
I see it as a control issue, in one corner there are
Microsoft and Linux while in the other there are
the hardware vendors. OS vendors come out with
ioctl replacements but can't resist enforcing policy.

As for type safety in linux, I stopped taking that
seriously when practicality of having C++ code
in the kernel was killed by "struct class".

>> Yes, CSMI should have had more Linux input when it was developed.  The
>> no-new IOCTL policy certainly came as a surprise to the authors. Still,
>> there doesn't seem to be any other usable cross-platform interface that
>> is acceptable to the linux community (or at least to Christoph)'s corner
> 
> 
> Beyond Linus's rant, ioctls have a practical limitation, too:  because
> they are untyped, we have to deal with stuff like the 32<->64 compat
> ioctl thunking.

Do you know where there are some clear guidelines
on the use of pointers in a structure passed to an
ioctl to lessen (or bypass) 32<->64 compat ioctl
thunking?

> Consider what an ioctl is, overall:  a domain-specific "do this
> operation" interface.  Which, further, is nothing but a wrapping of a
> "send message" + "receive response" interface.  There are several ways
> to do this in Linux:
> 
> * block driver.  a block driver is nothing but a message queue.  This is
> why James has suggested implementing SMP as a block driver.  People get
> stuck into thinking "block driver == block device", which is wrong.  The
> Linux block layer is nothing but a message queueing interface.
> 
> * netlink.  You connect to <an object> and send/receive messages.  Not
> strictly limited to networking, as this is used in some areas of the
> kernel now for generic event delivery.
> 
> * char driver.  Poor man's netlink.  Unless its done right, it suffers
> from the same binary problems as ioctls.  I don't recommend this path.
> 
> * sysfs.  This has no inherent message/queue properties by itself, and
> is less structured than blockdrvr or netlink, so dealing with more than
> one outstanding operation at a time requires some coding.
> 
> sysfs's attractiveness is in its ease of use.  It works with standard
> Unix filesystem tools.  You don't need to use a library to read
> information, cat(1) often suffices.  sysfs, since its normal ASCII
> (UTF8), also has none of the annoying 32<->64 compatibility issues that
> ioctls suffer from.

... and on the other side for sysfs are the loss of
layered error reporting, inability to supply auxiliary
attributes such as a request timeout and the possibility
that write()s will either be limited in size or segmented.

> Which is best?  I don't have a good answer.  Largely depends on the
> situation, particularly queueing needs.  Networking and storage are
> rapidly converging into "messaging", so the situation is highly fluid in
> any case.  Coming from a networking background, I sorta lean towards the
> solution noone has attempted yet:  netlink.

damn, we agree :-)

All in all, this is a good summary of the options available.

>> of it).  My personal preference is to hide this stuff in a library, so
>> the kernel implementation is hidden. But even a library needs an
>> underlying kernel implementation.
> 
> 
> Strongly agree here.  libc shelters userspace from [most] kernel
> changes, by exporting syscalls in a standard manner.  alsa-lib was
> created to shelter userspace from current and future changes in the
> kernel audio interface.  libsdi is quite reasonable.

Doug Gilbert

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Douglas Gilbert wrote:
> ioctls represent the most direct, unimpeded (by OS
> policy) route between the user space and a driver,
> potentially a few levels down a driver stack.

With the inevitable result that most ioctl code is poorly written, and 
causes bugs and special case synchronization problems :)  Driver writers 
love to stuff way too much stuff into ioctls, without thinking about 
overall design.


> I see it as a control issue, in one corner there are
> Microsoft and Linux while in the other there are
> the hardware vendors. OS vendors come out with
> ioctl replacements but can't resist enforcing policy.
> 
> As for type safety in linux, I stopped taking that
> seriously when practicality of having C++ code
> in the kernel was killed by "struct class".

C++ was never ever meant to work with the kernel.


>>>Yes, CSMI should have had more Linux input when it was developed.  The
>>>no-new IOCTL policy certainly came as a surprise to the authors. Still,
>>>there doesn't seem to be any other usable cross-platform interface that
>>>is acceptable to the linux community (or at least to Christoph)'s corner
>>
>>
>>Beyond Linus's rant, ioctls have a practical limitation, too:  because
>>they are untyped, we have to deal with stuff like the 32<->64 compat
>>ioctl thunking.
> 
> 
> Do you know where there are some clear guidelines
> on the use of pointers in a structure passed to an
> ioctl to lessen (or bypass) 32<->64 compat ioctl
> thunking?

Its impossible.  If you pass pointers, you need to thunk.  (Not even 
passing pointers via sysfs can avoid this.)  Consider running a 32-bit 
app (with 32-bit pointers and 32-bit ABI) on a 64-bit kernel.


>>Consider what an ioctl is, overall:  a domain-specific "do this
>>operation" interface.  Which, further, is nothing but a wrapping of a
>>"send message" + "receive response" interface.  There are several ways
>>to do this in Linux:
>>
>>* block driver.  a block driver is nothing but a message queue.  This is
>>why James has suggested implementing SMP as a block driver.  People get
>>stuck into thinking "block driver == block device", which is wrong.  The
>>Linux block layer is nothing but a message queueing interface.
>>
>>* netlink.  You connect to <an object> and send/receive messages.  Not
>>strictly limited to networking, as this is used in some areas of the
>>kernel now for generic event delivery.
>>
>>* char driver.  Poor man's netlink.  Unless its done right, it suffers
>>from the same binary problems as ioctls.  I don't recommend this path.
>>
>>* sysfs.  This has no inherent message/queue properties by itself, and
>>is less structured than blockdrvr or netlink, so dealing with more than
>>one outstanding operation at a time requires some coding.
>>
>>sysfs's attractiveness is in its ease of use.  It works with standard
>>Unix filesystem tools.  You don't need to use a library to read
>>information, cat(1) often suffices.  sysfs, since its normal ASCII
>>(UTF8), also has none of the annoying 32<->64 compatibility issues that
>>ioctls suffer from.
> 
> 
> ... and on the other side for sysfs are the loss of
> layered error reporting,

Only for the most simple interfaces.  sysfs is for exporting kernel data 
structures to userspace, using read(2) and write(2).  You can quite 
literally accomplish anything with that.  sysfs could export an event 
directory entry, and a response directory entry.  The response dirent 
could provide all the error reporting you could ever want.  That's just 
a 2-second off-the-cuff example.


> inability to supply auxiliary
> attributes such as a request timeout and the possibility
> that write()s will either be limited in size or segmented.

This entirely depends on the interface you define.  These problems are 
all surmountable.

Note, I'm _not_ suggesting this is the best route for an SMP interface, 
just stating sysfs generalities.  sysfs is flexible enough that it could 
completely replace SG_IO (though that would not be a wise idea).

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 02:19, Jeff Garzik wrote:
> With the inevitable result that most ioctl code is poorly written, and 
> causes bugs and special case synchronization problems :)  Driver writers 
> love to stuff way too much stuff into ioctls, without thinking about 
> overall design.

Could you please use adjectives, like "some" or "all" before
"Device writers"?  Could you also use a qualifier like, "excluding me",
etc.  Thanks.

> Only for the most simple interfaces.  sysfs is for exporting kernel data 
> structures to userspace, using read(2) and write(2).  You can quite 
> literally accomplish anything with that.  sysfs could export an event 
> directory entry, and a response directory entry.  The response dirent 
> could provide all the error reporting you could ever want.  That's just 
> a 2-second off-the-cuff example.

For the 3-second "off-the-cuff example" see how the "smp_portal" sysfs
binary attribute works in drivers/scsi/sas/sas_expander.c at the bottom
of the file, whereby user space first writes and then reads as much data
it is expecting.

(Thus there is _no_ hardcoding of the amount of data SMP request can
return (as is done in SDI).)

Here are the tiny functions for refresh:

/* ---------- SMP portal ---------- */

static ssize_t smp_portal_write(struct kobject *kobj, char *buf, loff_t offs,
				size_t size)
{
	struct domain_device *dev = to_dom_device(kobj);
	struct expander_device *ex = &dev->ex_dev;

	if (offs != 0)
		return -EFBIG;
	else if (size == 0)
		return 0;

	down_interruptible(&ex->smp_sema);
	if (ex->smp_req)
		kfree(ex->smp_req);
	ex->smp_req = kzalloc(size, GFP_USER);
	if (!ex->smp_req) {
		up(&ex->smp_sema);
		return -ENOMEM;
	}
	memcpy(ex->smp_req, buf, size);
	ex->smp_req_size = size;
	ex->smp_portal_pid = current->pid;
	up(&ex->smp_sema);

	return size;
}

static ssize_t smp_portal_read(struct kobject *kobj, char *buf, loff_t offs,
			       size_t size)
{
	struct domain_device *dev = to_dom_device(kobj);
	struct expander_device *ex = &dev->ex_dev;
	u8 *smp_resp;
	int res = -EINVAL;

	down_interruptible(&ex->smp_sema);
	if (!ex->smp_req || ex->smp_portal_pid != current->pid ||
	    offs != ex->smp_req_size)
		goto out;

	res = 0;
	if (size == 0)
		goto out;

	res = -ENOMEM;
	smp_resp = alloc_smp_resp(size);
	if (!smp_resp)
		goto out;
	res = smp_execute_task(dev, ex->smp_req, ex->smp_req_size,
			       smp_resp, size);
	if (!res) {
		memcpy(buf, smp_resp, size);
		res = size;
	}

	kfree(smp_resp);
out:
	kfree(ex->smp_req);
	ex->smp_req = NULL;
	ex->smp_req_size = 0;
	ex->smp_portal_pid = -1;
	up(&ex->smp_sema);
	return res;
}

> Note, I'm _not_ suggesting this is the best route for an SMP interface, 
> just stating sysfs generalities.  sysfs is flexible enough that it could 
> completely replace SG_IO (though that would not be a wise idea).

Indeed, sysfs _is_ very flexible, and with a single process open(2) it could
satisfy the transaction needed functionality.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/20/05 20:46, Jeff Garzik wrote:
> Consider what an ioctl is, overall:  a domain-specific "do this 
> operation" interface.  Which, further, is nothing but a wrapping of a 
> "send message" + "receive response" interface.  There are several ways 
> to do this in Linux:
> 
> * block driver.  a block driver is nothing but a message queue.  This is 

Not quite.  This maybe the way it operates, but it is called "block"
for a reason.

> why James has suggested implementing SMP as a block driver.  People get 
> stuck into thinking "block driver == block device", which is wrong.  The 
> Linux block layer is nothing but a message queueing interface.

Now, just because James suggested implementing the SMP service as a block
device you think this is the right thing to do?

How about this: Why not as a char device?

At least MS isn't suffering from the "no to specs" syndrome which
the Linux community seems to be suffering...

SMP as a Block device????

You need to see the history of SMP and its intended use.

> * netlink.  You connect to <an object> and send/receive messages.  Not 
> strictly limited to networking, as this is used in some areas of the 
> kernel now for generic event delivery.

This is very similar to the binary attr file "smp_portal" in
drivers/scsi/sas/sas_expander.c, bottom of file.

> * char driver.  Poor man's netlink.  Unless its done right, it suffers 
> from the same binary problems as ioctls.  I don't recommend this path.
> 
> * sysfs.  This has no inherent message/queue properties by itself, and 
> is less structured than blockdrvr or netlink, so dealing with more than 
> one outstanding operation at a time requires some coding.

Exactly! SMP. (binary attr file)

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Christoph Hellwig]

On Fri, Oct 21, 2005 at 01:48:45PM -0400, Luben Tuikov wrote:
> > why James has suggested implementing SMP as a block driver.  People get 
> > stuck into thinking "block driver == block device", which is wrong.  The 

                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> > Linux block layer is nothing but a message queueing interface.
> 
> Now, just because James suggested implementing the SMP service as a block
> device you think this is the right thing to do?
> 
> How about this: Why not as a char device?

you can implement a char device using the block layer.  See
drivers/scsi/{ch.c,osst.c,sg.c,st.c} for examples.

That beeing said I tried this approach.  It looks pretty cool when you
think about it, but the block layer is quite a bit too heavyweight for
queueing up a few SMP requests, and we need to carry too much useless
code around for it.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:04, Christoph Hellwig wrote:
>>How about this: Why not as a char device?
> 
> you can implement a char device using the block layer.  See
> drivers/scsi/{ch.c,osst.c,sg.c,st.c} for examples.

Christoph, you failed to see that my question was _rhetorical_.

> That beeing said I tried this approach.  It looks pretty cool when you
> think about it, but the block layer is quite a bit too heavyweight for
> queueing up a few SMP requests, and we need to carry too much useless
> code around for it.

That's the last reason not to implement SMP as a block device.
But this is good that you tried it and it "flopped".  This way
people will stop repeating "SMP... block device".

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Matthew Wilcox]

On Fri, Oct 21, 2005 at 02:12:49PM -0400, Luben Tuikov wrote:
> > That beeing said I tried this approach.  It looks pretty cool when you
> > think about it, but the block layer is quite a bit too heavyweight for
> > queueing up a few SMP requests, and we need to carry too much useless
> > code around for it.
> 
> That's the last reason not to implement SMP as a block device.
> But this is good that you tried it and it "flopped".  This way
> people will stop repeating "SMP... block device".

Block layer != Block device.

Nobody wants to implement SMP as a block device.

The question is whether the SMP interface should be implemented as part
of the block layer.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Douglas Gilbert]

Matthew Wilcox wrote:
> On Fri, Oct 21, 2005 at 02:12:49PM -0400, Luben Tuikov wrote:
> 
>>>That beeing said I tried this approach.  It looks pretty cool when you
>>>think about it, but the block layer is quite a bit too heavyweight for
>>>queueing up a few SMP requests, and we need to carry too much useless
>>>code around for it.
>>
>>That's the last reason not to implement SMP as a block device.
>>But this is good that you tried it and it "flopped".  This way
>>people will stop repeating "SMP... block device".
> 
> 
> Block layer != Block device.
> 
> Nobody wants to implement SMP as a block device.
> 
> The question is whether the SMP interface should be implemented as part
> of the block layer.

However, the block layer is used in the context of a
block device (and in some cases a char device).
If SAS domain discovery is done from the user space, and
the root file system is the far side of a SAS expander,
there are no suitable devices, just the SAS initiator
(HBA) which currently we cannot address via the block layer.


Doug Gilbert

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Douglas Gilbert wrote:
> However, the block layer is used in the context of a
> block device (and in some cases a char device).
> If SAS domain discovery is done from the user space, and
> the root file system is the far side of a SAS expander,
> there are no suitable devices, just the SAS initiator
> (HBA) which currently we cannot address via the block layer.

Invalid example.  All of the methods listed -- request_queue, netlink, 
chrdev, sysfs, ioctl -- will work just fine when the root filesystem is 
on the far side of a SAS expander.  These are just methods of 
communication, nothing more.

In your example -- userspace discovery required before root filesystem 
can be found -- a program running from initrd/initramfs would create an 
SMP device node, open it, and then proceed with the discovery and 
configuration process, which in turn creates the device nodes necessary 
to mount the root filesystem.

A request_queue is just a queue.  You are in complete control of who are 
the producer(s) of requests, and who are consumer(s).

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Jeff Garzik wrote:
> Douglas Gilbert wrote:
>> However, the block layer is used in the context of a
>> block device (and in some cases a char device).
>> If SAS domain discovery is done from the user space, and
>> the root file system is the far side of a SAS expander,
>> there are no suitable devices, just the SAS initiator
>> (HBA) which currently we cannot address via the block layer.

> Invalid example.  All of the methods listed -- request_queue, netlink, 
> chrdev, sysfs, ioctl -- will work just fine when the root filesystem is 
> on the far side of a SAS expander.  These are just methods of 
> communication, nothing more.
> 
> In your example -- userspace discovery required before root filesystem 
> can be found -- a program running from initrd/initramfs would create an 
> SMP device node, open it, and then proceed with the discovery and 
> configuration process, which in turn creates the device nodes necessary 
> to mount the root filesystem.
> 
> A request_queue is just a queue.  You are in complete control of who are 
> the producer(s) of requests, and who are consumer(s).


Since people are having such a tough time grasping the use of 
request_queue without an associated block device, here is a concrete 
example:  drivers/block/sx8.c.

sx8 creates a queue (grep for 'oob_q') specifically for handling 
discovery and configuration requests.  The only requests sent to this 
queue are I2O-ish management commands, never reads or writes.  Since 
they are management commands, these requests are NEVER associated with a 
block device.  Further, when sx8 discovery begins, sx8 block devices 
(and associated request_queues) simply don't exist.

Although sx8 management is entirely in-kernel, one could easily imagine 
how a userland interface (chrdev?) submits userspace commands into this 
queue.  Further, one can see how a host adapter could register one or 
more queues specifically for the transit of SMP commands.

NOTE:  THIS IS NOT AN ENDORSEMENT OF REQUEST QUEUES FOR SMP.  I merely 
wish to clear up misunderstandings about the block layer found in this 
thread.

It remains an open question whether the _complexity_ of this approach is 
more than is warranted for SMP.  But we've departed from that question, 
in this sub-thread :)

I merely illustrate that the block layer is being used _today_ for 
management commands.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Jeff Garzik <jgarzik@pobox.com> wrote:
> Since people are having such a tough time grasping the use of 

Please drop the generalizaion and FUD.

You are missing the point, and the point is that using the
block layer for SMP is so, so heavyweight, that no sane OS engineer
would do that.

Ready to see example:
 - see "smp_portal" in the complete and working SAS Transport Layer/Stack
from the 1st link of my signature.  See that the overhead of sending an SMP
task from user space to the domain device is _zero_, zero overhead.  As soon
as the user process does a read(2), the SMP task it wrote(2), is immediately
sent to the domain device, using the interconnect.  No bloat, no going around
in circles.

SDI does exactly the same things as it binds to the controller, the PCI device
implementing the domain interconnect functions (exported via SDI).

> It remains an open question whether the _complexity_ of this approach is 
> more than is warranted for SMP.  But we've departed from that question, 
> in this sub-thread :)

Let everyone see what is happening:
1. The industry submits a driver which clearly shows that SMP is not a block
device and there is no technical merit to do it this way.  The industry shows
a proper implementation of SMP access to the domain.
2. Then the Linux SCSI community decides, on their own, without ANY TECHNICAL
MERIT, or knowlege of the technology that they should try to implement SMP as
a block device.  Then they see that this isn't viable since it adds bloat and
SMP doesn't warrant it, and then we're back to 1.

The parent says: here is how you do it.  The child though, ignores the parent's
advice and does it its own way only to find out that the parent was right
all along.

My question is: at what point is this process going to stop?

   Luben


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Francois Romieu]

Luben Tuikov <ltuikov@yahoo.com> :
[...]
> Let everyone see what is happening:

Is there any chance that you can put this kind of content on a different
medium, say blob, rss feed, whatever instead of loading l-k ?

--
Ueimor

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Jeff Garzik <jgarzik@pobox.com> wrote:
> Invalid example.  All of the methods listed -- request_queue, netlink, 
> chrdev, sysfs, ioctl -- will work just fine when the root filesystem is 
> on the far side of a SAS expander.  These are just methods of 
> communication, nothing more.

Jeff, why don't you listen from time to time to people who work with
the technology on a daily basis who have experience with it, who
have _insight_ of the technology?  Such insight gives them great
intuition when it comes to design, among other things.

> In your example -- userspace discovery required before root filesystem 
> can be found -- a program running from initrd/initramfs would create an 
> SMP device node, open it, and then proceed with the discovery and 

SMP is part of the protocol, of what the device (PCI) implements.  It is always
there, just like phys.  You do not need to create it from user space. It
will be there for a user process to use, via say SDI.  SDI provides this as
part of the controller.  Read the SDI spec.

Insight of the whole architecture is irreplacable to create good design.

> configuration process, which in turn creates the device nodes necessary 
> to mount the root filesystem.

Also note that everyone does domain discovery in the kernel/FW and not
only for SAS but for other domains (even non-SCSI).  While domain
discovery is in the kernel/FW, _control_ of the domain is given to
user space, via say SDI -- everyone agrees on this.

   Luben

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/20/05 20:46, Jeff Garzik wrote:
> 
>>Consider what an ioctl is, overall:  a domain-specific "do this 
>>operation" interface.  Which, further, is nothing but a wrapping of a 
>>"send message" + "receive response" interface.  There are several ways 
>>to do this in Linux:
>>
>>* block driver.  a block driver is nothing but a message queue.  This is 
> 
> 
> Not quite.  This maybe the way it operates, but it is called "block"
> for a reason.

This illustrates you fundamentally don't understand a lot of Linux, and 
SCSI too.

Several non-blkdev device classes (Christoph listed them) use block 
layer request_queue for command transit, as does SG_IO and /dev/sg.


>>why James has suggested implementing SMP as a block driver.  People get 
>>stuck into thinking "block driver == block device", which is wrong.  The 
>>Linux block layer is nothing but a message queueing interface.
> 
> 
> Now, just because James suggested implementing the SMP service as a block
> device you think this is the right thing to do?

I very clearly said I don't know the best answer.  Perhaps you need to 
re-read the quoted email?


> How about this: Why not as a char device?

I covered that in the quoted email.


> At least MS isn't suffering from the "no to specs" syndrome which
> the Linux community seems to be suffering...

We have plenty of specs.  It's called source code.

You don't understand the Linux development process (think its more 
political than technical) and you don't understand even what a block 
driver is, and you wonder why you have difficulty getting code into the 
kernel?

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:18, Jeff Garzik wrote:
> This illustrates you fundamentally don't understand a lot of Linux, and 
> SCSI too.
> 
> Several non-blkdev device classes (Christoph listed them) use block 
> layer request_queue for command transit, as does SG_IO and /dev/sg.

When people start getting personal you know that they're losing it.

> We have plenty of specs.  It's called source code.
> 
> You don't understand the Linux development process (think its more 
> political than technical) and you don't understand even what a block 
> driver is, and you wonder why you have difficulty getting code into the 
> kernel?

Again, when people start getting personal, you know that they are losing it.

Thank you for spreading FUD -- I'm sure you've impressed your managament,
how great of a Linux programmer you are and how I don't know anything.
I'd suggest you keep pushing the politics _behind_ the scences.

Have a good day,
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 14:18, Jeff Garzik wrote:
>>We have plenty of specs.  It's called source code.
>>
>>You don't understand the Linux development process (think its more 
>>political than technical) and you don't understand even what a block 
>>driver is, and you wonder why you have difficulty getting code into the 
>>kernel?
> 
> 
> Again, when people start getting personal, you know that they are losing it.
> 
> Thank you for spreading FUD -- I'm sure you've impressed your managament,
> how great of a Linux programmer you are and how I don't know anything.
> I'd suggest you keep pushing the politics _behind_ the scences.


I'm trying to tell it like it is, in the hopes that you will eventually 
learn the process, and be a good upstream maintainer we can all work with.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:54, Jeff Garzik wrote:
>>Thank you for spreading FUD -- I'm sure you've impressed your managament,
>>how great of a Linux programmer you are and how I don't know anything.
>>I'd suggest you keep pushing the politics _behind_ the scences.
> 
> I'm trying to tell it like it is, in the hopes that you will eventually 
> learn the process, and be a good upstream maintainer we can all work with.

You're again spreading FUD and playing this political game hoping
your management is reading this and merely because of this they'll award you.
I cannot blame you -- how many of them read C code or specs...  I mean, even
"the community" has said "no" to specs.  All you're doing is just playing
your political game, spreading FUD.

And what "process" are you talking about?  Do you have any idea the
_amazement_ people get when learing that Linux is going to implement
showing all phys on the domain in sysfs...?  Do you have any idea
the effort in keeping customers happy when the whims of "the community"
change on an a hourly basis (as the community is learning)?  Shall I give
examples?

As to your reply above:
Jeff, not so according to people I work with on a daily basis from all
over the world.

When you said in your email:

On Oct 21, 2005, Jeff Garzik <jgarzik@pobox> wrote:
>
> We have plenty of specs.  It's called source code.

That pretty much seals it.

Have a nice day,
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 14:54, Jeff Garzik wrote:
> 
>>>Thank you for spreading FUD -- I'm sure you've impressed your managament,
>>>how great of a Linux programmer you are and how I don't know anything.
>>>I'd suggest you keep pushing the politics _behind_ the scences.
>>
>>I'm trying to tell it like it is, in the hopes that you will eventually 
>>learn the process, and be a good upstream maintainer we can all work with.
> 
> 
> You're again spreading FUD and playing this political game hoping
> your management is reading this and merely because of this they'll award you.
> I cannot blame you -- how many of them read C code or specs...  I mean, even
> "the community" has said "no" to specs.  All you're doing is just playing
> your political game, spreading FUD.

My only response is from SNL:  hear me now and believe me later.

I can only repeat the truth, and eventually (hope) you'll get it.


> And what "process" are you talking about?  Do you have any idea the
> _amazement_ people get when learing that Linux is going to implement
> showing all phys on the domain in sysfs...?  Do you have any idea
> the effort in keeping customers happy when the whims of "the community"
> change on an a hourly basis (as the community is learning)?  Shall I give
> examples?
> 
> As to your reply above:
> Jeff, not so according to people I work with on a daily basis from all
> over the world.
> 
> When you said in your email:
> 
> On Oct 21, 2005, Jeff Garzik <jgarzik@pobox> wrote:
> 
>>We have plenty of specs.  It's called source code.
> 
> 
> That pretty much seals it.

Shockingly, through all this stuff that annoys you, Linux is a wild and 
unqualified success.  ;-)

Linux is organic.  Its not cleanly laid out.  Its not specified in some 
document.  Linux is result of evolution as "nature" as dictated.  This 
drives some people nuts, but at the end of the day, it _works_.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Jeff Garzik wrote:
> Linux is result of evolution as "nature" as dictated.  This 
> drives some people nuts, but at the end of the day, it _works_.

The last two words sound a bit general.
-- 
Stefan Richter
-=====-=-=-= =-=- =-=-=
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 14:54, Jeff Garzik wrote:
>
> I'm trying to tell it like it is, in the hopes that you will eventually 
> learn the process, and be a good upstream maintainer we can all work with.

Look how you're using "all", generalizing left and right.  You just want to
create this FUD and spread this FUD that I don't work well with anyone.  Yep,
this is a pretty low blow.  While in fact did you talk to everyone I work with?

I say, when people are losing it on the technical front, they try to attack
personally and on political basis.

Have a nice day,
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 14:54, Jeff Garzik wrote:
> 
>>I'm trying to tell it like it is, in the hopes that you will eventually 
>>learn the process, and be a good upstream maintainer we can all work with.
> 
> 
> Look how you're using "all", generalizing left and right.  You just want to
> create this FUD and spread this FUD that I don't work well with anyone.  Yep,
> this is a pretty low blow.  While in fact did you talk to everyone I work with?
> 
> I say, when people are losing it on the technical front, they try to attack
> personally and on political basis.


The technical stuff got covered long ago.  Here are the basic basics:

* aic94xx needs to have the scsi-host-template in the LLDD, to fix 
improper layering.
* SAS generic code needs to use SAS transport class, which calls 
scsi_scan_target(), to avoid code duplication.
* other stuff I listed in my "analysis" email, including updating libata 
to support SAS+SATA hardware.

This is the stuff that I have been working on (nothing pushed to sas-2.6 
yet, as it doesn't yet boot locally).

If you were willing to do this stuff, _working with others_, then I 
would be off in happy happy SATA land right now, and you would have been 
nominated to be the Linux SAS maintainer.

Call it FUD, politics, personal attacks, wanking off to please 
manglement, whatever.  My goal has always been to (a) help Linux users 
by getting aic94xx+SAS upstream, and (b) try to help you understand why 
your code didn't go upstream verbatim, long after others have given up 
trying to do that.

	Jeff, he of infinite patience

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:39, Jeff Garzik wrote:
> The technical stuff got covered long ago.  Here are the basic basics:
> 
> * aic94xx needs to have the scsi-host-template in the LLDD, to fix 
> improper layering.
> * SAS generic code needs to use SAS transport class, which calls 
> scsi_scan_target(), to avoid code duplication.
> * other stuff I listed in my "analysis" email, including updating libata 
> to support SAS+SATA hardware.

All this bastardizes the code and the layering infrastructure.

If you're saying that there is "improper layering" you must be
smoking something really strong.

I'd suggest you talk to storage engineers with years of storage
experience about the design of the Linux SAS Stack.
I'd suggest you take a look at the USB and SBP code.
I'd suggest you take an overview of all of SAM and see how the protocols
fit together.
I'd suggest you take a look at the hardware design and see how and _where_
it fits and where it does not and _why_.
I'd suggest you take a look at the event management interface
(drivers/scsi/sas/sas_event.c) what it is, how it provides _all_
communication from LLDD to SAS Layer, filled in by the SAS Stack.
I'd suggest you take a look at the Execute Command SCSI RPC and the TMFs,
declared in the struct sas_ha_struct, filled in by the LLDD,
how they provide all communication from the SAS Stack to the LLDD (inteconnect).

I'm even ready to do a presentation in front of people explaining
the design of the Linux SAS Stack, why and how it works.  Specify
time and venue, please.

(I will put something up on the web site -- html and figures
to show the layering, function stubs, event management, etc.)

> This is the stuff that I have been working on (nothing pushed to sas-2.6 
> yet, as it doesn't yet boot locally).

I'm sorry to hear that.  Maybe you shouldn't have to bastardize the code.
Implement SDI and then everyone will be happy.  Look, I've alredy twice
posted code and templates, just grab it and put your name there.

Think about it: there is no reason to mess with the code.  It is
present at the link below, it works and people are using it.  It is also
sound and spec complient.  There are other subsystems which use the exact
same layering (namely USB Storage and SBP) which is consistent with the
SCSI Architecture Model all new protocols abide by.

> If you were willing to do this stuff, _working with others_, then I 
> would be off in happy happy SATA land right now, and you would have been 
> nominated to be the Linux SAS maintainer.

You seem to be traveling a path which I've already traveled.

Jeff, if you want an Adaptec SAS driver which has the host template
_in_ the driver, you can use "adp94xx" submitted earlier this year
by Adaptec to "the community".

"The community" rejected it. Why?  Because "common SAS tasks should
be in a common layer".  Well there you have the Linux SAS Stack and
aic94xx at the link below (my sig), which does separate common SAS
tasks and the interconnect _as per architecture and spec_.  Apparently
this still isn't good enough for "the community".

You see how this going around in circles just never seems to end.

One day one thing, another day the opposite, etc.

> Call it FUD, politics, personal attacks, wanking off to please 
> manglement, whatever.  My goal has always been to (a) help Linux users 
> by getting aic94xx+SAS upstream, and (b) try to help you understand why 
> your code didn't go upstream verbatim, long after others have given up 
> trying to do that.

(a) There are Linux users using "aic94xx+SAS" right now.  They've
downloaded a working Linux code from the link at my signature.

(b) I think it was because Linux says "no to specs" and you say that
"specs are the source code".

> 	Jeff, he of infinite patience

Everyone is very pleased with you.  You will go places.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 15:39, Jeff Garzik wrote:
> 
>>The technical stuff got covered long ago.  Here are the basic basics:
>>
>>* aic94xx needs to have the scsi-host-template in the LLDD, to fix 
>>improper layering.
>>* SAS generic code needs to use SAS transport class, which calls 
>>scsi_scan_target(), to avoid code duplication.
>>* other stuff I listed in my "analysis" email, including updating libata 
>>to support SAS+SATA hardware.
> 
> 
> All this bastardizes the code and the layering infrastructure.
> 
> If you're saying that there is "improper layering" you must be
> smoking something really strong.

I already described why.  Examples are DMA boundary and s/g limit, among 
others.  When confronted with this, you proposed an additional hardware 
information struct which duplicates Scsi_Host_Template.

Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
driver is implemented.  The host template is in the low level driver, 
while most of the code is common code, implemented elsewhere.



>>If you were willing to do this stuff, _working with others_, then I 
>>would be off in happy happy SATA land right now, and you would have been 
>>nominated to be the Linux SAS maintainer.
> 
> 
> You seem to be traveling a path which I've already traveled.
> 
> Jeff, if you want an Adaptec SAS driver which has the host template
> _in_ the driver, you can use "adp94xx" submitted earlier this year
> by Adaptec to "the community".
> 
> "The community" rejected it. Why?  Because "common SAS tasks should
> be in a common layer".  Well there you have the Linux SAS Stack and
> aic94xx at the link below (my sig), which does separate common SAS
> tasks and the interconnect _as per architecture and spec_.  Apparently
> this still isn't good enough for "the community".
> 
> You see how this going around in circles just never seems to end.
> 
> One day one thing, another day the opposite, etc.

Your current code is much closer to the desired end result, as it 
separates out SAS common code.  That's why its being used as a base.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 17:12, Jeff Garzik wrote:
> 
> I already described why.  Examples are DMA boundary and s/g limit, among 
> others.  When confronted with this, you proposed an additional hardware 
> information struct which duplicates Scsi_Host_Template.

I told you -- I have this in the struct asd_ha_struct and it was merely
a downplay that I didn't include the same thing in struct sas_ha_struct.

> Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 

No, this is the solution which would turn everything upside down.
The easiest and smallest solution is to just include this tiny struct
and end this.  It would have 0 impact on code.  In fact I'll
implement it now and push it to the git tree. ;-)

The host template _mixes_ hw, scsi core, and protocol knowlege into
one ugly blob.  I've given this argument before, several times.

> driver is implemented.  The host template is in the low level driver, 
> while most of the code is common code, implemented elsewhere.

libata isn't without architectural problems.  What strikes me is
that you think that libata-scsi is SATL.

You are so much better off renaming it to satl.c and given
the knowlege you've gained over the years and the backing you have
from engineers from companies, start with it at device level.  I, as
I'm sure other (not to name names) will be more than happy to contribute
if you started this.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 17:12, Jeff Garzik wrote:
> 
>>I already described why.  Examples are DMA boundary and s/g limit, among 
>>others.  When confronted with this, you proposed an additional hardware 
>>information struct which duplicates Scsi_Host_Template.
> 
> 
> I told you -- I have this in the struct asd_ha_struct and it was merely
> a downplay that I didn't include the same thing in struct sas_ha_struct.
> 
> 
>>Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
> 
> 
> No, this is the solution which would turn everything upside down.
> The easiest and smallest solution is to just include this tiny struct
> and end this.  It would have 0 impact on code.  In fact I'll
> implement it now and push it to the git tree. ;-)
> 
> The host template _mixes_ hw, scsi core, and protocol knowlege into
> one ugly blob.

True.

If you do not like the current situation, evolve the SCSI core (and all 
drivers) to where you think they should be.

The correct answer is NOT to duplicate information between 
Scsi_Host_Template and Lubens_Hardware_Struct.


>>driver is implemented.  The host template is in the low level driver, 
>>while most of the code is common code, implemented elsewhere.
> 
> 
> libata isn't without architectural problems.  What strikes me is
> that you think that libata-scsi is SATL.

The only things that matter are (a) what the code is now, and (b) what 
changes are needed to get where we need to be.

Thus, regardless of whether or not libata-scsi meets the needs of 
SAS+SATA hardware, libata-scsi is where all SCSI<->ATA translation 
should occur.  If you are dissatisfied, evolve the code to where it 
needs to be.


> You are so much better off renaming it to satl.c and given

Naming is completely irrelevant.  Just modify the code.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 17:41, Jeff Garzik wrote:
>>>I already described why.  Examples are DMA boundary and s/g limit, among 
>>>others.  When confronted with this, you proposed an additional hardware 
>>>information struct which duplicates Scsi_Host_Template.
>>
>>
>>I told you -- I have this in the struct asd_ha_struct and it was merely
>>a downplay that I didn't include the same thing in struct sas_ha_struct.

Here is the commit in question:
http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a

>>>Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
>>
>>
>>No, this is the solution which would turn everything upside down.
>>The easiest and smallest solution is to just include this tiny struct
>>and end this.  It would have 0 impact on code.  In fact I'll
>>implement it now and push it to the git tree. ;-)
>>
>>The host template _mixes_ hw, scsi core, and protocol knowlege into
>>one ugly blob.
> 
> 
> True.
> 
> If you do not like the current situation, evolve the SCSI core (and all 
> drivers) to where you think they should be.

While the architecture in my mind is clear, I cannot do this myself
(and for all drivers).  Such a change would be gradual, involving more
than one developer, for more than one (new) driver, etc.

First we need SDI to make everyone happy.  This way HP/LSI/Adaptec
can move quickest to the customers with minimal pain to the customers
and to the companies which would have to change software (minimal change).

> Thus, regardless of whether or not libata-scsi meets the needs of 
> SAS+SATA hardware, libata-scsi is where all SCSI<->ATA translation 
> should occur.  If you are dissatisfied, evolve the code to where it 
> needs to be.

Ok.

> Naming is completely irrelevant.  Just modify the code.

Ok.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> On 10/21/05 17:41, Jeff Garzik wrote:
> 
>>>>I already described why.  Examples are DMA boundary and s/g limit, among 
>>>>others.  When confronted with this, you proposed an additional hardware 
>>>>information struct which duplicates Scsi_Host_Template.
>>>
>>>
>>>I told you -- I have this in the struct asd_ha_struct and it was merely
>>>a downplay that I didn't include the same thing in struct sas_ha_struct.
> 
> 
> Here is the commit in question:
> http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a

This effectively illustrates the wrong thing to do:  duplicating 
information that's already in Scsi_Host_Template.

Just use Scsi_Host_Template in the LLDD and see where that goes.


>>>>Solution?  Just use Scsi_Host_Template.  Take a look at how each libata 
>>>
>>>
>>>No, this is the solution which would turn everything upside down.
>>>The easiest and smallest solution is to just include this tiny struct
>>>and end this.  It would have 0 impact on code.  In fact I'll
>>>implement it now and push it to the git tree. ;-)
>>>
>>>The host template _mixes_ hw, scsi core, and protocol knowlege into
>>>one ugly blob.
>>
>>
>>True.
>>
>>If you do not like the current situation, evolve the SCSI core (and all 
>>drivers) to where you think they should be.
> 
> 
> While the architecture in my mind is clear, I cannot do this myself
> (and for all drivers).  Such a change would be gradual, involving more
> than one developer, for more than one (new) driver, etc.

Correct.  That's why there is resistance to aic94xx's approach of 
creating a totally new "strict SAM" path, existing in parallel with the 
traditional SCSI core.  You need to evolve the existing code to get 
there.  Such changes are gradual, involving more than one developer, etc.

We don't need one small set of SCSI drivers behaving differently from 
the vast majority of existing SCSI drivers.

Hear me now, and believe me later:  we all largely agree on the points 
you've raised about legacy crapola in the SCSI core.  James, Christoph, 
myself, and several others disagree with your assertion that the old 
SCSI core should exist in parallel with your new SCSI core.

We differ on the path, not the goal.  As a thought experiment, you could 
try simply implementing the changes requested, and see where that goes.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Jeff Garzik wrote:
> Luben Tuikov wrote:
>>>>> Examples are DMA boundary and s/g limit, 
>>>>> among others.  When confronted with this, you proposed an 
>>>>> additional hardware information struct which duplicates 
>>>>> Scsi_Host_Template.
>>>>
>>>> I told you -- I have this in the struct asd_ha_struct and it was merely
>>>> a downplay that I didn't include the same thing in struct 
>>>> sas_ha_struct.
>>
>> Here is the commit in question:
>> http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a 
> 
> This effectively illustrates the wrong thing to do:  duplicating 
> information that's already in Scsi_Host_Template.
> 
> Just use Scsi_Host_Template in the LLDD and see where that goes.

Will cmd_per_lun, sg_tablesize, max_sectors, dma_boundary, 
use_clustering ever have to be adjusted specifically for a SAS hardware?

Obviuosly none of this is required _at the moment_. IOW neither the 
introduction of a sas_ha_hw_profile nor a pass-through of 
scsi_host_template down to SAS interconnect drivers is required right 
now. So why do one or the other now? Isn't it a sensible rule to not 
solve problems now which do not exist yet?

(I guess Luben only introduced sas_ha_hw_profile to demonstrate that 
there will never be an absolute requirement for scsi_host_template --- 
in its present form --- to be visible in a SAS transport layer <-> SAS 
interconnect driver interface. And there are certainly more alternatives 
to these two approaches.)
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:
> 
> Will cmd_per_lun, sg_tablesize, max_sectors, dma_boundary, 
> use_clustering ever have to be adjusted specifically for a SAS hardware?

No hardware SAS chip I know of needs any of those legacy limitations.
Neither BCM8603 nor Fusion MPT.

Those limitations are purely Parallel SCSI.

I just included it, to show proof of concept -- when the architecure and
layering is correct, how easy it is to do it.  But you're right, it is
not needed.

> Obviuosly none of this is required _at the moment_. IOW neither the 
> introduction of a sas_ha_hw_profile nor a pass-through of 
> scsi_host_template down to SAS interconnect drivers is required right 
> now. So why do one or the other now? Isn't it a sensible rule to not 
> solve problems now which do not exist yet?

This is exactly the rule I followed when developing the SAS Transport
Layer for Linux.  Furthermore, _that_ rule, to never overengineer, I learned
from Linux.  Sadly the politics are too deep and that rule applies only
to what is convenient, at least in Linux SCSI.

   Luben


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Jeff Garzik wrote:
> Luben Tuikov wrote:
>>>> The host template _mixes_ hw, scsi core, and protocol knowlege into
>>>> one ugly blob.
>>>
>>> True.
>>>
>>> If you do not like the current situation, evolve the SCSI core (and 
>>> all drivers) to where you think they should be.
>>
>> While the architecture in my mind is clear, I cannot do this myself
>> (and for all drivers).  Such a change would be gradual, involving more
>> than one developer, for more than one (new) driver, etc.
> 
> Correct.  That's why there is resistance to aic94xx's approach of 
> creating a totally new "strict SAM" path, existing in parallel with the 
> traditional SCSI core.  You need to evolve the existing code to get 
> there.  Such changes are gradual, involving more than one developer, etc.
> 
> We don't need one small set of SCSI drivers behaving differently from 
> the vast majority of existing SCSI drivers.
> 
> Hear me now, and believe me later:  we all largely agree on the points 
> you've raised about legacy crapola in the SCSI core.  James, Christoph, 
> myself, and several others disagree with your assertion that the old 
> SCSI core should exist in parallel with your new SCSI core.
> 
> We differ on the path, not the goal.  As a thought experiment, you could 
> try simply implementing the changes requested, and see where that goes.

I am not familiar with most parts of the SCSI subsystem. However from 
what I understood, some existing concepts in the core need to be removed 
from SCSI core entirely, others need to be cleaned up WRT what they mean 
and how they represent it. It seems to me that a practical path to go 
would be:

A. Post mock-ups and pseudo code about how to change the core, discuss.
B. Set up a scsi-cleanup tree. In this tree,
      1. renovate the core (thereby break all command set drivers and
         all transport subsystems),
      2. update ~2 command set drivers and ~2 transport subsystems
      3. validate the renovated core,
      4. fix the conceptual errors of the renovated core (as well as
         first few discovered bugs in the implementation),
      5. update all other command set drivers,
      6. update all transport subsystems where resources to do so are
         available,
      7. test all command set drivers as far as hardware is accessible,
      8. test the updated transport subsystems as far as hardware is
         accessible,
      9. fix prominent bugs.
C. In mainline,
      i. mark all drivers which cannot be updated in the mid term as
         scheduled for temporary feature removal (i.e. they will be
         broken for an undetermined period),
     ii. mark all drivers which have been updated in scsi-cleanup tree,
         but were not thoroughly tested, as scheduled for temporary
         feature regression (i.e. they will be experimental for an
         undetermined period).
D. Push scsi-cleanup tree to -mm and shake out bugs.
E. Push to mainline.
F. Fix remaining drivers as time goes by, remove drivers which remain
    broken for too long.

Most steps may overlap, some steps may repeat. Step A is fortunately 
already going on for quite some time.

Step 1.-4. could involve much dispute, thus taking much more time than 
technically necessary -or- (and that would be less fortunate) being 
dragged out into later stages when a conceptually stabilized core is 
desirable. Much of that may be prevented in step A.

I doubt that the desired cleanup of the SCSI core could be done 
on-the-go, i.e. without temporary breakage of larger parts of the 
subsystem (out of mainline). But then again, I don't know much of the 
subsystem, so what am I talking about here?

Also, long-term breakage of smaller parts of the SCSI subsystem in 
mainline is to be expected; breakage which is to be announced and scheduled.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Christoph Hellwig]

On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
> A. Post mock-ups and pseudo code about how to change the core, discuss.
> B. Set up a scsi-cleanup tree. In this tree,
>      1. renovate the core (thereby break all command set drivers and
>         all transport subsystems),

No way.  Doing things from scatch is a really bad idea.  See how far we came
with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break the
world.  Please submit changes to fix _one_ thing at a time and fix all users.
Repeat until done or you don't care anymore.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Sergey Panov]

On Sat, 2005-10-22 at 11:58 +0100, Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
> > A. Post mock-ups and pseudo code about how to change the core, discuss.
> > B. Set up a scsi-cleanup tree. In this tree,
> >      1. renovate the core (thereby break all command set drivers and
> >         all transport subsystems),
> 
> No way.  Doing things from scatch is a really bad idea.  See how far we came
> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break the
> world.  Please submit changes to fix _one_ thing at a time and fix all users.
> Repeat until done or you don't care anymore.

 It is a mistake to think that you can not do a big rework and keep SCSI
sub-system stable. You just have to make sure the OLD way is supported
for as log as it is needed.

 E.g. before moving in the RIGHT direction and weeding out parallel SCSI
atavism from the common SCSI layer (or should I say SAM, to please
Luben) you can insert "generic" transport layer at the bottom (just like
Luben inserted his SAS) and use it to keep all currently supported
drivers stable. SCSI mid-layer can be reworked in any number of steps,
even in one step, because is not a big thing, it is just 11k lines plus
4k of recently added transport modules.

> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Christoph Hellwig]

On Sat, Oct 22, 2005 at 11:28:30AM -0400, Sergey Panov wrote:
>  It is a mistake to think that you can not do a big rework and keep SCSI
> sub-system stable. You just have to make sure the OLD way is supported
> for as log as it is needed.

No.  Rewriting something from scratch is horrible engineering practice.
It's impossible to very huge changes, small incremental changes OTOH
allow easier planning, easier calculation of the risks and cost and most
import better test coverage.  There's nothing specific to scsi or linux
kernel code about it.  It'd suggest you read:

 http://www.joelonsoftware.com/articles/fog0000000069.html

or various similar articles.  Full scale rewrites almost never work
out.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Sergey Panov]

On Sat, 2005-10-22 at 18:19 +0100, Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 11:28:30AM -0400, Sergey Panov wrote:
> >  It is a mistake to think that you can not do a big rework and keep SCSI
> > sub-system stable. You just have to make sure the OLD way is supported
> > for as log as it is needed.
> 
> No.  Rewriting something from scratch is horrible engineering practice.

Most of the time. Besides "rework" is not necessarily "rewrite from
scratch", most of the time it means "modification" of the existing
system.

> It's impossible to very huge changes, small incremental changes OTOH
> allow easier planning, easier calculation of the risks and cost and most
> import better test coverage.  There's nothing specific to scsi or linux
> kernel code about it.  It'd suggest you read:
> 
>  http://www.joelonsoftware.com/articles/fog0000000069.html

Bad example -- just count number of lines in drivers/scsi/scsi*.c
and in Netscape 4.0 and you'll see why.
That does not mean I advocate throughing out current SCSI mid layer and
writing a new one. As I can tell, no one on that list is proposing the
"rewrite from scratch" approach.
  I just was trying to point out that Luben's transport "layers" in
place of transport "modules-appendages" simplifies that
migration/evolution.   

> or various similar articles.  Full scale rewrites almost never work
> out.


Sergey Panov

===========================================

I expressed my personal opinion and I am not speaking for anyone else.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/22/05 13:38, Sergey Panov wrote:
>   I just was trying to point out that Luben's transport "layers" in
> place of transport "modules-appendages" simplifies that
> migration/evolution.

True. "modules-appendages" just makes everything messy.  There is
so much SPI in the host template... Take for example recovery,
or as Linux SCSI calls it Error Handling (EH).

EH in Linux SCSI is SPI.  This is a fact, and anyone can look at the
code to convince themselves.

In order to properly get rid ourselves of HCIL, recovery should
be offloaded to the transport _layer_ which communicates with
the transport/interconnect to get things going, as shown in the SAS
Transport Stack at the link in my signature, USB* or SBP subsystems.

* I've been asking Alan to properly implement USB recovery
in the USB Storage subsystem...  Maybe one day...

That is, the layer above knows nothing of the layer below, but
each layer provides well defined functionality, and they all
work in tandem.  This is exactly what allows for quick growth
and rich future (of a design).

In such a design, SCSI Core would be very small and the task
paths would be very straightforward as shown in the SAS Transport
Stack in the link in my signature, USB or SBP subsystems.

BTW, "modules-appendages" very truly describes the current state
of Linux SCSI and this can be seen from, for example the scsi host
structure, where those "modules-appendages" are "hooked".

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Alan Cox]

On Sad, 2005-10-22 at 18:19 +0100, Christoph Hellwig wrote:
> No.  Rewriting something from scratch is horrible engineering practice.

There I must disagree strongly. There are times when it is the right
thing to do because the original is unsalvagable (see drivers/ide). 

OTOH I do not believe the scsi layer is in this state. 


Luben, it isn't about layering. Linux does layering and does it strongly
but it does not do unneccessary layering. If the layer above is wrong
then adapt it step by step without breaking existing stuff (or making
sure the existing changes to drivers are clean and testable).

That process works. There is very little left of the original "Linux"
and most of what is left is the stuff that most needs maintenance (eg
floppy.c, tty layer). In time they too will change.

Don't go around the scsi layer for something generic, go through it.
Going around things or adding ugly hacks is fine (and justified) for a
specific single piece of hardware that is somehow brain damaged, its not
the right approach for something likely to be mainstream and generic.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/22/05 13:19, Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 11:28:30AM -0400, Sergey Panov wrote:
> 
>> It is a mistake to think that you can not do a big rework and keep SCSI
>>sub-system stable. You just have to make sure the OLD way is supported
>>for as log as it is needed.
> 
> 
> No.  Rewriting something from scratch is horrible engineering practice.

Off Topic:
	Discourse on Christoph's "Rewriting something from scratch
		is horrible engineering practice"

Christoph,

Imagine a for ( ) { } loop spanning 5000 lines.  Imagine never using
functions to separate things.  But imagine that a _factory_ is using
this code on its production line and that code, although badly written,
does work and keeps the production line going and thousands of
people working.

They hire an engineer to make it _managable_ and supportable -- this
warrants understanding the production line, what the code does, what it
controls and how.  Understanding how the factory workers use it and what
they expect.  Understanding the code (which may not be as easy).  Then it
is rewritten so that it can be easily supported and maintained.

This is real life example.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Alan Cox]

On Llu, 2005-10-24 at 09:51 -0400, Luben Tuikov wrote:
> controls and how.  Understanding how the factory workers use it and what
> they expect.  Understanding the code (which may not be as easy).  Then it
> is rewritten so that it can be easily supported and maintained.

Very very rarely, because it means down time and supporting two systems
at once. Take a look at the australian customs fiasco or the british
passport office disaster to see why (actually almost any large
government IT project where politics dictated 'write new stuff so I can
announce it in parliament').

The smart factory update would occur piece by piece. Starting with the
most pressing problems (ie fastest ROI) and working to a plan that ends
up with the system modular and clean.

You don't turn a steel plant off for a software upgrade.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/24/05 11:41, Alan Cox wrote:
> On Llu, 2005-10-24 at 09:51 -0400, Luben Tuikov wrote:
> 
>>controls and how.  Understanding how the factory workers use it and what
>>they expect.  Understanding the code (which may not be as easy).  Then it
>>is rewritten so that it can be easily supported and maintained.
> 
> 
> Very very rarely, because it means down time and supporting two systems
> at once. Take a look at the australian customs fiasco or the british
> passport office disaster to see why (actually almost any large
> government IT project where politics dictated 'write new stuff so I can
> announce it in parliament').
> 
> The smart factory update would occur piece by piece. Starting with the
> most pressing problems (ie fastest ROI) and working to a plan that ends
> up with the system modular and clean.
> 
> You don't turn a steel plant off for a software upgrade.

There was 0 (zero) effective downtime to the factory.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Regala]

Luben Tuikov <luben_tuikov@adaptec.com> disait dernièrement que :

> On 10/24/05 11:41, Alan Cox wrote:
>> On Llu, 2005-10-24 at 09:51 -0400, Luben Tuikov wrote:
>> 
>>>controls and how.  Understanding how the factory workers use it and what
>>>they expect.  Understanding the code (which may not be as easy).  Then it
>>>is rewritten so that it can be easily supported and maintained.
>> 
>> 
>> Very very rarely, because it means down time and supporting two systems
>> at once. Take a look at the australian customs fiasco or the british
>> passport office disaster to see why (actually almost any large
>> government IT project where politics dictated 'write new stuff so I can
>> announce it in parliament').
>> 
>> The smart factory update would occur piece by piece. Starting with the
>> most pressing problems (ie fastest ROI) and working to a plan that ends
>> up with the system modular and clean.
>> 
>> You don't turn a steel plant off for a software upgrade.
>
> There was 0 (zero) effective downtime to the factory.

but refactoring can be done in incremental pieces, can't be ?
rewriting it from scratch is, in this very case, really for the sake
of self-pride and brain-masturbation.
Bravo

This is not a really convincing example...

-- 
Mathieu Segaud
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/24/05 11:59, Regala wrote:
> but refactoring can be done in incremental pieces, can't be ?
> rewriting it from scratch is, in this very case, really for the sake
> of self-pride and brain-masturbation.
> Bravo
> 
> This is not a really convincing example...
> 
> -- 
> Mathieu Segaud

They, the factory, wanted it rewritten from scratch,
so that they can maintain and support it themselves.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Regala]

Luben Tuikov <luben_tuikov@adaptec.com> disait dernièrement que :

> On 10/24/05 11:41, Alan Cox wrote:
>> On Llu, 2005-10-24 at 09:51 -0400, Luben Tuikov wrote:
>> 
>>>controls and how.  Understanding how the factory workers use it and what
>>>they expect.  Understanding the code (which may not be as easy).  Then it
>>>is rewritten so that it can be easily supported and maintained.
>> 
>> 
>> Very very rarely, because it means down time and supporting two systems
>> at once. Take a look at the australian customs fiasco or the british
>> passport office disaster to see why (actually almost any large
>> government IT project where politics dictated 'write new stuff so I can
>> announce it in parliament').
>> 
>> The smart factory update would occur piece by piece. Starting with the
>> most pressing problems (ie fastest ROI) and working to a plan that ends
>> up with the system modular and clean.
>> 
>> You don't turn a steel plant off for a software upgrade.
>
> There was 0 (zero) effective downtime to the factory.

but refactoring can be done in incremental pieces, can't be ?
rewriting it from scratch is, in this very case, really for the sake
of self-pride and brain-masturbation.
Bravo

This is not a really convincing example...

-- 
Mathieu Segaud

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Christoph Hellwig <hch@infradead.org> wrote:
> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
> > A. Post mock-ups and pseudo code about how to change the core, discuss.
> > B. Set up a scsi-cleanup tree. In this tree,
> >      1. renovate the core (thereby break all command set drivers and
> >         all transport subsystems),
> 
> No way.  Doing things from scatch is a really bad idea.  See how far we came
> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break
> the
> world.  Please submit changes to fix _one_ thing at a time and fix all users.
> Repeat until done or you don't care anymore.

No offence Christoph, but who are you again?

There is a clear reason why you among others do not want new architecture.
And that reason is (people) obsoletion.

Such political stance cannot go on forever -- just look at History.
Sooner or later things change and they change radically.  The question
is How prepared are you/we to cope with this (inevitable) change?

Either way, obsoletion or adoption -- think about it, it doesn't only apply
to computer and OS design, it applies to everything.

    Luben




-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Jeff Garzik]

Luben Tuikov wrote:
> No offence Christoph, but who are you again?
> 
> There is a clear reason why you among others do not want new architecture.
> And that reason is (people) obsoletion.
> 
> Such political stance cannot go on forever -- just look at History.
> Sooner or later things change and they change radically.  The question
> is How prepared are you/we to cope with this (inevitable) change?


THIS HAS FUCK-ALL TO DO WITH POLITICS.

Its about working with the SCSI layer, not around it.

	Jeff

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
>>A. Post mock-ups and pseudo code about how to change the core, discuss.
>>B. Set up a scsi-cleanup tree. In this tree,
>>     1. renovate the core (thereby break all command set drivers and
>>        all transport subsystems),
> 
> No way.  Doing things from scatch is a really bad idea.  See how far we came
> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break the
> world.  Please submit changes to fix _one_ thing at a time and fix all users.
> Repeat until done or you don't care anymore.

I agree with you. Alas my wording was misunderstandable und obviously 
carried a wrong tone.

I did not say "replace the core" in step 1. Also, the breakage which I 
refer to in step 1 would have to be immediately corrected in step 2 
(although not for the whole subsystem at once, to allow for a fast cycle 
of validation of what happened in step 1). Furthermore I specifically 
said that most steps may (let me add: and should) overlap.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[David Lang]

On Sat, 22 Oct 2005, Stefan Richter wrote:

> Christoph Hellwig wrote:
>> On Sat, Oct 22, 2005 at 12:42:27PM +0200, Stefan Richter wrote:
>>> A. Post mock-ups and pseudo code about how to change the core, discuss.
>>> B. Set up a scsi-cleanup tree. In this tree,
>>>     1. renovate the core (thereby break all command set drivers and
>>>        all transport subsystems),
>> 
>> No way.  Doing things from scatch is a really bad idea.  See how far we 
>> came
>> with Linux 2.6 scsi vs 2.4 scsi without throwing everything away and break 
>> the
>> world.  Please submit changes to fix _one_ thing at a time and fix all 
>> users.
>> Repeat until done or you don't care anymore.
>
> I agree with you. Alas my wording was misunderstandable und obviously carried 
> a wrong tone.
>
> I did not say "replace the core" in step 1. Also, the breakage which I refer 
> to in step 1 would have to be immediately corrected in step 2 (although not 
> for the whole subsystem at once, to allow for a fast cycle of validation of 
> what happened in step 1). Furthermore I specifically said that most steps may 
> (let me add: and should) overlap.

Stefan,
   we are supposed to be on a 2-month release cycle, with all major changes 
going in in the first two weeks of that cycle. This timeframe doesn't 
leave you any noticable time to implement your steps seperatly (and zero 
testing between them). as a result, in practice your proposal amounts to a 
big-bang approach, and/or results in releases that are known-broken.

and while you suggest putting this in -mm, remember that the -mm kernel 
needs to be useable so that people can test it, and it is on the same 
schedule as the main kernel so again you can't have known-broken things 
(of this scale) there either.

David Lang

-- 
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
  -- C.A.R. Hoare

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Stefan Richter]

David Lang wrote:
> Stefan,
>   we are supposed to be on a 2-month release cycle, with all major 
> changes going in in the first two weeks of that cycle. This timeframe 
> doesn't leave you any noticable time to implement your steps seperatly 
> (and zero testing between them). as a result, in practice your proposal 
> amounts to a big-bang approach, and/or results in releases that are 
> known-broken.

Experimental branches of subsystems usually cannot (nor need to) be
bound to a release cycle.

> and while you suggest putting this in -mm, remember that the -mm kernel 
> needs to be useable so that people can test it, and it is on the same 
> schedule as the main kernel so again you can't have known-broken things 
> (of this scale) there either.

I assumed there would be a few unmaintained bits (of marginal "scale")
left which could not be updated, but it seems now that won't be the
case.
-- 
Stefan Richter
-=====-=-=-= =-=- ==--=
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[David Lang]

On Sat, 22 Oct 2005, Stefan Richter wrote:

> Jeff Garzik wrote:
>> 
>> We differ on the path, not the goal.  As a thought experiment, you could 
>> try simply implementing the changes requested, and see where that goes.
>
> I doubt that the desired cleanup of the SCSI core could be done on-the-go, 
> i.e. without temporary breakage of larger parts of the subsystem (out of 
> mainline). But then again, I don't know much of the subsystem, so what am I 
> talking about here?
>
> Also, long-term breakage of smaller parts of the SCSI subsystem in mainline 
> is to be expected; breakage which is to be announced and scheduled.

Stefan, what you and Luben are missing is that big-bang changes like you 
are proposing are simply not acceptable anymore.

a few years ago when the 2.5 kernel series opened a similar big-bang 
approach was attempted for the IDE drivers. the instability that resulted 
(and rumors of the instability being worse then it was) eliminated a lot 
of people from testing things. things finally got bad enough that the 
entire system was reverted, and then a developer (one who had previously 
stated that such drastic changes were impossible) sat down and produced a 
long series of patches, each of which did a small amount of changes, each 
of which left the kernel in a working state (and each of which provided an 
advantage that could be identified at the time of the patch, either a 
better abstraction or a code cleanup). over a very few months (especially 
relative to the time spent working on the big-bang patches) the entire 
system was re-written.

This is what Jeff is trying to tell you. you can't just produce an 
entirely new SCSI subsystem and drop it into the kernel one day, you can 
all agree on a goal (this has almost been done, but nto quite), but that's 
only the first step. After you have some idea of the goal you then have to 
look at how to move to that goal without breaking things in the meantime. 
This requires that each step along the way keeps things working and is 
relativly straightforward in and of itself.

this definantly sounds a LOT harder then the 'throw it out and replace it 
all' approach, and it is (from the point of view of the programmer), 
however the result ends up being far more reliable as the process forces 
better examination of all the details, and allows more people to 
understand what's happening each step of the way. And since there are no 
releases that are made unuseable for people deliberatly, you also get 
extensive testing of all the steps along the way. This not only finds bugs 
sooner, it also makes it far more obvious where performance issues show up

so please accept that you aren't going to be able to replace any 
significant system in one massive change and instead start looking for 
ways to move the existing design towards where you want it to be and you 
will receive a lot of assistance in the process instead of banging heads 
with everyone.

David Lang

-- 
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
  -- C.A.R. Hoare

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Luben Tuikov]

--- David Lang <david.lang@digitalinsight.com> wrote:
> Stefan, what you and Luben are missing is that big-bang changes like you 
> are proposing are simply not acceptable anymore.

Take a look at the link below, working SAS code with the current
kernel.  No one is trying to produce big-bang changes.  This is what
Jeff want's you to believe that I'm trying to do -- this is part of his
political game and FUD and apparently he's succeeding.

I repeat: no on is trying to produce a new "big-bang changes" as you
call them. See the working code in the link at my sig.  This is
_current_ Linus's tree with SAS Stack in it.

Don't yield to the FUD.

> This is what Jeff is trying to tell you. you can't just produce an 
> entirely new SCSI subsystem and drop it into the kernel one day, you can 

No, this is what he wants you to believe that I'm trying to do.  This is what
people start doing when they loose on technological ground.  Do not
yield to the FUD and politics.  See the working code of current Linus's tree
with SAS in it in my sig link.

I repeat, no one is trying to replace anything or throw anything out.
This is just FUD and politics which Jeff is spreading because he's got
an agenda in convincing certain people certain things.

See the working SAS code in the link below -- current Linus' tree with
SAS in it.  Soon enough it would contain SDI back-end and then HP and LSI
would be able to plug right in, and I think there's an engineer out there
who's already implemented a front end.

    Luben


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Stefan Richter]

David Lang wrote:
> On Sat, 22 Oct 2005, Stefan Richter wrote:
>> I doubt that the desired cleanup of the SCSI core could be done 
>> on-the-go, i.e. without temporary breakage of larger parts of the 
>> subsystem (out of mainline). But then again, I don't know much of the 
>> subsystem, so what am I talking about here?
>>
>> Also, long-term breakage of smaller parts of the SCSI subsystem in 
>> mainline is to be expected; breakage which is to be announced and 
>> scheduled.
> 
> Stefan, what you and Luben are missing is that big-bang changes like you 
> are proposing are simply not acceptable anymore.

I agree with you that big-bang changes are not acceptable. This wasn't 
however what I had in mind.

> a few years ago when the 2.5 kernel series opened a similar big-bang 
> approach was attempted for the IDE drivers. the instability that 
> resulted (and rumors of the instability being worse then it was) 
> eliminated a lot of people from testing things. things finally got bad 
> enough that the entire system was reverted, and then a developer (one 
> who had previously stated that such drastic changes were impossible) sat 
> down and produced a long series of patches, each of which did a small 
> amount of changes, each of which left the kernel in a working state (and 
> each of which provided an advantage that could be identified at the time 
> of the patch, either a better abstraction or a code cleanup). over a 
> very few months (especially relative to the time spent working on the 
> big-bang patches) the entire system was re-written.

What I proposed was to "renovate" a _part_ of the SCSI subsystem (the 
core and the interfaces of the rest of the subsystem to the core) with 
the 2.6 kernel as a basis. This is a huge difference to 2.5. Changes 
took place everywhere in 2.5, many of them drastic.

I agree with you that the incremental approach is preferrable whenever 
possible. I even believe that this method could be applied to the SCSI 
core cleanup. However, concerns were voiced that this method would 
effectively lead to a double SCSI core: The old one, and a new one in 
parallel to it which doesn't share much code with the old one. (At least 
for some time, perhaps for a too long time.)

It has been said multiple times that this would not be desirable for 
reasons of /a/ more kernel bloat (while the goal was to remove existing 
bloat and lay foundations to avoid future bloat), and /b/ massive 
maintenance burden of two parallel infrastructures.

So that's why I said that *short-term* breakage right on top and right 
below the core should be accepted.

Again, the huge difference to the 2.5 times would be that all this would 
happen on top of a relatively stable kernel. (Stable in two senses.)

> This is what Jeff is trying to tell you. you can't just produce an 
> entirely new SCSI subsystem and drop it into the kernel one day, you can 

What I was referring to was to clean up a _part_ of the subsystem (the 
core), not to replace the subsystem. I admit though that my wording left 
much room for misunderstanding.

Furthermore, note that the "scsi-cleanup tree" which I referred to is 
not meant to be a fork. It should merely be another working stage before 
the -mm stage. And let me add that this stage should be left as soon as 
possible.

> all agree on a goal (this has almost been done, but nto quite), but 
> that's only the first step. After you have some idea of the goal you 
> then have to look at how to move to that goal without breaking things in 
> the meantime. This requires that each step along the way keeps things 
> working and is relativly straightforward in and of itself.
> 
> this definantly sounds a LOT harder then the 'throw it out and replace 
> it all' approach, and it is (from the point of view of the programmer), 
> however the result ends up being far more reliable as the process forces 
> better examination of all the details,

I agree with you on that. Again, although my post may have sounded like 
it, I did not want to advocate the "throw it out and replace it all" 
approach.

> and allows more people to 
> understand what's happening each step of the way.

Absolutely.

However I don't agree with you that _every_ little step must keep 
everything working. I believe that this may actually make the transition 
less easy to follow.

> And since there are no 
> releases that are made unuseable for people deliberatly, you also get 

I did not suggest to make unuseable releases.

> extensive testing of all the steps along the way. This not only finds 
> bugs sooner, it also makes it far more obvious where performance issues 
> show up

I agree with you on these advantages of the "(try to) keep everything 
working after each patch". Although the monitoring of performance is 
less important during the initial stage of the cleanup of the core.

> so please accept that you aren't going to be able to replace any 
> significant system in one massive change

Agreed. This is absolutely not what should be done.

> and instead start looking for 
> ways to move the existing design towards where you want it to be and you 
> will receive a lot of assistance in the process instead of banging heads 
> with everyone.

OK.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Christoph Hellwig]

On Sat, Oct 22, 2005 at 07:41:08PM +0200, Stefan Richter wrote:
> What I was referring to was to clean up a _part_ of the subsystem (the 
> core), not to replace the subsystem. I admit though that my wording left 
> much room for misunderstanding.
> 
> Furthermore, note that the "scsi-cleanup tree" which I referred to is 
> not meant to be a fork. It should merely be another working stage before 
> the -mm stage. And let me add that this stage should be left as soon as 
> possible.

gosh, could you please shut up and code now?

There's been various TODO items posted:

 (1) my TODO list for making the core HCIL idependent
 (2) finishing the transition to remove struct scsi_request and only
     send down S/G list
 (3) get rid of legacy host probing

and we'd all be happy if you added to the list.  Talking doesn't change
anything, please submit patches and help moving things forward.  Thanks.

And please start a new thread for such suggestions, this thread has finally
managed to get into my killfile.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Stefan Richter]

Christoph Hellwig wrote:
> On Sat, Oct 22, 2005 at 07:41:08PM +0200, Stefan Richter wrote:
[obviously something annoying]
> 
> gosh, could you please shut up and code now?

I am shutting up right now, am in the process of learning more about 
SCSI and Linux' implementation of it, and will submit patches as soon as 
I am able to. This will take much time though as I am new here and 
cannot devote as much time to it as I liked, as it is not my profession.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attachedPHYs)

[Sergey Panov]

On Sat, 2005-10-22 at 18:51 +0100, Christoph Hellwig wrote:
> There's been various TODO items posted:

I thought "been posted" warrants at least a reference to the post. I am,
for one, have not seen that TODO list.

Sergey

===================================================================
I am expressing my personal opinion and not speaking for anyone else.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Stefan Richter]

I wrote:
>      1. renovate the core (thereby break all command set drivers and
>         all transport subsystems),
>      2. update ~2 command set drivers and ~2 transport subsystems
>      3. validate the renovated core,
>      4. fix the conceptual errors of the renovated core (as well as
>         first few discovered bugs in the implementation),
>      5. update all other command set drivers,
>      6. update all transport subsystems where resources to do so are
>         available,

Step 6 probably involves the creation of a SPI transport layer. It 
implements former SPI related functionality of the core and may receive 
former code of the core. BTW, it may be a good idea to really defer this 
to step 6 instead of doing so in step 2.
-- 
Stefan Richter
-=====-=-=-= =-=- =-==-
http://arcgraph.de/sr/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

--- Jeff Garzik <jgarzik@pobox.com> wrote:
>
http://linux.adaptec.com/sas/git/?p=linux-2.6-sas.git;a=commit;h=785747ddc631f7618d728a377346965f7db2256a
> 
> This effectively illustrates the wrong thing to do:  duplicating 
> information that's already in Scsi_Host_Template.

No, it doesn't.  It shows you that accomodating of this is so very easy. Easy
and straightforward, just as the comming shows.

> Just use Scsi_Host_Template in the LLDD and see where that goes.

We've done this already: adp94xx submitted earlier this year, rejected by
the community, because "common SAS tasks should go to a common layer".  Now
you _have_ that common layer, the SAS Transport Stack/Layer.  But the
requirements of the community seem to change on a daily basis.

It looks like "the community" got more than it bargained for, and since
it doesn't understand it, it wants to bastardize the code and the layering.

> > While the architecture in my mind is clear, I cannot do this myself
> > (and for all drivers).  Such a change would be gradual, involving more
> > than one developer, for more than one (new) driver, etc.
> 
> Correct.  That's why there is resistance to aic94xx's approach of 
> creating a totally new "strict SAM" path, existing in parallel with the 

No, it is not correct -- you complety turn around anything I say to make
yourself look right.

Here is what I meant: the resistance is because a few people of Linux SCSI
want things done _their_ way, using _their_ code, reinventing code and concepts
done already by engineers who use the technology every day.

Why else is there a COMPLETLY BROKEN and WRONG "SAS transport attributes"?
Politics and influence, thats why.  So that you can say "change already
existing code".  So in effect politically "correct" people can submit
COMPLETY WRONG AND BROKEN CODE with their name on it, and when there is
a proper code wating to go in, they can BLOCK IT and ask that
current BROKEN code be changed (beyond recognition) code which made it
in, NOT because of technological merit but because of political influence.

Linux SCSI people should start _listening_ to technology engineers who
work with this technology every day.  If they showed this open attitude,
I can tell you that _a lot more_ engineers would get involved to _improve_
further Linux SCSI, who are holding out now just because of this "my way
or no way" attitude of a few folks at Linux SCSI.

Furthermore, "aic94xx" is NOT CREATING ANYTHING NEW.  This is how
USB Storage and SBP already work.  There is clearly separated transport
layer, independent of the interconnect.  That layer depends on the type
of device connected.  Using such an infrastructure you can layer more
varied architectures, as opposed to dictating that aic94xx needs the
host template in it, short-circuting proper layring.

I repeat again: apd94xx already has the Host Template IN IT.
I repeat again: nothing in the Hardware of AIC-94XX Host Adapter has
_anything_ to do with Host Templates.  Host Templates are a SCSI thing.
The hardware implements an access point to the _interconnect_: Execute
Command SCSI RPC and TMFs.  It is completely backwards to force something
at a place where it does not belong.
I repeat again: USB Storage and SBP _already_ implement such an architecture
as shown by SAS.

Had we had multiple Transport Layers back when SCSI Core was written (imported
from BSD?) I can tell you that there'd be such a layering disposistion as
shown in USB/SBP/SAS _already_ in the kernel -- just because IT MAKES SENSE.

> traditional SCSI core.  You need to evolve the existing code to get 
> there.  Such changes are gradual, involving more than one developer, etc.

Thank you for the mockery.  What I meant is that I don't mind working towards
this goal, with storage engineers who _know_ and _understand_ SCSI Technology,
not necessarily SAS, but SCSI in general.  There is only a few folks
like that here at lsml who are active and at least one whom I know of
who is not active.

What actually seems to be happening at Linux SCSI is not "working together"
but "If you do not do it _my_ way, I'll spin your words and say that
you are not willing to ``work together''." which is _exactly_ what you
do.

Is it possible for Linux SCSI to learn that they are doing something wrong?
Is it possible for Linux SCSI to learn that they need to listen and learn
from company engineers, as opposed to doing things _their_ way?
Apparently not.

> We don't need one small set of SCSI drivers behaving differently from 
> the vast majority of existing SCSI drivers.

USB Storage and SBP are already like that.

Then again why are you saying this?  Would you loose control?
Cannot you see that such changes are necessary to go forward towards
the future?  Do you think that such changes would be happening only
in Linux SCSI?

If Linux overall has the attitude you are showing, Linux would go extinct
in just a few years.  Just enough time for such antics to get to paying
customers (who not necessarily have technological know-how, but it would
eventually get to them).

> Hear me now, and believe me later:  we all largely agree on the points

You are a very good politican -- just keep repeating this.
 
> you've raised about legacy crapola in the SCSI core.  James, Christoph, 
> myself, and several others disagree with your assertion that the old 
> SCSI core should exist in parallel with your new SCSI core.

This isn't my new SCSI Core.  I don't have a new SCSI Core.
But I'm flattered that you feel this way.

> We differ on the path, not the goal.  As a thought experiment, you could 
> try simply implementing the changes requested, and see where that goes.

Been there, done that: adp94xx, rejected by none other but "the community".
Here is now the Linux SAS Stack and aic94xx, working completely fine,
full source code, git tree, patch file, whole working tree (Linus' as of
Friday evening + SAS Stack + aic94xx) present at the link in my sig.

   Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Matthew Wilcox]

On Fri, Oct 21, 2005 at 03:22:09PM -0400, Luben Tuikov wrote:
> On 10/21/05 14:54, Jeff Garzik wrote:
> >
> > I'm trying to tell it like it is, in the hopes that you will eventually 
> > learn the process, and be a good upstream maintainer we can all work with.
> 
> Look how you're using "all", generalizing left and right.  You just want to
> create this FUD and spread this FUD that I don't work well with anyone.  Yep,
> this is a pretty low blow.  While in fact did you talk to everyone I work with?

Since you asked, I found you quite impossible to deal with at the OLS
BOF you held, and I wasn't the only one.  I was hoping you'd learn, but
you seem unwilling to listen to other people's ideas.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:41, Matthew Wilcox wrote:
> 
> Since you asked, I found you quite impossible to deal with at the OLS
> BOF you held, and I wasn't the only one.  I was hoping you'd learn, but
> you seem unwilling to listen to other people's ideas.

Be specific, be technical.  All I did was merely present SAS.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Matthew Wilcox]

On Fri, Oct 21, 2005 at 03:48:04PM -0400, Luben Tuikov wrote:
> On 10/21/05 15:41, Matthew Wilcox wrote:
> > 
> > Since you asked, I found you quite impossible to deal with at the OLS
> > BOF you held, and I wasn't the only one.  I was hoping you'd learn, but
> > you seem unwilling to listen to other people's ideas.
> 
> Be specific, be technical.  All I did was merely present SAS.

No.  What was advertised was a SCSI BOF which you then took over and
spent the entire time talking about the Adaptec SAS driver.  You weren't
interested in discussing wider SCSI issues.  You weren't interested in
talking about how other vendors implemented SAS.  You weren't interested
in discussing how we could get the best possible SAS interface in Linux.
You shut down other people when they tried to discuss these things.
It was a complete waste of time.

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:54, Matthew Wilcox wrote:
> 
> No.  What was advertised was a SCSI BOF which you then took over and

It was advertised as "SAS BOF" -- the person who wrote that on the message
board (reading this list currently) can verify that.

> spent the entire time talking about the Adaptec SAS driver.  You weren't
> interested in discussing wider SCSI issues.  You weren't interested in
> talking about how other vendors implemented SAS.  You weren't interested
> in discussing how we could get the best possible SAS interface in Linux.

I still am.  What everyone now wants is SDI. And as you can see I've posted
several times _code_ and templates as to how to do a backend which would
work as per spec and a front end which would be adjustable to the whims of
"the community", sg/sysfs/whatever1/whatever2.

I think SDI will completely satisfy everyone's needs, independently of
the fact whether the the protocol is hidden in the FW or not.

In fact Fusion MPT is very cool: you only add a few PCI IDs and your
hw works with the same driver!  And if you care about protocol
specifics: use SDI.

But the community wanted involvement so then you say: "No! Give us
your hardware, we'll do it for you." and then you get into this
never-ending goose chase, implementing the wrong thing, the wrong way,
as opposed to _listening_ to what is actually wanted.

> You shut down other people when they tried to discuss these things.
> It was a complete waste of time.

Sorry you feel this way.  I don't remember you saying anything about
SAS.
	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Arjan van de Ven]

> I say, when people are losing it on the technical front, they try to attack
> personally and on political basis.

Luben,

to be honest, a lot of people got exactly the impression that you've
been doing that from the start in this saga, by attacking Christophs and
James' integrity early on etc. Jeff has been just about the only person
who ignored that and kept talking to you, and now you're attacking him
too... I'm not sure what you're trying to achieve. But I'm pretty sure
this way isn't the best or easiest way to achieve that. 

Greetings,
   Arjan van de Ven

Re: ioctls, etc. (was Re: [PATCH 1/4] sas: add flag for locally attached PHYs)

[Luben Tuikov]

On 10/21/05 15:46, Arjan van de Ven wrote:
>>I say, when people are losing it on the technical front, they try to attack
>>personally and on political basis.
> 
> 
> Luben,
> 
> to be honest, a lot of people got exactly the impression that you've
> been doing that from the start in this saga, by attacking Christophs and
> James' integrity early on etc. Jeff has been just about the only person
> who ignored that and kept talking to you, and now you're attacking him
> too... I'm not sure what you're trying to achieve. But I'm pretty sure
> this way isn't the best or easiest way to achieve that. 

No, look back on this thread -- he started a personal attack first.
Here is the email:
http://marc.theaimsgroup.com/?l=linux-scsi&m=112991877716308&w=2

This is where the "you don't understand" personal attacks from Jeff
start.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

On Thu, 2005-10-20 at 18:01 -0600, Andrew Patterson wrote:

> Yes, CSMI should have had more Linux input when it was developed.  The
> no-new IOCTL policy certainly came as a surprise to the authors. Still,
> there doesn't seem to be any other usable cross-platform interface that
> is acceptable to the linux community (or at least to Christoph)'s corner
> of it).  My personal preference is to hide this stuff in a library, so
> the kernel implementation is hidden. But even a library needs an
> underlying kernel implementation.


but why didn't CSMI specify the library interface then? If it did that,
then none of the "ioctls are deprecated for 3 years now" would have come
as a surprise. Or rather, it maybe would have, but it wouldn't have
mattered to the management app.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Andrew Patterson]

On Fri, 2005-10-21 at 11:06 +0200, Arjan van de Ven wrote:
> On Thu, 2005-10-20 at 18:01 -0600, Andrew Patterson wrote:
> 
> > Yes, CSMI should have had more Linux input when it was developed.  The
> > no-new IOCTL policy certainly came as a surprise to the authors. Still,
> > there doesn't seem to be any other usable cross-platform interface that
> > is acceptable to the linux community (or at least to Christoph)'s corner
> > of it).  My personal preference is to hide this stuff in a library, so
> > the kernel implementation is hidden. But even a library needs an
> > underlying kernel implementation.
> 
> 
> but why didn't CSMI specify the library interface then? If it did that,
> then none of the "ioctls are deprecated for 3 years now" would have come
> as a surprise. Or rather, it maybe would have, but it wouldn't have
> mattered to the management app.
> 

I asked one of the authors, Steve Fairchild, about this.  He said that
they had had a bad experience with HBAAPI, the FC library.  They had a
lot of problems with vendors keeping the library up to date, multiple
vendors shipping multiple incompatible versions, etc.  I wasn't entirely
convinced, since they seem to have just replaced one set of problems for
another.

Andrew

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

> > 
> > but why didn't CSMI specify the library interface then? If it did that,
> > then none of the "ioctls are deprecated for 3 years now" would have come
> > as a surprise. Or rather, it maybe would have, but it wouldn't have
> > mattered to the management app.
> > 
> 
> I asked one of the authors, Steve Fairchild, about this.  He said that
> they had had a bad experience with HBAAPI, the FC library.


>   They had a
> lot of problems with vendors keeping the library up to date

that's not too hard (and up to date is relative)

> , multiple
> vendors shipping multiple incompatible versions, etc. 

which sounds like they didn't get the spec right.. if the spec was good
that couldn't happen at all...

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/21/05 13:05, Andrew Patterson wrote:
>>but why didn't CSMI specify the library interface then? If it did that,
>>then none of the "ioctls are deprecated for 3 years now" would have come
>>as a surprise. Or rather, it maybe would have, but it wouldn't have
>>mattered to the management app.
> 
> I asked one of the authors, Steve Fairchild, about this.  He said that
> they had had a bad experience with HBAAPI, the FC library.  They had a
> lot of problems with vendors keeping the library up to date, multiple
> vendors shipping multiple incompatible versions, etc.  I wasn't entirely
> convinced, since they seem to have just replaced one set of problems for
> another.

The user app would read the SDI major and minor from SDI_GET_DRIVER_INFO
and then know which spec to use.

After that just sending bytes of data (forming units) one after the other
(as in "smp_portal") would suffice for 32/64 bit (similarly to SG).

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 20:01, Andrew Patterson wrote:
> 
> My guess is that what current app writers want is to use IOCTL's so they
> don't have to special-case Linux.  Next best thing would be something
> that closely approaches it, to avoid re-writing a lot of code.  

"to avoid re-writing a lot of code." is also what I'm driving at.
I'm sure HP as well as LSI have a lot of user space programs using
the SDI interface.

>>Rejected by whom?  "The community" or by you?
> 
> 
> I believe there is a common understanding that IOCTL's are bad and
> should be avoided. See:
> 
> http://lkml.org/lkml/2001/5/20/81

I wasn't talking about IOCTLs.  I was talking in general about the
SDI spec.  As you can see for yourself, I suggested an SDI interface
which is agnostic to the front end.

>>>HP decided to move ahead despite that and did a huge mis-services to
>>>their customers.
> 
> 
> Perhaps.  Given that there seems to be no alternative, we don't have
> much of a choice.

Please do *NOT* cross-reply.  It was Christoph who said that, NOT ME.

>>>It's not my problem if big companies can't listen and do things their
>>>customers have to pay for, and it's certainly not our job to work
>>>around their idiocy.
> 
> 
> Yes, CSMI should have had more Linux input when it was developed.  The
> no-new IOCTL policy certainly came as a surprise to the authors. Still,
> there doesn't seem to be any other usable cross-platform interface that
> is acceptable to the linux community (or at least to Christoph)'s corner
> of it).  My personal preference is to hide this stuff in a library, so
> the kernel implementation is hidden. But even a library needs an
> underlying kernel implementation.

Please do *NOT* cross-reply.  It was Christoph who said that, NOT ME.

>>Bold statment.
>>
>>Who should "big companies" listen to?  You?  "The community?"
>>Are you saying "big companies" whould listen to Linux which
>>says "no to specs" among other things?
>>
>>Often enough what "big companies" agree on and use and deploy is
>>what Linux (you?) should _listen_ to, try to understand and maybe
>>get out of the way.
>>
> 
> 
> Big companies often want to do things in a proprietary fashion.  I
> personally would prefer to see a standard's-based approach.

I think SDI is what we're talking about here.

>>It is all about customer satisfaction, which is completely
>>foreign to Linux, simply because of the _nature_ of Linux.
> 
> 
> More bold statements? ;-)

No, its just reality.  It appears that you have had very little
interaction with your software engineers about "compatibility".
Also read the Linux blurb on kernel.org, the "What is Linux?" one.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Douglas Gilbert]

Moore, Eric Dean wrote:
> On Wednesday, October 19, 2005 3:00 PM, Luben Tuikov wrote: 
> 
> 
>>On 10/19/05 16:08, Moore, Eric Dean wrote:

<snip>

> Luben suggestion(correct me if I'm wrong) is this should be 
> done thru user space, instead of exporting this in the 
> /sys/class/sas_phy attributes.  Thus we need to export a 
> SMP_PASSTHRU mechanism.  I believe there is discussion
> by Christoph ( and I believe Luben backs this idea) of doing 
> all this thru the /dev/sg interface.

Folks,
Just a suggestion if you want to bring /dev/sg into
the picture for a SMP (and even a STP) passthrough.
The host could define a lun (pick a number) and the
LLDD could supply the target code for it (as scsi_debug
does for example).

Details need to be sorted out (e.g. pdt of "host" lun)
and a way to pass an 8 byte SAS address through a
SCSI cdb (there are lots of vendor specific SCSI
opcodes available). For a STP pass through, we
have the SAT ATA PASS THROUGH SCSI commands already
defined (although they don't allow for an 8
byte target (SAS) address).

SG_IO code paths through the SCSI subsystem are pretty
well tested and are available in all lk 2.4 and 2.6
kernels. All OSes that I am aware of have some sort
of SCSI pass through mechanism so this approach bypasses
the objection to adding new ioctls (from both Microsoft
and Linux).

The idea is not original. Windows 2000 has a lun for
just this purpose (and it was dropped in Windows 2003).

Doug Gilbert

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Jeff Garzik]

Douglas Gilbert wrote:
> Just a suggestion if you want to bring /dev/sg into
> the picture for a SMP (and even a STP) passthrough.

Yep.  That's another possibility, that falls into the 'block driver' 
category [see list of categories in my 'ioctl' email], since it is 
translated into a 'special' block request, and then sent to a normal 
request_queue.

Jens Axboe's bsg[1] has also been mentioned in the context of SMP.

	Jeff


[1] http://lwn.net/Articles/96547/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Douglas Gilbert]

Jeff Garzik wrote:
> Douglas Gilbert wrote:
> 
>> Just a suggestion if you want to bring /dev/sg into
>> the picture for a SMP (and even a STP) passthrough.
> 
> 
> Yep.  That's another possibility, that falls into the 'block driver'
> category [see list of categories in my 'ioctl' email], since it is
> translated into a 'special' block request, and then sent to a normal
> request_queue.
> 
> Jens Axboe's bsg[1] has also been mentioned in the context of SMP.

Jeff,
Yes, I like the bsg model: one well known device (/dev/bsg),
get a file descriptor and then bind to a "target". It
reminded me of a network interface.

As presented, it only bound to a block (or char?) device
node (e.g. /dev/sda). I asked some questions about widening
the variety of things it could bind to but Peter Jones
suggested that binding was a necessary evil that should
not be encouraged.

So perhaps we could bind to a SAS host ... (and a SCSI device
for those pesky things like safte, ses and wluns).

Doug Gilbert

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 21:50, Douglas Gilbert wrote:
>>Luben suggestion(correct me if I'm wrong) is this should be 
>>done thru user space, instead of exporting this in the 
>>/sys/class/sas_phy attributes.  Thus we need to export a 
>>SMP_PASSTHRU mechanism.  I believe there is discussion
>>by Christoph ( and I believe Luben backs this idea) of doing 
>>all this thru the /dev/sg interface.
> 
> 
> Folks,
> Just a suggestion if you want to bring /dev/sg into
> the picture for a SMP (and even a STP) passthrough.
> The host could define a lun (pick a number) and the
> LLDD could supply the target code for it (as scsi_debug
> does for example).

Doug, there is no reason to follow someone else's OS
broken path.

Your question above is SDI addressability. And the answer
is _right_ in the SDI spec: SDI is addressable as per
"controller".  That is each "controller" on the PCI/IO
bus provides an SDI portal.  This is overly evident
by at least looking at the SDI_GET_CNTLR_CONFIG function.

This allows for controllers who do not have SCSI LLDDs
but block or RAID LLDD to also be able to export an SDI
portal.

So in effect the SDI portal would have to be hooked to the
PCI device which claims a base class of 01h -- mass storage
controller.

So when the provider calls 
	sdi_register_provider(sdi_prov);

it fill out the pcidev member in

struct sdi_provider_struct {
	char           *name;
	struct pci_dev *pcidev;
	...
	struct sdi_functions_struct sdi_functions;
	...
};

from my example yesterday.  And the file drivers/scsi/sdi/sdi.c
creates a binary file attr (front end, could be another way)
in  /sys/devices/pciABCD:EF/.../sdi_portal.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

RE: [PATCH 1/4] sas: add flag for locally attached PHYs

[Moore, Eric Dean]

On Thursday, October 20, 2005 9:29 AM,  Luben Tuikov wrote:
 
> >  That's why I need the local_attached flag as an internal
> > helper - it's not actually exposed to userland so the implementation
> 
> 1) LSI does have SMP pass through.
> 2) You absolutely do not need to represent all phys on the domain,
> which is, as I said, a completely _crazy_ idea.  Look, even 
> SDI doesn't
> want you to do that.  What SDI wants you to do is issue SMP out.
> LSI does have SMP pass through in FW so use that.
> 3) All you care about is SDI, and SDI has a _spec_.  Your job is
> twofold:
> 	- implement drivers/scsi/sdi.c so we can all use it,
> 	- implement the implementation/hw dependent way to
> 	access SDI functionality for Fusion as described above
> 	where I mention that this would be implementation dependent
> 	(dispatching to).
> 

Thanks Luben - My concern is converting our SDI/CSMI ioctls
into what is acceptable in mainstream kernel.  Agreed with Luben,
all the expander data gathering, and configuration can be done 
in user space, which is what is being done by the HP application
folks which are currently using our internal CSMI enabled drivers.
 
I added expander support a couple rev's back since none of the 
non-direct attached sas/sata devices were not scanned and reported
to the block layer above.  This was a problem when our sas driver 
was converted to work with sas_transport.

Best Regards,
Eric Moore

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 11:45, Moore, Eric Dean wrote:
> 
> Thanks Luben - My concern is converting our SDI/CSMI ioctls
> into what is acceptable in mainstream kernel.  Agreed with Luben,
> all the expander data gathering, and configuration can be done 
> in user space, which is what is being done by the HP application
> folks which are currently using our internal CSMI enabled drivers.

Well, this is very good news.  This means less deployment time,
less code which needs to be changed, less code which needs
to go through testing and faster customer reach.

This should suggest Christoph "which way it should be done."

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

RE: [PATCH 1/4] sas: add flag for locally attached PHYs

[Moore, Eric Dean]

On Wednesday, October 19, 2005 1:24 PM, Luben Tuikov wrote: 

> On 10/19/05 14:01, Christoph Hellwig wrote:
> > Add a flag to mark a PHY as attached to the HBA as opposed 
> to beeing on
> > an expander.  This is needed because various features are 
> only supported
> > on those.  This is a crude hack, the proper fix would be to use
> > different classes for host-attached vs expander phys.  I'm 
> looking into
> > that.
> 
> The phy isn't quite "attached" -- i.e. at software level you do not
> care about that.  At least I've never heard anyone (not of 
> "the community")
> say that a phy is "attached".
> 

Luben:  He is adding support for linkerrors and link/phy reset;
these for for CSMI/SDI API.

Can you in your driver retreive link errors for the links on the expanders?

Same question for sending link and phy resets for expander phys? 
If so, then perhaps that check should be removed, and lets let the lld
decide whether these attibutes should return -EINVAL.

For fusion, these new attributes only work for the phys on the hba.
The problem is the firmware implmentation for `link errors and 
link/phy reset` is addressed only by the phy identifier for the phys
attacted directly on the host adapter.  Doesn't work for phys on
the expander. 

> You don't need to represent that.  While you can, you 
> completely do not
> need to do it.  All you should care about is the _port_.  Take a look
> at SAS section 4.
> 
> Also take a look at:
> drivers/scsi/sas/sas_phy.c,
> drivers/scsi/sas/sas_port.c and
> drivers/scsi/sas/sas_discover.c .

What and where can I get this?

Best regards,
Eric Moore

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/19/05 16:08, Moore, Eric Dean wrote:
> Luben:  He is adding support for linkerrors and link/phy reset;
> these for for CSMI/SDI API.
> 
> Can you in your driver retreive link errors for the links on the expanders?

Yes. (offloaded to user space)

You use the user space program called "expander_conf.c" to
send SMP requests and receive SMP responses.

An example output of "expander_conf.c" for a domain
is shown here: (everything you want to know about expanders):

http://linux.adaptec.com/sas/Announce_2.txt

The domain of the above is shown here:

http://linux.adaptec.com/sas/Announce_1.txt

> Same question for sending link and phy resets for expander phys? 

Yes, for both local phys and domain device phys, any device
claiming an SMP port.

Local phys
----------

If the LLDD provides this, it fills out the function stub:

	/* Phy management */
	int (*lldd_control_phy)(struct sas_phy *, enum phy_func);

in the "struct sas_ha_struct" structure before it registers
with the SAS Transport Layer.  See include/scsi/sas/sas_class.h
drivers/scsi/aic94xx/aic94xx_init.c::asd_register_sas_ha(), and
drivers/scsi/sas/README.

(When the SAS LLDD (aic94xx) registers with the SAS Transport
 layer, it declares all the phys the HA has; this is in
 struct sas_ha_struct::struct sas_phy  **sas_phy;  This is so
 that the SAS Transport layer can control the phys to form
 ports, etc, as stipulated in
 http://linux.adaptec.com/sas/Announce_0.txt.)

Then you do:

	sas_ha->lldd_control_phy(phy, func);

where,

	struct sas_ha_struct *sas_ha;
	struct sas_phy *phy;
	enum   phy_func func;

See
include/scsi/sas/sas.h and
include/scsi/sas/sas_class.h

Phys on Domain devices with an SMP port
---------------------------------------

Execute SCSI Command RPC is used to send
SMP tasks:

This is from drivers/scsi/sas/sas_expander.c:

<< line 480 >>
#define PC_REQ_SIZE  44
#define PC_RESP_SIZE 8

static int smp_phy_control(struct domain_device *dev, int phy_id,
			   enum phy_func phy_func)
{
	u8 *pc_req;
	u8 *pc_resp;
	int res;

	pc_req = alloc_smp_req(PC_REQ_SIZE);
	if (!pc_req)
		return -ENOMEM;

	pc_resp = alloc_smp_resp(PC_RESP_SIZE);
	if (!pc_resp) {
		kfree(pc_req);
		return -ENOMEM;
	}

	pc_req[1] = SMP_PHY_CONTROL;
	pc_req[9] = phy_id;
	pc_req[10]= phy_func;

	res = smp_execute_task(dev, pc_req, PC_REQ_SIZE, pc_resp,PC_RESP_SIZE);

	kfree(pc_resp);
	kfree(pc_req);
	return res;
}

Then you can look up the implementation of

static int smp_execute_task(struct domain_device *dev, void *req, int req_size,
			    void *resp, int resp_size)

on line 175 of drivers/scsi/sas/sas_expander.c, and you can follow
it up to
	task->dev->port->ha->lldd_execute_task(task, 1, GFP_KERNEL);

where lldd_execute_task() is implemented in
drivers/scsi/aic94xx/aic94xx_task.c, on line 555:

int asd_execute_task(struct sas_task *task, const int num,
		     unsigned long gfp_flags)

which immediately sends the task out on the interconnect.

(Nugget: "dev->port" doesn't _necessarily_ mean that
 "dev" /formed/ "port", but that "dev" is reachable
 /through/ "port". "dev" may or may not form "port".)

> If so, then perhaps that check should be removed, and lets let the lld
> decide whether these attibutes should return -EINVAL.

In the SAS Transport Layer, all this has been off loaded to user space.
See drivers/scsi/sas/README "Expander management from User Space"
section (bottom of file).

> For fusion, these new attributes only work for the phys on the hba.

For aic94xx, these attributes are supported for any SMP port,
i.e. local phys and any phys out there which claim SMP.

> The problem is the firmware implmentation for `link errors and 
> link/phy reset` is addressed only by the phy identifier for the phys
> attacted directly on the host adapter.  Doesn't work for phys on
> the expander. 

What you need is a general facility to send SMP tasks.

See drivers/scsi/sas/sas_expander.c,
    include/scsi/sas/sas_task.h      and
    drivers/scsi/aic94xx/aic94xx_task.c

>>You don't need to represent that.  While you can, you 
>>completely do not
>>need to do it.  All you should care about is the _port_.  Take a look
>>at SAS section 4.
>>
>>Also take a look at:
>>drivers/scsi/sas/sas_phy.c,
>>drivers/scsi/sas/sas_port.c and
>>drivers/scsi/sas/sas_discover.c .
> 
> 
> What and where can I get this?

http://linux.adaptec.com/sas/
Also see my sig.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

On Wed, Oct 19, 2005 at 05:00:03PM -0400, Luben Tuikov wrote:
> On 10/19/05 16:08, Moore, Eric Dean wrote:
> > Luben:  He is adding support for linkerrors and link/phy reset;
> > these for for CSMI/SDI API.
> > 
> > Can you in your driver retreive link errors for the links on the expanders?
> 
> Yes. (offloaded to user space)
> 
> You use the user space program called "expander_conf.c" to
> send SMP requests and receive SMP responses.

Yes, exactly.  On the other hand we don't have standard
hardware-indepent ways of doing that for the PHYs of the HBA because
the programming interfaces are to different and need to abstract it away
in the kernel.  That's why I need the local_attached flag as an internal
helper - it's not actually exposed to userland so the implementation

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 10:11, Christoph Hellwig wrote:
> On Wed, Oct 19, 2005 at 05:00:03PM -0400, Luben Tuikov wrote:
> 
>>On 10/19/05 16:08, Moore, Eric Dean wrote:
>>
>>>Luben:  He is adding support for linkerrors and link/phy reset;
>>>these for for CSMI/SDI API.
>>>
>>>Can you in your driver retreive link errors for the links on the expanders?
>>
>>Yes. (offloaded to user space)
>>
>>You use the user space program called "expander_conf.c" to
>>send SMP requests and receive SMP responses.
> 
> 
> Yes, exactly.  On the other hand we don't have standard
> hardware-indepent ways of doing that for the PHYs of the HBA because
> the programming interfaces are to different and need to abstract it away
> in the kernel.

The _protocol_ way is SMP.  And LSI does implement SMP passthrough.

You have two scenarios: local phys and SMP.

Note that your task as wanted by HP/LSI/etc (all who use Fusion MPT)
is to implement SDI/CSMI, see http://www.t10.org/drafts.htm#sdi- .

If you implement this, as say, drivers/scsi/sdi.c, then I'd
be more than happy to register with it so as to export
SDI/CSMI functionality.

E.g. if you look at SDI_GET_PHY_INFO you will notice
that this is only for phys of the Host Adapter, NOT all
phys on the domain (which is a _crazy_ concept).

Most of the code has been written right in the SDI spec
sdi-r00.pdf.  You can start from there.  And most of the concepts
have been implemented in say LSI's FW and in my code, the SAS
Transport Layer.

E.g. you can see how "port identifier is u8" which is exactly
what, I'm sure, LSI's fw uses (or can fit in), and as you can see
in my code that I use (I actually use "int").

Now, _registration_ of SDI by different entities would be common,
say, sdi_register_provider(struct ...); Also the _mechanism_
to access that facility would be common*.  But _dispatching_
would be implementation dependent as you can see LSI would
do it differently and Adaptec would do it differently and
XYZ would do it differently, etc.

* I say common because you can use sg, in which case the
IOCTLs described in the spec are directly translated into
"headers" to be issued down sg.

Your task is clearly defined by a spec (sdi-r00.pdf).
Most of the functions defined therein are direct SAS concepts
and, as you can see, translate to an SMP function, or to a TMF,
since this is where they were taken from to begin with.

There is no need for translation layers upon translation
layers.  The task is clearly defined: LSI wants SDI/CSMI.
You have a CSMI spec.  Implement it.  This would make
everyone's job so much easier without destroying already
existing _good_ code.

>  That's why I need the local_attached flag as an internal
> helper - it's not actually exposed to userland so the implementation

1) LSI does have SMP pass through.
2) You absolutely do not need to represent all phys on the domain,
which is, as I said, a completely _crazy_ idea.  Look, even SDI doesn't
want you to do that.  What SDI wants you to do is issue SMP out.
LSI does have SMP pass through in FW so use that.
3) All you care about is SDI, and SDI has a _spec_.  Your job is
twofold:
	- implement drivers/scsi/sdi.c so we can all use it,
	- implement the implementation/hw dependent way to
	access SDI functionality for Fusion as described above
	where I mention that this would be implementation dependent
	(dispatching to).

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

<lots of useless rants snipped>

Yes, we have two cases, local phy and phy on an expander.  The proper
way to support remote phys is via smp passthru as we hopefully agree.
Right now the sas transport class doesn't do smp passthru (although
I'm working on it) so we only implement local link stats.  Once we have
smp passthru we can support link stats, in one of two ways:

 - do it in userland as in your example program.  in that case we don't
   need to do anything in kernel land and need the local_attached flag
   to disable this functionality for expander phys.
 - do it in kernel, in which case the transport class must call the smp
   passthru entry point of the driver to query link stats, so it needs
   the local_attached flag aswell for that.

What's your problem with the flag now?

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 11:49, Christoph Hellwig wrote:
> <lots of useless rants snipped>

Hi Christoph,

For some reason I thought that I was quoting specs
and showing constructive ideas (i.e. things you can sit down
and start coding).

I'm sorry that you feel that it is "useless rants".

On a topic note: It appears that the "sas transport attributes" are
nothing more than SDI/CSMI.

> Yes, we have two cases, local phy and phy on an expander.  The proper
> way to support remote phys is via smp passthru as we hopefully agree.
> Right now the sas transport class doesn't do smp passthru (although
> I'm working on it) so we only implement local link stats.

You are so much better off implementing SDI/CSMI as per the spec.

Normally, at companies, projects have a "specificaion" and "required
functionality".  What is the "specificaion" and "required functionality"
at hand?

Note that there are LSI/HP customers who are completely _happy_
to use CSMI as provided by LSI and they are not interested in 
what-have-you-not.

The easiest way to make those customers happy and keep them happy is
to provide general SDI/CSMI facitliy and _you have a spec on that_.

> Once we have
> smp passthru we can support link stats, in one of two ways:
> 
>  - do it in userland as in your example program.  in that case we don't
>    need to do anything in kernel land and need the local_attached flag
>    to disable this functionality for expander phys.
>  - do it in kernel, in which case the transport class must call the smp
>    passthru entry point of the driver to query link stats, so it needs
>    the local_attached flag aswell for that.
> 
> What's your problem with the flag now?

It is ugly, it is a (brown) paper bag solution, it is a work around.

Take a look at the phy attributes as per the SAS spec.  Then take
a look at struct sas_phy in include/scsi/sas/sas_class.h.  Do you
see such a flag in either reference?

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

[PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

Add a flag to mark a PHY as attached to the HBA as opposed to beeing on
an expander.  This is needed because various features are only supported
on those.  This is a crude hack, the proper fix would be to use
different classes for host-attached vs expander phys.  I'm looking into
that.


Signed-off-by: Christoph Hellwig <hch@lst.de>

Index: linux-2.6/drivers/message/fusion/mptsas.c
===================================================================
--- linux-2.6.orig/drivers/message/fusion/mptsas.c	2005-10-19 15:30:29.000000000 +0200
+++ linux-2.6/drivers/message/fusion/mptsas.c	2005-10-19 19:49:35.000000000 +0200
@@ -760,7 +760,7 @@
 }
 
 static int mptsas_probe_one_phy(struct device *dev,
-		struct mptsas_phyinfo *phy_info, int index)
+		struct mptsas_phyinfo *phy_info, int index, int local)
 {
 	struct sas_phy *port;
 	int error;
@@ -853,6 +853,9 @@
 		break;
 	}
 
+	if (local)
+		port->local_attached = 1;
+
 	error = sas_phy_add(port);
 	if (error) {
 		sas_phy_free(port);
@@ -918,7 +921,7 @@
 		}
 
 		mptsas_probe_one_phy(&ioc->sh->shost_gendev,
-				     &port_info->phy_info[i], *index);
+				     &port_info->phy_info[i], *index, 1);
 		(*index)++;
 	}
 
@@ -989,7 +992,8 @@
 			}
 		}
 
-		mptsas_probe_one_phy(parent, &port_info->phy_info[i], *index);
+		mptsas_probe_one_phy(parent, &port_info->phy_info[i],
+				     *index, 0);
 		(*index)++;
 	}
 
Index: linux-2.6/include/scsi/scsi_transport_sas.h
===================================================================
--- linux-2.6.orig/include/scsi/scsi_transport_sas.h	2005-10-19 15:30:29.000000000 +0200
+++ linux-2.6/include/scsi/scsi_transport_sas.h	2005-10-19 19:49:35.000000000 +0200
@@ -56,6 +56,9 @@
 	enum sas_linkrate	maximum_linkrate;
 	u8			port_identifier;
 
+	/* internal state */
+	unsigned int		local_attached : 1;
+
 	/* link error statistics */
 	u32			invalid_dword_count;
 	u32			running_disparity_error_count;
Index: linux-2.6/drivers/scsi/scsi_transport_sas.c
===================================================================
--- linux-2.6.orig/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:49:35.000000000 +0200
+++ linux-2.6/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:50:55.000000000 +0200
@@ -266,6 +266,9 @@
 	struct sas_internal *i = to_sas_internal(shost->transportt);	\
 	int error;							\
 									\
+	if (!phy->local_attached)					\
+		return -EINVAL;						\
+									\
 	error = i->f->get_linkerrors(phy);				\
 	if (error)							\
 		return error;						\

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/19/05 14:01, Christoph Hellwig wrote:
> Add a flag to mark a PHY as attached to the HBA as opposed to beeing on
> an expander.  This is needed because various features are only supported
> on those.  This is a crude hack, the proper fix would be to use
> different classes for host-attached vs expander phys.  I'm looking into
> that.

The phy isn't quite "attached" -- i.e. at software level you do not
care about that.  At least I've never heard anyone (not of "the community")
say that a phy is "attached".

You don't need to represent that.  While you can, you completely do not
need to do it.  All you should care about is the _port_.  Take a look
at SAS section 4.

Also take a look at:
drivers/scsi/sas/sas_phy.c,
drivers/scsi/sas/sas_port.c and
drivers/scsi/sas/sas_discover.c .

Storage GUI applications can read the expander configuration using SMP
and show this as I've shown in Announcement 1 here:
http://marc.theaimsgroup.com/?l=linux-scsi&m=112629509826900&w=2

All you should care about is the _local_ phys since those are
under your control, and the port formed.

	Luben


> 
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> Index: linux-2.6/drivers/message/fusion/mptsas.c
> ===================================================================
> --- linux-2.6.orig/drivers/message/fusion/mptsas.c	2005-10-19 15:30:29.000000000 +0200
> +++ linux-2.6/drivers/message/fusion/mptsas.c	2005-10-19 19:49:35.000000000 +0200
> @@ -760,7 +760,7 @@
>  }
>  
>  static int mptsas_probe_one_phy(struct device *dev,
> -		struct mptsas_phyinfo *phy_info, int index)
> +		struct mptsas_phyinfo *phy_info, int index, int local)
>  {
>  	struct sas_phy *port;
>  	int error;
> @@ -853,6 +853,9 @@
>  		break;
>  	}
>  
> +	if (local)
> +		port->local_attached = 1;
> +
>  	error = sas_phy_add(port);
>  	if (error) {
>  		sas_phy_free(port);
> @@ -918,7 +921,7 @@
>  		}
>  
>  		mptsas_probe_one_phy(&ioc->sh->shost_gendev,
> -				     &port_info->phy_info[i], *index);
> +				     &port_info->phy_info[i], *index, 1);
>  		(*index)++;
>  	}
>  
> @@ -989,7 +992,8 @@
>  			}
>  		}
>  
> -		mptsas_probe_one_phy(parent, &port_info->phy_info[i], *index);
> +		mptsas_probe_one_phy(parent, &port_info->phy_info[i],
> +				     *index, 0);
>  		(*index)++;
>  	}
>  
> Index: linux-2.6/include/scsi/scsi_transport_sas.h
> ===================================================================
> --- linux-2.6.orig/include/scsi/scsi_transport_sas.h	2005-10-19 15:30:29.000000000 +0200
> +++ linux-2.6/include/scsi/scsi_transport_sas.h	2005-10-19 19:49:35.000000000 +0200
> @@ -56,6 +56,9 @@
>  	enum sas_linkrate	maximum_linkrate;
>  	u8			port_identifier;
>  
> +	/* internal state */
> +	unsigned int		local_attached : 1;
> +
>  	/* link error statistics */
>  	u32			invalid_dword_count;
>  	u32			running_disparity_error_count;
> Index: linux-2.6/drivers/scsi/scsi_transport_sas.c
> ===================================================================
> --- linux-2.6.orig/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:49:35.000000000 +0200
> +++ linux-2.6/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:50:55.000000000 +0200
> @@ -266,6 +266,9 @@
>  	struct sas_internal *i = to_sas_internal(shost->transportt);	\
>  	int error;							\
>  									\
> +	if (!phy->local_attached)					\
> +		return -EINVAL;						\
> +									\
>  	error = i->f->get_linkerrors(phy);				\
>  	if (error)							\
>  		return error;						\
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/19/05 15:24, Luben Tuikov wrote:
> On 10/19/05 14:01, Christoph Hellwig wrote:
> 
>>Add a flag to mark a PHY as attached to the HBA as opposed to beeing on
>>an expander.  This is needed because various features are only supported
>>on those.  This is a crude hack, the proper fix would be to use
>>different classes for host-attached vs expander phys.  I'm looking into
>>that.
> 
> 
> The phy isn't quite "attached" -- i.e. at software level you do not
> care about that.  At least I've never heard anyone (not of "the community")
> say that a phy is "attached".
> 
> You don't need to represent that.  While you can, you completely do not
> need to do it.  All you should care about is the _port_.  Take a look
> at SAS section 4.
> 
> Also take a look at:
> drivers/scsi/sas/sas_phy.c,
> drivers/scsi/sas/sas_port.c and
> drivers/scsi/sas/sas_discover.c .
> 
> Storage GUI applications can read the expander configuration using SMP
> and show this as I've shown in Announcement 1 here:
> http://marc.theaimsgroup.com/?l=linux-scsi&m=112629509826900&w=2

And Announcement 2 (more specifically using the user space configuration
utility program "expander_conf.c"):
http://marc.theaimsgroup.com/?l=linux-scsi&m=112629509318354&w=2

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

 
> All you should care about is the _local_ phys since those are
> under your control, and the port formed.
> 
> 	Luben