[PATCH 1/4] sas: add flag for locally attached PHYs

[PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

Add a flag to mark a PHY as attached to the HBA as opposed to beeing on
an expander.  This is needed because various features are only supported
on those.  This is a crude hack, the proper fix would be to use
different classes for host-attached vs expander phys.  I'm looking into
that.


Signed-off-by: Christoph Hellwig <hch@lst.de>

Index: linux-2.6/drivers/message/fusion/mptsas.c
===================================================================
--- linux-2.6.orig/drivers/message/fusion/mptsas.c	2005-10-19 15:30:29.000000000 +0200
+++ linux-2.6/drivers/message/fusion/mptsas.c	2005-10-19 19:49:35.000000000 +0200
@@ -760,7 +760,7 @@
 }
 
 static int mptsas_probe_one_phy(struct device *dev,
-		struct mptsas_phyinfo *phy_info, int index)
+		struct mptsas_phyinfo *phy_info, int index, int local)
 {
 	struct sas_phy *port;
 	int error;
@@ -853,6 +853,9 @@
 		break;
 	}
 
+	if (local)
+		port->local_attached = 1;
+
 	error = sas_phy_add(port);
 	if (error) {
 		sas_phy_free(port);
@@ -918,7 +921,7 @@
 		}
 
 		mptsas_probe_one_phy(&ioc->sh->shost_gendev,
-				     &port_info->phy_info[i], *index);
+				     &port_info->phy_info[i], *index, 1);
 		(*index)++;
 	}
 
@@ -989,7 +992,8 @@
 			}
 		}
 
-		mptsas_probe_one_phy(parent, &port_info->phy_info[i], *index);
+		mptsas_probe_one_phy(parent, &port_info->phy_info[i],
+				     *index, 0);
 		(*index)++;
 	}
 
Index: linux-2.6/include/scsi/scsi_transport_sas.h
===================================================================
--- linux-2.6.orig/include/scsi/scsi_transport_sas.h	2005-10-19 15:30:29.000000000 +0200
+++ linux-2.6/include/scsi/scsi_transport_sas.h	2005-10-19 19:49:35.000000000 +0200
@@ -56,6 +56,9 @@
 	enum sas_linkrate	maximum_linkrate;
 	u8			port_identifier;
 
+	/* internal state */
+	unsigned int		local_attached : 1;
+
 	/* link error statistics */
 	u32			invalid_dword_count;
 	u32			running_disparity_error_count;
Index: linux-2.6/drivers/scsi/scsi_transport_sas.c
===================================================================
--- linux-2.6.orig/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:49:35.000000000 +0200
+++ linux-2.6/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:50:55.000000000 +0200
@@ -266,6 +266,9 @@
 	struct sas_internal *i = to_sas_internal(shost->transportt);	\
 	int error;							\
 									\
+	if (!phy->local_attached)					\
+		return -EINVAL;						\
+									\
 	error = i->f->get_linkerrors(phy);				\
 	if (error)							\
 		return error;						\

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/19/05 14:01, Christoph Hellwig wrote:
> Add a flag to mark a PHY as attached to the HBA as opposed to beeing on
> an expander.  This is needed because various features are only supported
> on those.  This is a crude hack, the proper fix would be to use
> different classes for host-attached vs expander phys.  I'm looking into
> that.

The phy isn't quite "attached" -- i.e. at software level you do not
care about that.  At least I've never heard anyone (not of "the community")
say that a phy is "attached".

You don't need to represent that.  While you can, you completely do not
need to do it.  All you should care about is the _port_.  Take a look
at SAS section 4.

Also take a look at:
drivers/scsi/sas/sas_phy.c,
drivers/scsi/sas/sas_port.c and
drivers/scsi/sas/sas_discover.c .

Storage GUI applications can read the expander configuration using SMP
and show this as I've shown in Announcement 1 here:
http://marc.theaimsgroup.com/?l=linux-scsi&m=112629509826900&w=2

All you should care about is the _local_ phys since those are
under your control, and the port formed.

	Luben


> 
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> Index: linux-2.6/drivers/message/fusion/mptsas.c
> ===================================================================
> --- linux-2.6.orig/drivers/message/fusion/mptsas.c	2005-10-19 15:30:29.000000000 +0200
> +++ linux-2.6/drivers/message/fusion/mptsas.c	2005-10-19 19:49:35.000000000 +0200
> @@ -760,7 +760,7 @@
>  }
>  
>  static int mptsas_probe_one_phy(struct device *dev,
> -		struct mptsas_phyinfo *phy_info, int index)
> +		struct mptsas_phyinfo *phy_info, int index, int local)
>  {
>  	struct sas_phy *port;
>  	int error;
> @@ -853,6 +853,9 @@
>  		break;
>  	}
>  
> +	if (local)
> +		port->local_attached = 1;
> +
>  	error = sas_phy_add(port);
>  	if (error) {
>  		sas_phy_free(port);
> @@ -918,7 +921,7 @@
>  		}
>  
>  		mptsas_probe_one_phy(&ioc->sh->shost_gendev,
> -				     &port_info->phy_info[i], *index);
> +				     &port_info->phy_info[i], *index, 1);
>  		(*index)++;
>  	}
>  
> @@ -989,7 +992,8 @@
>  			}
>  		}
>  
> -		mptsas_probe_one_phy(parent, &port_info->phy_info[i], *index);
> +		mptsas_probe_one_phy(parent, &port_info->phy_info[i],
> +				     *index, 0);
>  		(*index)++;
>  	}
>  
> Index: linux-2.6/include/scsi/scsi_transport_sas.h
> ===================================================================
> --- linux-2.6.orig/include/scsi/scsi_transport_sas.h	2005-10-19 15:30:29.000000000 +0200
> +++ linux-2.6/include/scsi/scsi_transport_sas.h	2005-10-19 19:49:35.000000000 +0200
> @@ -56,6 +56,9 @@
>  	enum sas_linkrate	maximum_linkrate;
>  	u8			port_identifier;
>  
> +	/* internal state */
> +	unsigned int		local_attached : 1;
> +
>  	/* link error statistics */
>  	u32			invalid_dword_count;
>  	u32			running_disparity_error_count;
> Index: linux-2.6/drivers/scsi/scsi_transport_sas.c
> ===================================================================
> --- linux-2.6.orig/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:49:35.000000000 +0200
> +++ linux-2.6/drivers/scsi/scsi_transport_sas.c	2005-10-19 19:50:55.000000000 +0200
> @@ -266,6 +266,9 @@
>  	struct sas_internal *i = to_sas_internal(shost->transportt);	\
>  	int error;							\
>  									\
> +	if (!phy->local_attached)					\
> +		return -EINVAL;						\
> +									\
>  	error = i->f->get_linkerrors(phy);				\
>  	if (error)							\
>  		return error;						\
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/19/05 15:24, Luben Tuikov wrote:
> On 10/19/05 14:01, Christoph Hellwig wrote:
> 
>>Add a flag to mark a PHY as attached to the HBA as opposed to beeing on
>>an expander.  This is needed because various features are only supported
>>on those.  This is a crude hack, the proper fix would be to use
>>different classes for host-attached vs expander phys.  I'm looking into
>>that.
> 
> 
> The phy isn't quite "attached" -- i.e. at software level you do not
> care about that.  At least I've never heard anyone (not of "the community")
> say that a phy is "attached".
> 
> You don't need to represent that.  While you can, you completely do not
> need to do it.  All you should care about is the _port_.  Take a look
> at SAS section 4.
> 
> Also take a look at:
> drivers/scsi/sas/sas_phy.c,
> drivers/scsi/sas/sas_port.c and
> drivers/scsi/sas/sas_discover.c .
> 
> Storage GUI applications can read the expander configuration using SMP
> and show this as I've shown in Announcement 1 here:
> http://marc.theaimsgroup.com/?l=linux-scsi&m=112629509826900&w=2

And Announcement 2 (more specifically using the user space configuration
utility program "expander_conf.c"):
http://marc.theaimsgroup.com/?l=linux-scsi&m=112629509318354&w=2

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

 
> All you should care about is the _local_ phys since those are
> under your control, and the port formed.
> 
> 	Luben

RE: [PATCH 1/4] sas: add flag for locally attached PHYs

[Moore, Eric Dean]

On Wednesday, October 19, 2005 1:24 PM, Luben Tuikov wrote: 

> On 10/19/05 14:01, Christoph Hellwig wrote:
> > Add a flag to mark a PHY as attached to the HBA as opposed 
> to beeing on
> > an expander.  This is needed because various features are 
> only supported
> > on those.  This is a crude hack, the proper fix would be to use
> > different classes for host-attached vs expander phys.  I'm 
> looking into
> > that.
> 
> The phy isn't quite "attached" -- i.e. at software level you do not
> care about that.  At least I've never heard anyone (not of 
> "the community")
> say that a phy is "attached".
> 

Luben:  He is adding support for linkerrors and link/phy reset;
these for for CSMI/SDI API.

Can you in your driver retreive link errors for the links on the expanders?

Same question for sending link and phy resets for expander phys? 
If so, then perhaps that check should be removed, and lets let the lld
decide whether these attibutes should return -EINVAL.

For fusion, these new attributes only work for the phys on the hba.
The problem is the firmware implmentation for `link errors and 
link/phy reset` is addressed only by the phy identifier for the phys
attacted directly on the host adapter.  Doesn't work for phys on
the expander. 

> You don't need to represent that.  While you can, you 
> completely do not
> need to do it.  All you should care about is the _port_.  Take a look
> at SAS section 4.
> 
> Also take a look at:
> drivers/scsi/sas/sas_phy.c,
> drivers/scsi/sas/sas_port.c and
> drivers/scsi/sas/sas_discover.c .

What and where can I get this?

Best regards,
Eric Moore

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/19/05 16:08, Moore, Eric Dean wrote:
> Luben:  He is adding support for linkerrors and link/phy reset;
> these for for CSMI/SDI API.
> 
> Can you in your driver retreive link errors for the links on the expanders?

Yes. (offloaded to user space)

You use the user space program called "expander_conf.c" to
send SMP requests and receive SMP responses.

An example output of "expander_conf.c" for a domain
is shown here: (everything you want to know about expanders):

http://linux.adaptec.com/sas/Announce_2.txt

The domain of the above is shown here:

http://linux.adaptec.com/sas/Announce_1.txt

> Same question for sending link and phy resets for expander phys? 

Yes, for both local phys and domain device phys, any device
claiming an SMP port.

Local phys
----------

If the LLDD provides this, it fills out the function stub:

	/* Phy management */
	int (*lldd_control_phy)(struct sas_phy *, enum phy_func);

in the "struct sas_ha_struct" structure before it registers
with the SAS Transport Layer.  See include/scsi/sas/sas_class.h
drivers/scsi/aic94xx/aic94xx_init.c::asd_register_sas_ha(), and
drivers/scsi/sas/README.

(When the SAS LLDD (aic94xx) registers with the SAS Transport
 layer, it declares all the phys the HA has; this is in
 struct sas_ha_struct::struct sas_phy  **sas_phy;  This is so
 that the SAS Transport layer can control the phys to form
 ports, etc, as stipulated in
 http://linux.adaptec.com/sas/Announce_0.txt.)

Then you do:

	sas_ha->lldd_control_phy(phy, func);

where,

	struct sas_ha_struct *sas_ha;
	struct sas_phy *phy;
	enum   phy_func func;

See
include/scsi/sas/sas.h and
include/scsi/sas/sas_class.h

Phys on Domain devices with an SMP port
---------------------------------------

Execute SCSI Command RPC is used to send
SMP tasks:

This is from drivers/scsi/sas/sas_expander.c:

<< line 480 >>
#define PC_REQ_SIZE  44
#define PC_RESP_SIZE 8

static int smp_phy_control(struct domain_device *dev, int phy_id,
			   enum phy_func phy_func)
{
	u8 *pc_req;
	u8 *pc_resp;
	int res;

	pc_req = alloc_smp_req(PC_REQ_SIZE);
	if (!pc_req)
		return -ENOMEM;

	pc_resp = alloc_smp_resp(PC_RESP_SIZE);
	if (!pc_resp) {
		kfree(pc_req);
		return -ENOMEM;
	}

	pc_req[1] = SMP_PHY_CONTROL;
	pc_req[9] = phy_id;
	pc_req[10]= phy_func;

	res = smp_execute_task(dev, pc_req, PC_REQ_SIZE, pc_resp,PC_RESP_SIZE);

	kfree(pc_resp);
	kfree(pc_req);
	return res;
}

Then you can look up the implementation of

static int smp_execute_task(struct domain_device *dev, void *req, int req_size,
			    void *resp, int resp_size)

on line 175 of drivers/scsi/sas/sas_expander.c, and you can follow
it up to
	task->dev->port->ha->lldd_execute_task(task, 1, GFP_KERNEL);

where lldd_execute_task() is implemented in
drivers/scsi/aic94xx/aic94xx_task.c, on line 555:

int asd_execute_task(struct sas_task *task, const int num,
		     unsigned long gfp_flags)

which immediately sends the task out on the interconnect.

(Nugget: "dev->port" doesn't _necessarily_ mean that
 "dev" /formed/ "port", but that "dev" is reachable
 /through/ "port". "dev" may or may not form "port".)

> If so, then perhaps that check should be removed, and lets let the lld
> decide whether these attibutes should return -EINVAL.

In the SAS Transport Layer, all this has been off loaded to user space.
See drivers/scsi/sas/README "Expander management from User Space"
section (bottom of file).

> For fusion, these new attributes only work for the phys on the hba.

For aic94xx, these attributes are supported for any SMP port,
i.e. local phys and any phys out there which claim SMP.

> The problem is the firmware implmentation for `link errors and 
> link/phy reset` is addressed only by the phy identifier for the phys
> attacted directly on the host adapter.  Doesn't work for phys on
> the expander. 

What you need is a general facility to send SMP tasks.

See drivers/scsi/sas/sas_expander.c,
    include/scsi/sas/sas_task.h      and
    drivers/scsi/aic94xx/aic94xx_task.c

>>You don't need to represent that.  While you can, you 
>>completely do not
>>need to do it.  All you should care about is the _port_.  Take a look
>>at SAS section 4.
>>
>>Also take a look at:
>>drivers/scsi/sas/sas_phy.c,
>>drivers/scsi/sas/sas_port.c and
>>drivers/scsi/sas/sas_discover.c .
> 
> 
> What and where can I get this?

http://linux.adaptec.com/sas/
Also see my sig.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

On Wed, Oct 19, 2005 at 05:00:03PM -0400, Luben Tuikov wrote:
> On 10/19/05 16:08, Moore, Eric Dean wrote:
> > Luben:  He is adding support for linkerrors and link/phy reset;
> > these for for CSMI/SDI API.
> > 
> > Can you in your driver retreive link errors for the links on the expanders?
> 
> Yes. (offloaded to user space)
> 
> You use the user space program called "expander_conf.c" to
> send SMP requests and receive SMP responses.

Yes, exactly.  On the other hand we don't have standard
hardware-indepent ways of doing that for the PHYs of the HBA because
the programming interfaces are to different and need to abstract it away
in the kernel.  That's why I need the local_attached flag as an internal
helper - it's not actually exposed to userland so the implementation

RE: [PATCH 1/4] sas: add flag for locally attached PHYs

[Moore, Eric Dean]

On Wednesday, October 19, 2005 3:00 PM, Luben Tuikov wrote: 

> 
> On 10/19/05 16:08, Moore, Eric Dean wrote:
> > Luben:  He is adding support for linkerrors and link/phy reset;
> > these for for CSMI/SDI API.
> > 
> > Can you in your driver retreive link errors for the links 
> on the expanders?
> 
> Yes. (offloaded to user space)
> 

I had a very good discussion with Luben over the 
phone last night.

Luben clarified this for me in regards to expanders:

http://www.t10.org/drafts.htm#sas2 
The linkerrors can be done by issuing a SMP passthru
sending SMP_REPORT_ERROR_LOG. Section 10.4.3.6

The link/phy reset can be done by issuing a SMP passthru
sending SMP_PHY_CONTROL. Section 10.4.3.10

The mpt fusion f/w supports smp passthru, and we could for
these attributes do SMP passthru if we wanted to return
this data for expanders.  Our f/w only provides a easy special 
wrapper config page for hba phys. For expanders, we
are going to have to do smp passthru.

So Christoph what should we do in driver?  Ignore returning
data for expanders?  

Luben suggestion(correct me if I'm wrong) is this should be 
done thru user space, instead of exporting this in the 
/sys/class/sas_phy attributes.  Thus we need to export a 
SMP_PASSTHRU mechanism.  I believe there is discussion
by Christoph ( and I believe Luben backs this idea) of doing 
all this thru the /dev/sg interface.


> 
> > What and where can I get this?
> 
> http://linux.adaptec.com/sas/
> Also see my sig.

Thanks. I will check it out.

Eric Moore

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 10:11, Christoph Hellwig wrote:
> On Wed, Oct 19, 2005 at 05:00:03PM -0400, Luben Tuikov wrote:
> 
>>On 10/19/05 16:08, Moore, Eric Dean wrote:
>>
>>>Luben:  He is adding support for linkerrors and link/phy reset;
>>>these for for CSMI/SDI API.
>>>
>>>Can you in your driver retreive link errors for the links on the expanders?
>>
>>Yes. (offloaded to user space)
>>
>>You use the user space program called "expander_conf.c" to
>>send SMP requests and receive SMP responses.
> 
> 
> Yes, exactly.  On the other hand we don't have standard
> hardware-indepent ways of doing that for the PHYs of the HBA because
> the programming interfaces are to different and need to abstract it away
> in the kernel.

The _protocol_ way is SMP.  And LSI does implement SMP passthrough.

You have two scenarios: local phys and SMP.

Note that your task as wanted by HP/LSI/etc (all who use Fusion MPT)
is to implement SDI/CSMI, see http://www.t10.org/drafts.htm#sdi- .

If you implement this, as say, drivers/scsi/sdi.c, then I'd
be more than happy to register with it so as to export
SDI/CSMI functionality.

E.g. if you look at SDI_GET_PHY_INFO you will notice
that this is only for phys of the Host Adapter, NOT all
phys on the domain (which is a _crazy_ concept).

Most of the code has been written right in the SDI spec
sdi-r00.pdf.  You can start from there.  And most of the concepts
have been implemented in say LSI's FW and in my code, the SAS
Transport Layer.

E.g. you can see how "port identifier is u8" which is exactly
what, I'm sure, LSI's fw uses (or can fit in), and as you can see
in my code that I use (I actually use "int").

Now, _registration_ of SDI by different entities would be common,
say, sdi_register_provider(struct ...); Also the _mechanism_
to access that facility would be common*.  But _dispatching_
would be implementation dependent as you can see LSI would
do it differently and Adaptec would do it differently and
XYZ would do it differently, etc.

* I say common because you can use sg, in which case the
IOCTLs described in the spec are directly translated into
"headers" to be issued down sg.

Your task is clearly defined by a spec (sdi-r00.pdf).
Most of the functions defined therein are direct SAS concepts
and, as you can see, translate to an SMP function, or to a TMF,
since this is where they were taken from to begin with.

There is no need for translation layers upon translation
layers.  The task is clearly defined: LSI wants SDI/CSMI.
You have a CSMI spec.  Implement it.  This would make
everyone's job so much easier without destroying already
existing _good_ code.

>  That's why I need the local_attached flag as an internal
> helper - it's not actually exposed to userland so the implementation

1) LSI does have SMP pass through.
2) You absolutely do not need to represent all phys on the domain,
which is, as I said, a completely _crazy_ idea.  Look, even SDI doesn't
want you to do that.  What SDI wants you to do is issue SMP out.
LSI does have SMP pass through in FW so use that.
3) All you care about is SDI, and SDI has a _spec_.  Your job is
twofold:
	- implement drivers/scsi/sdi.c so we can all use it,
	- implement the implementation/hw dependent way to
	access SDI functionality for Fusion as described above
	where I mention that this would be implementation dependent
	(dispatching to).

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

RE: [PATCH 1/4] sas: add flag for locally attached PHYs

[Moore, Eric Dean]

On Thursday, October 20, 2005 9:29 AM,  Luben Tuikov wrote:
 
> >  That's why I need the local_attached flag as an internal
> > helper - it's not actually exposed to userland so the implementation
> 
> 1) LSI does have SMP pass through.
> 2) You absolutely do not need to represent all phys on the domain,
> which is, as I said, a completely _crazy_ idea.  Look, even 
> SDI doesn't
> want you to do that.  What SDI wants you to do is issue SMP out.
> LSI does have SMP pass through in FW so use that.
> 3) All you care about is SDI, and SDI has a _spec_.  Your job is
> twofold:
> 	- implement drivers/scsi/sdi.c so we can all use it,
> 	- implement the implementation/hw dependent way to
> 	access SDI functionality for Fusion as described above
> 	where I mention that this would be implementation dependent
> 	(dispatching to).
> 

Thanks Luben - My concern is converting our SDI/CSMI ioctls
into what is acceptable in mainstream kernel.  Agreed with Luben,
all the expander data gathering, and configuration can be done 
in user space, which is what is being done by the HP application
folks which are currently using our internal CSMI enabled drivers.
 
I added expander support a couple rev's back since none of the 
non-direct attached sas/sata devices were not scanned and reported
to the block layer above.  This was a problem when our sas driver 
was converted to work with sas_transport.

Best Regards,
Eric Moore

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

<lots of useless rants snipped>

Yes, we have two cases, local phy and phy on an expander.  The proper
way to support remote phys is via smp passthru as we hopefully agree.
Right now the sas transport class doesn't do smp passthru (although
I'm working on it) so we only implement local link stats.  Once we have
smp passthru we can support link stats, in one of two ways:

 - do it in userland as in your example program.  in that case we don't
   need to do anything in kernel land and need the local_attached flag
   to disable this functionality for expander phys.
 - do it in kernel, in which case the transport class must call the smp
   passthru entry point of the driver to query link stats, so it needs
   the local_attached flag aswell for that.

What's your problem with the flag now?

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 11:25, Moore, Eric Dean wrote:
> 
> I had a very good discussion with Luben over the 
> phone last night.
> 
> Luben clarified this for me in regards to expanders:
> 
> http://www.t10.org/drafts.htm#sas2 
> The linkerrors can be done by issuing a SMP passthru
> sending SMP_REPORT_ERROR_LOG. Section 10.4.3.6

Example of it being used is shown in this user space program
drivers/scsi/sas/expanders_conf.c:343:
static int report_phy_error_log(char *smp_portal_name, int phy_id)

> The link/phy reset can be done by issuing a SMP passthru
> sending SMP_PHY_CONTROL. Section 10.4.3.10

Example of it being use is shown in
drivers/scsi/sas/sas_expander.c:484:
static int smp_phy_control(struct domain_device *dev, int phy_id,
			   enum phy_func phy_func)

> The mpt fusion f/w supports smp passthru, and we could for
> these attributes do SMP passthru if we wanted to return
> this data for expanders.

> Our f/w only provides a easy special 
> wrapper config page for hba phys.

And this is correct in in line with SDI/SAS.

This method can be used by drivers/scsi/sdi.c.
And it fits both yours and our simplementation.

Dispatching to your implementation would go to FW, whose
interface you'll have to implement.  Remember you have
2 tasks: drivers/scsi/sdi.c and the FW dependent way
of dispatching this to the FW for LSI.

As to ours, it would go to

	/* Phy management */
	int (*lldd_control_phy)(struct sas_phy *, enum phy_func);

function stub in the of the sas_ha_struct structure as I showed
yesterday in this thread.

> For expanders, we
> are going to have to do smp passthru.

Makes sense.

> So Christoph what should we do in driver?  Ignore returning
> data for expanders?  
> 
> Luben suggestion(correct me if I'm wrong) is this should be 
> done thru user space, instead of exporting this in the 
> /sys/class/sas_phy attributes.  Thus we need to export a 
> SMP_PASSTHRU mechanism.  I believe there is discussion
> by Christoph ( and I believe Luben backs this idea) of doing 
> all this thru the /dev/sg interface.

sg would be a good choice if the ioctl headers can be "translated"
easily into sg headers -- this would make the whole task
so much easier and straightforward.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

On Thu, Oct 20, 2005 at 09:25:54AM -0600, Moore, Eric Dean wrote:
> So Christoph what should we do in driver?  Ignore returning
> data for expanders?  

As you mentioned doing link reset and reporting stats on expanders
is done using SMP, and thus totally independent of the driver used.
It should be either implemented in the transport class or in userland.
Doing it in the transport class has the advantage of a unified interface
for hba phys and expander phys, doing it in userland has the advantage
of less kernel bloat.

> Luben suggestion(correct me if I'm wrong) is this should be 
> done thru user space, instead of exporting this in the 
> /sys/class/sas_phy attributes.  Thus we need to export a 
> SMP_PASSTHRU mechanism.  I believe there is discussion
> by Christoph ( and I believe Luben backs this idea) of doing 
> all this thru the /dev/sg interface.

I've done a prototype using an SG_IO interface, but I'm not totally
happy about.  I've also seen lubens interface and I'm not totally
happy about it either.  Getting the interface right for SMP passthru
is very hard.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 11:49, Christoph Hellwig wrote:
> <lots of useless rants snipped>

Hi Christoph,

For some reason I thought that I was quoting specs
and showing constructive ideas (i.e. things you can sit down
and start coding).

I'm sorry that you feel that it is "useless rants".

On a topic note: It appears that the "sas transport attributes" are
nothing more than SDI/CSMI.

> Yes, we have two cases, local phy and phy on an expander.  The proper
> way to support remote phys is via smp passthru as we hopefully agree.
> Right now the sas transport class doesn't do smp passthru (although
> I'm working on it) so we only implement local link stats.

You are so much better off implementing SDI/CSMI as per the spec.

Normally, at companies, projects have a "specificaion" and "required
functionality".  What is the "specificaion" and "required functionality"
at hand?

Note that there are LSI/HP customers who are completely _happy_
to use CSMI as provided by LSI and they are not interested in 
what-have-you-not.

The easiest way to make those customers happy and keep them happy is
to provide general SDI/CSMI facitliy and _you have a spec on that_.

> Once we have
> smp passthru we can support link stats, in one of two ways:
> 
>  - do it in userland as in your example program.  in that case we don't
>    need to do anything in kernel land and need the local_attached flag
>    to disable this functionality for expander phys.
>  - do it in kernel, in which case the transport class must call the smp
>    passthru entry point of the driver to query link stats, so it needs
>    the local_attached flag aswell for that.
> 
> What's your problem with the flag now?

It is ugly, it is a (brown) paper bag solution, it is a work around.

Take a look at the phy attributes as per the SAS spec.  Then take
a look at struct sas_phy in include/scsi/sas/sas_class.h.  Do you
see such a flag in either reference?

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 11:45, Moore, Eric Dean wrote:
> 
> Thanks Luben - My concern is converting our SDI/CSMI ioctls
> into what is acceptable in mainstream kernel.  Agreed with Luben,
> all the expander data gathering, and configuration can be done 
> in user space, which is what is being done by the HP application
> folks which are currently using our internal CSMI enabled drivers.

Well, this is very good news.  This means less deployment time,
less code which needs to be changed, less code which needs
to go through testing and faster customer reach.

This should suggest Christoph "which way it should be done."

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 12:01, Christoph Hellwig wrote:
> 
> As you mentioned doing link reset and reporting stats on expanders
> is done using SMP, and thus totally independent of the driver used.

Not quite. "Dispatching to" would be driver dependent.  If you implement
sdi_register_provider(struct ...); then you can make everyone happy
in the _shortest_ amount of time.

I'd suggest a reading of the SDI, SAS and SAM specs for an insight
of how it all ties together.

> It should be either implemented in the transport class or in userland.
> Doing it in the transport class has the advantage of a unified interface
> for hba phys and expander phys, doing it in userland has the advantage
> of less kernel bloat.

You already know the answer to that.

"customers" and "customer satisfaction" is a foreign
concept with the community, being the nature of Linux as it is,
but what you should want to is:
	- make less applications (which you may not know of)
	  break,
	- make less assumptions already taken break.

Those applications are already using SDI and only the interface
would change ("/dev/sg") if you were to implement
drivers/scsi/sdi.c.

> I've done a prototype using an SG_IO interface, but I'm not totally
> happy about.

Care to say why?

> I've also seen lubens interface and I'm not totally
> happy about it either.

Care to say why?

> Getting the interface right for SMP passthru is very hard.

1) You have the SDI spec which _tells you_ how to do it.
2) There are (HP) applications which already use that (1).

You have to stay focused on _what is required_.

Why are you trying to reinvent "SMP passthrough"?
Why not just make the customers happy?

If you implement drivers/scsi/sdi.c, as:

int  sdi_register_provider(struct sdi_provider_struct *sdi_prov);
void sdi_unregister_provider(struct sdi_provider_struct *sdi_prov);

everyone will be _more than happy_ to plug right into it.
And those are the only 2 functions you need to EXPORT_SYMBOL_GPL.

Everything else (SDI_ functions) would be in the struct as stubs
and the provider would set them if they implement them.

LSI/HP can plug right into it with *minimum of overhead*,
testing etc, both from their application side and from kernel
side.  And this is important to both customers and companies.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Christoph Hellwig]

On Thu, Oct 20, 2005 at 12:51:15PM -0400, Luben Tuikov wrote:
> On 10/20/05 12:01, Christoph Hellwig wrote:
> > 
> > As you mentioned doing link reset and reporting stats on expanders
> > is done using SMP, and thus totally independent of the driver used.
> 
> Not quite. "Dispatching to" would be driver dependent.  If you implement
> sdi_register_provider(struct ...); then you can make everyone happy
> in the _shortest_ amount of time.
> 
> I'd suggest a reading of the SDI, SAS and SAM specs for an insight
> of how it all ties together.

Thanks Luben.  You might not be able to imagine it, but there are people
other than you who are able to read specs, and despite having read them
they can actually talk like a normal human without referencing them in
every second sentence.  That beeing said SDI is not a final
specification.  It's a draft that's probablt not going to make it as a
t10 standard because the main pushers have backed of a bit again.

We are not going to implement SDI in the kernel.  Long before SDI or
it's predecessor, HP CSMI were designed we made it clear we're not going
to accept wide ioctl-APIs, especially when they're even bad for old
ioctl API standards.  The CSMI spec has been passed around in an early
phase and been totally rejected, I think even publically on linux-scsi.
HP decided to move ahead despite that and did a huge mis-services to
their customers.  It's not my problem if big companies can't listen and
do things their customers have to pay for, and it's certainly not our
job to work around their idiocy.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

> We are not going to implement SDI in the kernel.  Long before SDI or
> it's predecessor, HP CSMI were designed we made it clear we're not going
> to accept wide ioctl-APIs, especially when they're even bad for old
> ioctl API standards.  The CSMI spec has been passed around in an early
> phase and been totally rejected, 

Just to say AYE on this:

the problem with these things is that they define an IOCTL interface,
while a library level interface would have been a LOT better. By far.
That way the spec doesn't define the implementation, but only a real
INTERFACE while allowing flexibility and compatibility. IOCTLs are just
very very much the wrong layer to put such an interface. (just as
posix/sus specify the (g)libc interface and not the kernel interface,
although of course they impact it somewhat and glibc can for several
things do a straight pass-through to the raw kernel interface).

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 13:03, Christoph Hellwig wrote:
>>I'd suggest a reading of the SDI, SAS and SAM specs for an insight
>>of how it all ties together.
> 
> 
> Thanks Luben.  You might not be able to imagine it, but there are people
> other than you who are able to read specs, and despite having read them
> they can actually talk like a normal human without referencing them in
> every second sentence.  That beeing said SDI is not a final
> specification.  It's a draft that's probablt not going to make it as a
> t10 standard because the main pushers have backed of a bit again.

The whole point is that LSI/HP/Adaptec are _using_ SDI in one form
or another and would like to have it officially in the kernel.

> We are not going to implement SDI in the kernel.  Long before SDI or

Who is "we"?

> it's predecessor, HP CSMI were designed we made it clear we're not going
> to accept wide ioctl-APIs, especially when they're even bad for old

You do not have to take section 4 of SDI literally, but what everyone
wants is the _functionality_ of section 5 of SDI.  That can be implemented
as a backend, see my next message to Arjan on this thread, and the front
end can be anything at all (sg/sysfs/whatever1/whatever2).

> ioctl API standards.  The CSMI spec has been passed around in an early
> phase and been totally rejected, I think even publically on linux-scsi.

Rejected by whom?  "The community" or by you?

Let me understand this properly: LSI/HP/Adaptec are using SDI in one
form or another and would like to see something like it in the kernel,
but "the community", the "we" you refer to above, rejects it?

> HP decided to move ahead despite that and did a huge mis-services to
> their customers.

Yes, because there is a _need_ for it.

> It's not my problem if big companies can't listen and do things their
> customers have to pay for, and it's certainly not our job to work
> around their idiocy.

Bold statment.

Who should "big companies" listen to?  You?  "The community?"
Are you saying "big companies" whould listen to Linux which
says "no to specs" among other things?

Often enough what "big companies" agree on and use and deploy is
what Linux (you?) should _listen_ to, try to understand and maybe
get out of the way.

It is all about customer satisfaction, which is completely
foreign to Linux, simply because of the _nature_ of Linux.

Why is there a need to _reinvent_ things?
Aren't there engineers at those "big companies" who know
OS design and programming?

Why do you feel the need to antiquate engineers from
companies submitting drivers/patches/solutions to the
Linux kernel?

Surely this isn't good for the Linux kernel in the long term.
Or is it?

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 13:22, Arjan van de Ven wrote:
>>We are not going to implement SDI in the kernel.  Long before SDI or
>>it's predecessor, HP CSMI were designed we made it clear we're not going
>>to accept wide ioctl-APIs, especially when they're even bad for old
>>ioctl API standards.  The CSMI spec has been passed around in an early
>>phase and been totally rejected, 
> 
> 
> Just to say AYE on this:
> 
> the problem with these things is that they define an IOCTL interface,
> while a library level interface would have been a LOT better. By far.
> That way the spec doesn't define the implementation, but only a real
> INTERFACE while allowing flexibility and compatibility. IOCTLs are just
> very very much the wrong layer to put such an interface. (just as
> posix/sus specify the (g)libc interface and not the kernel interface,
> although of course they impact it somewhat and glibc can for several
> things do a straight pass-through to the raw kernel interface).

Hi Arjan, how are you?

As I mentioned already, you do not have to take section 4 of SDI
literally.  If you implement section 5 of SDI, the back end, then
the front end is ajustable: sg/sysfs/whathaveyou1/whathaveyou2, etc.

Here is an excerpt sent by me to another storage engineer:
--cut--
Example of include/scsi/sdi/sdi.h (back-end):

/* This struct describes section 5 of the SDI spec */
struct sdi_functions_struct {
	int (*sdi_get_driver_info)(struct sdi_header *header,
				   struct sdi_driver_info *info);
	int (*sdi_get_controller_config)(struct sdi_header *header,
					 struct sdi_controller_config *config);
	/* the rest of the stub SDI functions defined here */
	...
};
	
/* This struct describes the provider */
struct sdi_provider_struct {
	char *provider_name; /* some name, whatever, may not be necessary */
	...
	struct sdi_functions_struct sdi_functions;   /* see above */
	...
};

And then the _calling_ of those functions is either via sg or sysfs
or whatever makes people happy (done in drivers/scsi/sdi.c).

Dispatching of those is driver/fw/hw/whatever dependent depending on
the provider and what they registered.

This will live in drivers/scsi/sdi.c and include/scsi/sdi/sdi.h
--cut--

You see?  You don't need IOCTLs, yet you can be fully compliant
to the spec so that already existing user space management programs
can transition with minimal change to their code (over to the front
end: sg/whatever).  This cuts down production costs in testing,
verification/etc.  Makes things cheaper to deploy and customers
are happy.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Andrew Patterson]

On Thu, 2005-10-20 at 16:02 -0400, Luben Tuikov wrote:
> On 10/20/05 13:03, Christoph Hellwig wrote:
> >>I'd suggest a reading of the SDI, SAS and SAM specs for an insight
> >>of how it all ties together.
> > 
> > 
> > Thanks Luben.  You might not be able to imagine it, but there are people
> > other than you who are able to read specs, and despite having read them
> > they can actually talk like a normal human without referencing them in
> > every second sentence.  That beeing said SDI is not a final
> > specification.  It's a draft that's probablt not going to make it as a
> > t10 standard because the main pushers have backed of a bit again.
> 
> The whole point is that LSI/HP/Adaptec are _using_ SDI in one form
> or another and would like to have it officially in the kernel.

I would hope that other users/vendors might find a use for this as well.

> 
> > We are not going to implement SDI in the kernel.  Long before SDI or
> 
> Who is "we"?
> 
> > it's predecessor, HP CSMI were designed we made it clear we're not going
> > to accept wide ioctl-APIs, especially when they're even bad for old
> 

CSMI is a semi-proprietary spec.  SDI is an attempt to turn this into an
open spec.  One of the reasons it has not been made into a standard is
the whole IOCTL issue.  Other OS's, Windows, HP-UX, Solaris allow them,
Linux has recently banned them (at least in the SCSI sub-system). 

> You do not have to take section 4 of SDI literally, but what everyone
> wants is the _functionality_ of section 5 of SDI. 

My guess is that what current app writers want is to use IOCTL's so they
don't have to special-case Linux.  Next best thing would be something
that closely approaches it, to avoid re-writing a lot of code.  

>  That can be implemented
> as a backend, see my next message to Arjan on this thread, and the front
> end can be anything at all (sg/sysfs/whatever1/whatever2).
> 
> > ioctl API standards.  The CSMI spec has been passed around in an early
> > phase and been totally rejected, I think even publically on linux-scsi.

The LSI driver attempted go in with CSMI. One of the reasons it was
rejected was because it used IOCTL's to implement CSMI.  I don't think
CSMI itself was rejected (other than the fact that is required IOCTL's
of course).  The SDI authors are willing to use another type of
interface, but have had no help in suggesting alternatives.

> 
> Rejected by whom?  "The community" or by you?

I believe there is a common understanding that IOCTL's are bad and
should be avoided. See:

http://lkml.org/lkml/2001/5/20/81

> 
> Let me understand this properly: LSI/HP/Adaptec are using SDI in one
> form or another and would like to see something like it in the kernel,
> but "the community", the "we" you refer to above, rejects it?
> 
> > HP decided to move ahead despite that and did a huge mis-services to
> > their customers.

Perhaps.  Given that there seems to be no alternative, we don't have
much of a choice.

> 
> Yes, because there is a _need_ for it.
> 
> > It's not my problem if big companies can't listen and do things their
> > customers have to pay for, and it's certainly not our job to work
> > around their idiocy.

Yes, CSMI should have had more Linux input when it was developed.  The
no-new IOCTL policy certainly came as a surprise to the authors. Still,
there doesn't seem to be any other usable cross-platform interface that
is acceptable to the linux community (or at least to Christoph)'s corner
of it).  My personal preference is to hide this stuff in a library, so
the kernel implementation is hidden. But even a library needs an
underlying kernel implementation.

> 
> Bold statment.
> 
> Who should "big companies" listen to?  You?  "The community?"
> Are you saying "big companies" whould listen to Linux which
> says "no to specs" among other things?
> 
> Often enough what "big companies" agree on and use and deploy is
> what Linux (you?) should _listen_ to, try to understand and maybe
> get out of the way.
> 

Big companies often want to do things in a proprietary fashion.  I
personally would prefer to see a standard's-based approach.

> It is all about customer satisfaction, which is completely
> foreign to Linux, simply because of the _nature_ of Linux.

More bold statements? ;-)

> 
> Why is there a need to _reinvent_ things?
> Aren't there engineers at those "big companies" who know
> OS design and programming?
> 
> Why do you feel the need to antiquate engineers from
> companies submitting drivers/patches/solutions to the
> Linux kernel?
> 
> Surely this isn't good for the Linux kernel in the long term.
> Or is it?
> 
> 	Luben

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Douglas Gilbert]

Moore, Eric Dean wrote:
> On Wednesday, October 19, 2005 3:00 PM, Luben Tuikov wrote: 
> 
> 
>>On 10/19/05 16:08, Moore, Eric Dean wrote:

<snip>

> Luben suggestion(correct me if I'm wrong) is this should be 
> done thru user space, instead of exporting this in the 
> /sys/class/sas_phy attributes.  Thus we need to export a 
> SMP_PASSTHRU mechanism.  I believe there is discussion
> by Christoph ( and I believe Luben backs this idea) of doing 
> all this thru the /dev/sg interface.

Folks,
Just a suggestion if you want to bring /dev/sg into
the picture for a SMP (and even a STP) passthrough.
The host could define a lun (pick a number) and the
LLDD could supply the target code for it (as scsi_debug
does for example).

Details need to be sorted out (e.g. pdt of "host" lun)
and a way to pass an 8 byte SAS address through a
SCSI cdb (there are lots of vendor specific SCSI
opcodes available). For a STP pass through, we
have the SAT ATA PASS THROUGH SCSI commands already
defined (although they don't allow for an 8
byte target (SAS) address).

SG_IO code paths through the SCSI subsystem are pretty
well tested and are available in all lk 2.4 and 2.6
kernels. All OSes that I am aware of have some sort
of SCSI pass through mechanism so this approach bypasses
the objection to adding new ioctls (from both Microsoft
and Linux).

The idea is not original. Windows 2000 has a lun for
just this purpose (and it was dropped in Windows 2003).

Doug Gilbert

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Jeff Garzik]

Douglas Gilbert wrote:
> Just a suggestion if you want to bring /dev/sg into
> the picture for a SMP (and even a STP) passthrough.

Yep.  That's another possibility, that falls into the 'block driver' 
category [see list of categories in my 'ioctl' email], since it is 
translated into a 'special' block request, and then sent to a normal 
request_queue.

Jens Axboe's bsg[1] has also been mentioned in the context of SMP.

	Jeff


[1] http://lwn.net/Articles/96547/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Douglas Gilbert]

Jeff Garzik wrote:
> Douglas Gilbert wrote:
> 
>> Just a suggestion if you want to bring /dev/sg into
>> the picture for a SMP (and even a STP) passthrough.
> 
> 
> Yep.  That's another possibility, that falls into the 'block driver'
> category [see list of categories in my 'ioctl' email], since it is
> translated into a 'special' block request, and then sent to a normal
> request_queue.
> 
> Jens Axboe's bsg[1] has also been mentioned in the context of SMP.

Jeff,
Yes, I like the bsg model: one well known device (/dev/bsg),
get a file descriptor and then bind to a "target". It
reminded me of a network interface.

As presented, it only bound to a block (or char?) device
node (e.g. /dev/sda). I asked some questions about widening
the variety of things it could bind to but Peter Jones
suggested that binding was a necessary evil that should
not be encouraged.

So perhaps we could bind to a SAS host ... (and a SCSI device
for those pesky things like safte, ses and wluns).

Doug Gilbert

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

On Thu, 2005-10-20 at 16:10 -0400, Luben Tuikov wrote:
> You see?  You don't need IOCTLs, yet you can be fully compliant
> to the spec so that already existing user space management programs
> can transition with minimal change to their code (over to the front
> end: sg/whatever). 

this is EXACTLY my point.  A proper API would NOT need the userland apps
to change if the underlying mechanism changes. Or in other words: today
it may be SG, tomorrow BSG or SysFS, and when we open 2.7 the mechanism
may be WokkieWokkie. The implementation of the interface library may
need to change for this, but the management app should not. 

There is a parallel in the glibc/posix/sus world: With the 2.6 kernel
there is a new syscall mechanism (sysenter) on x86 kernels available.
NONE of the applications needed changing, fopen() remained fopen(), not
just as API but also as ABI. All that needed some adjustments were glibc
internals.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

On Thu, 2005-10-20 at 18:01 -0600, Andrew Patterson wrote:

> Yes, CSMI should have had more Linux input when it was developed.  The
> no-new IOCTL policy certainly came as a surprise to the authors. Still,
> there doesn't seem to be any other usable cross-platform interface that
> is acceptable to the linux community (or at least to Christoph)'s corner
> of it).  My personal preference is to hide this stuff in a library, so
> the kernel implementation is hidden. But even a library needs an
> underlying kernel implementation.


but why didn't CSMI specify the library interface then? If it did that,
then none of the "ioctls are deprecated for 3 years now" would have come
as a surprise. Or rather, it maybe would have, but it wouldn't have
mattered to the management app.

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Andrew Patterson]

On Fri, 2005-10-21 at 11:06 +0200, Arjan van de Ven wrote:
> On Thu, 2005-10-20 at 18:01 -0600, Andrew Patterson wrote:
> 
> > Yes, CSMI should have had more Linux input when it was developed.  The
> > no-new IOCTL policy certainly came as a surprise to the authors. Still,
> > there doesn't seem to be any other usable cross-platform interface that
> > is acceptable to the linux community (or at least to Christoph)'s corner
> > of it).  My personal preference is to hide this stuff in a library, so
> > the kernel implementation is hidden. But even a library needs an
> > underlying kernel implementation.
> 
> 
> but why didn't CSMI specify the library interface then? If it did that,
> then none of the "ioctls are deprecated for 3 years now" would have come
> as a surprise. Or rather, it maybe would have, but it wouldn't have
> mattered to the management app.
> 

I asked one of the authors, Steve Fairchild, about this.  He said that
they had had a bad experience with HBAAPI, the FC library.  They had a
lot of problems with vendors keeping the library up to date, multiple
vendors shipping multiple incompatible versions, etc.  I wasn't entirely
convinced, since they seem to have just replaced one set of problems for
another.

Andrew

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Arjan van de Ven]

> > 
> > but why didn't CSMI specify the library interface then? If it did that,
> > then none of the "ioctls are deprecated for 3 years now" would have come
> > as a surprise. Or rather, it maybe would have, but it wouldn't have
> > mattered to the management app.
> > 
> 
> I asked one of the authors, Steve Fairchild, about this.  He said that
> they had had a bad experience with HBAAPI, the FC library.


>   They had a
> lot of problems with vendors keeping the library up to date

that's not too hard (and up to date is relative)

> , multiple
> vendors shipping multiple incompatible versions, etc. 

which sounds like they didn't get the spec right.. if the spec was good
that couldn't happen at all...

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 20:01, Andrew Patterson wrote:
> 
> My guess is that what current app writers want is to use IOCTL's so they
> don't have to special-case Linux.  Next best thing would be something
> that closely approaches it, to avoid re-writing a lot of code.  

"to avoid re-writing a lot of code." is also what I'm driving at.
I'm sure HP as well as LSI have a lot of user space programs using
the SDI interface.

>>Rejected by whom?  "The community" or by you?
> 
> 
> I believe there is a common understanding that IOCTL's are bad and
> should be avoided. See:
> 
> http://lkml.org/lkml/2001/5/20/81

I wasn't talking about IOCTLs.  I was talking in general about the
SDI spec.  As you can see for yourself, I suggested an SDI interface
which is agnostic to the front end.

>>>HP decided to move ahead despite that and did a huge mis-services to
>>>their customers.
> 
> 
> Perhaps.  Given that there seems to be no alternative, we don't have
> much of a choice.

Please do *NOT* cross-reply.  It was Christoph who said that, NOT ME.

>>>It's not my problem if big companies can't listen and do things their
>>>customers have to pay for, and it's certainly not our job to work
>>>around their idiocy.
> 
> 
> Yes, CSMI should have had more Linux input when it was developed.  The
> no-new IOCTL policy certainly came as a surprise to the authors. Still,
> there doesn't seem to be any other usable cross-platform interface that
> is acceptable to the linux community (or at least to Christoph)'s corner
> of it).  My personal preference is to hide this stuff in a library, so
> the kernel implementation is hidden. But even a library needs an
> underlying kernel implementation.

Please do *NOT* cross-reply.  It was Christoph who said that, NOT ME.

>>Bold statment.
>>
>>Who should "big companies" listen to?  You?  "The community?"
>>Are you saying "big companies" whould listen to Linux which
>>says "no to specs" among other things?
>>
>>Often enough what "big companies" agree on and use and deploy is
>>what Linux (you?) should _listen_ to, try to understand and maybe
>>get out of the way.
>>
> 
> 
> Big companies often want to do things in a proprietary fashion.  I
> personally would prefer to see a standard's-based approach.

I think SDI is what we're talking about here.

>>It is all about customer satisfaction, which is completely
>>foreign to Linux, simply because of the _nature_ of Linux.
> 
> 
> More bold statements? ;-)

No, its just reality.  It appears that you have had very little
interaction with your software engineers about "compatibility".
Also read the Linux blurb on kernel.org, the "What is Linux?" one.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/20/05 21:50, Douglas Gilbert wrote:
>>Luben suggestion(correct me if I'm wrong) is this should be 
>>done thru user space, instead of exporting this in the 
>>/sys/class/sas_phy attributes.  Thus we need to export a 
>>SMP_PASSTHRU mechanism.  I believe there is discussion
>>by Christoph ( and I believe Luben backs this idea) of doing 
>>all this thru the /dev/sg interface.
> 
> 
> Folks,
> Just a suggestion if you want to bring /dev/sg into
> the picture for a SMP (and even a STP) passthrough.
> The host could define a lun (pick a number) and the
> LLDD could supply the target code for it (as scsi_debug
> does for example).

Doug, there is no reason to follow someone else's OS
broken path.

Your question above is SDI addressability. And the answer
is _right_ in the SDI spec: SDI is addressable as per
"controller".  That is each "controller" on the PCI/IO
bus provides an SDI portal.  This is overly evident
by at least looking at the SDI_GET_CNTLR_CONFIG function.

This allows for controllers who do not have SCSI LLDDs
but block or RAID LLDD to also be able to export an SDI
portal.

So in effect the SDI portal would have to be hooked to the
PCI device which claims a base class of 01h -- mass storage
controller.

So when the provider calls 
	sdi_register_provider(sdi_prov);

it fill out the pcidev member in

struct sdi_provider_struct {
	char           *name;
	struct pci_dev *pcidev;
	...
	struct sdi_functions_struct sdi_functions;
	...
};

from my example yesterday.  And the file drivers/scsi/sdi/sdi.c
creates a binary file attr (front end, could be another way)
in  /sys/devices/pciABCD:EF/.../sdi_portal.

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/

Re: [PATCH 1/4] sas: add flag for locally attached PHYs

[Luben Tuikov]

On 10/21/05 13:05, Andrew Patterson wrote:
>>but why didn't CSMI specify the library interface then? If it did that,
>>then none of the "ioctls are deprecated for 3 years now" would have come
>>as a surprise. Or rather, it maybe would have, but it wouldn't have
>>mattered to the management app.
> 
> I asked one of the authors, Steve Fairchild, about this.  He said that
> they had had a bad experience with HBAAPI, the FC library.  They had a
> lot of problems with vendors keeping the library up to date, multiple
> vendors shipping multiple incompatible versions, etc.  I wasn't entirely
> convinced, since they seem to have just replaced one set of problems for
> another.

The user app would read the SDI major and minor from SDI_GET_DRIVER_INFO
and then know which spec to use.

After that just sending bytes of data (forming units) one after the other
(as in "smp_portal") would suffice for 32/64 bit (similarly to SG).

	Luben
-- 
http://linux.adaptec.com/sas/
http://www.adaptec.com/sas/