REPORT LUNS

REPORT LUNS

[Josef Möllers]

[-- Attachment #1: Type: text/plain, Size: 391 bytes --]

Hi,

Wile trying to figure out how to ignore non-existing LUNs on RAID
devices, I have fallen across an error in scsilun_to_int() where it
doesn't behave as the comment suggests and a problematic piece of code
in scsi_report_lun_scan().

Patch is included.

-- 
Josef Möllers (Pinguinpfleger bei FSC)
	If failure had no penalty success would not be a prize
						-- T.  Pratchett

[-- Attachment #2: patch-2.5.70a --]
[-- Type: text/plain, Size: 996 bytes --]

--- linux-2.5.70/drivers/scsi/scsi_scan.c	2003-05-27 03:00:46.000000000 +0200
+++ linux-2.5.70a/drivers/scsi/scsi_scan.c	2003-06-13 15:54:22.000000000 +0200
@@ -884,8 +884,8 @@
 
 	lun = 0;
 	for (i = 0; i < sizeof(lun); i += 2)
-		lun = lun | (((scsilun->scsi_lun[i] << 8) |
-			      scsilun->scsi_lun[i + 1]) << (i * 8));
+		lun = lun | (((scsilun->scsi_lun[i] & 0x3f) << (i * 8 + 1)) |
+			      ((scsilun->scsi_lun[i + 1]) << (i * 8)));
 	return lun;
 }
 
@@ -1027,13 +1027,18 @@
 	 * the header, so start at 1 and go up to and including num_luns.
 	 */
 	for (lunp = &lun_data[1]; lunp <= &lun_data[num_luns]; lunp++) {
+		int j;
+
 		lun = scsilun_to_int(lunp);
 
 		/*
 		 * Check if the unused part of lunp is non-zero, and so
 		 * does not fit in lun.
 		 */
-		if (memcmp(&lunp->scsi_lun[sizeof(lun)], "\0\0\0\0", 4)) {
+		for (j = sizeof(lun); j < sizeof(struct scsi_lun); j++)
+		    if (lunp->scsi_lun[j] != 0)
+			break;
+		if (j < sizeof(struct scsi_lun)) {
 			int i;
 
 			/*

Re: REPORT LUNS

[Patrick Mansfield]

On Fri, Jun 13, 2003 at 04:04:48PM +0200, Josef Möllers wrote:
> Hi,
> 
> Wile trying to figure out how to ignore non-existing LUNs on RAID
> devices, I have fallen across an error in scsilun_to_int() where it
> doesn't behave as the comment suggests and a problematic piece of code
> in scsi_report_lun_scan().
> 
> Patch is included.

If we clear the upper bits (address method, per SAM-3), someone has to
eventually set them again, and this is not handled today in linux scsi
core.

What adapter and driver are you using? AFAIK the Emulex driver figures out
the address method (high two bits) and ors it with each LUN sent down.

I don't recall what happens with the qlogic driver, I thought it had some
special code to handle these, but can't find it.

For SPI it seems OK to ignore the upper bits since they are truncated.
Maybe some devices work OK this way - they set an address method of 10b or
01b, and then don't care that the LUN comes in without the high bits set.

So the change might work for some adapters, but break on others.

We are probably broken for anyone using the 11b extended logical unit
addressing method (with or without your patch).

Can you change your RAID device to use the peripheral device addressing
method - address method of 00b - instead?

In any case we should do what actually works, given that we are
translating an 8 byte LUN to a (host ordered) int, and later converting
that back to an 8 or 1 byte LUN (for qlogic, to a 2 byte LUN that is then
converted to an 8 byte LUN).

> @@ -1027,13 +1027,18 @@
>  	 * the header, so start at 1 and go up to and including num_luns.
>  	 */
>  	for (lunp = &lun_data[1]; lunp <= &lun_data[num_luns]; lunp++) {
> +		int j;
> +
>  		lun = scsilun_to_int(lunp);
>  
>  		/*
>  		 * Check if the unused part of lunp is non-zero, and so
>  		 * does not fit in lun.
>  		 */
> -		if (memcmp(&lunp->scsi_lun[sizeof(lun)], "\0\0\0\0", 4)) {
> +		for (j = sizeof(lun); j < sizeof(struct scsi_lun); j++)
> +		    if (lunp->scsi_lun[j] != 0)
> +			break;
> +		if (j < sizeof(struct scsi_lun)) {
>  			int i;
>  
>  			/*

What does the above patch fix? It looks like they both check if the last
four bytes are 0.

PS: Going offline really soon now for about a week.

-- Patrick Mansfield
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: REPORT LUNS

[Josef Möllers]

Patrick Mansfield wrote:
> 
> On Fri, Jun 13, 2003 at 04:04:48PM +0200, Josef Möllers wrote:
> > Hi,
> >
> > Wile trying to figure out how to ignore non-existing LUNs on RAID
> > devices, I have fallen across an error in scsilun_to_int() where it
> > doesn't behave as the comment suggests and a problematic piece of code
> > in scsi_report_lun_scan().
> >
> > Patch is included.
> 
> If we clear the upper bits (address method, per SAM-3), someone has to
> eventually set them again, and this is not handled today in linux scsi
> core.

The comment said that the upper two bits were ignored which they were
not in the original code, so, apart from fixing the shifts, I also added
the mask to keep the code in sync with the comment. I haven't yet
followed the use of the value thus created.

> > @@ -1027,13 +1027,18 @@
> >        * the header, so start at 1 and go up to and including num_luns.
> >        */
> >       for (lunp = &lun_data[1]; lunp <= &lun_data[num_luns]; lunp++) {
> > +             int j;
> > +
> >               lun = scsilun_to_int(lunp);
> >
> >               /*
> >                * Check if the unused part of lunp is non-zero, and so
> >                * does not fit in lun.
> >                */
> > -             if (memcmp(&lunp->scsi_lun[sizeof(lun)], "\0\0\0\0", 4)) {
> > +             for (j = sizeof(lun); j < sizeof(struct scsi_lun); j++)
> > +                 if (lunp->scsi_lun[j] != 0)
> > +                     break;
> > +             if (j < sizeof(struct scsi_lun)) {
> >                       int i;
> >
> >                       /*
> 
> What does the above patch fix? It looks like they both check if the last
> four bytes are 0.

It does so, iff sizeof(lun) is 4 and sizeof(struct scsi_lun) is 8.
Perhaps it was more a question of personal taste to change the code so
that it also works for 64 bit ints.

-- 
Josef Möllers (Pinguinpfleger bei FSC)
	If failure had no penalty success would not be a prize
						-- T.  Pratchett
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: REPORT LUNS

[Patrick Mansfield]

On Mon, Jun 16, 2003 at 08:55:18AM +0200, Josef Möllers wrote:
> Patrick Mansfield wrote:
> > 
> > On Fri, Jun 13, 2003 at 04:04:48PM +0200, Josef Möllers wrote:
> > > Hi,
> > >
> > > Wile trying to figure out how to ignore non-existing LUNs on RAID
> > > devices, I have fallen across an error in scsilun_to_int() where it
> > > doesn't behave as the comment suggests and a problematic piece of code
> > > in scsi_report_lun_scan().
> > >
> > > Patch is included.
> > 
> > If we clear the upper bits (address method, per SAM-3), someone has to
> > eventually set them again, and this is not handled today in linux scsi
> > core.
> 
> The comment said that the upper two bits were ignored which they were
> not in the original code, so, apart from fixing the shifts, I also added
> the mask to keep the code in sync with the comment. I haven't yet
> followed the use of the value thus created.

Ignored meaning "does nothing with it", when we should probably do
something - proper conversion, an error or warning output. The current
code might work (with high bits set), if we did not exceed the
host->max_lun. We might need an 8 byte lun everywhere to make it work
right for all cases.

Can you give more details of the problem you are seeing, and software
(adapter driver) and hardware in use?

I did not notice your shift changes - I don't see how they work or what
it fixes.

If scsi_lun was: 0x1 0x2 0x3 0x4 0 0 0 0

Current code:

First loop in scsilun_to_int, with i = 0:

	lun = 0 | ( (0x1 << 8 | 0x2) << 0) = 0x0102

Second loop i = 2:

	lun = 0x0102 | ( (0x3 << 8 | 0x4) << 16) = 0x03040102

Your patch would give:

First loop i = 0:

	lun = 0 | ((0x1 & 0x3f) << 1) | (0x2 << 0) = 0x02

Second loop i = 2:

	lun = 0x02 | ( ((0x3 & 0x3f) << 17) | (0x4 << 16) = 0x00060002

-- Patrick Mansfield
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: REPORT LUNS

[Josef Möllers]

Patrick Mansfield wrote:
> 
> On Mon, Jun 16, 2003 at 08:55:18AM +0200, Josef Möllers wrote:
> > Patrick Mansfield wrote:
> > >
> > > On Fri, Jun 13, 2003 at 04:04:48PM +0200, Josef Möllers wrote:
> > > > Hi,
> > > >
> > > > Wile trying to figure out how to ignore non-existing LUNs on RAID
> > > > devices, I have fallen across an error in scsilun_to_int() where it
> > > > doesn't behave as the comment suggests and a problematic piece of code
> > > > in scsi_report_lun_scan().
> > > >
> > > > Patch is included.
> > >
> > > If we clear the upper bits (address method, per SAM-3), someone has to
> > > eventually set them again, and this is not handled today in linux scsi
> > > core.
> >
> > The comment said that the upper two bits were ignored which they were
> > not in the original code, so, apart from fixing the shifts, I also added
> > the mask to keep the code in sync with the comment. I haven't yet
> > followed the use of the value thus created.
> 
> Ignored meaning "does nothing with it", when we should probably do
> something - proper conversion, an error or warning output. The current
> code might work (with high bits set), if we did not exceed the
> host->max_lun. We might need an 8 byte lun everywhere to make it work
> right for all cases.

OK.

> Can you give more details of the problem you are seeing, and software
> (adapter driver) and hardware in use?

The main problem we have is with current 2.4 kernels that some RAID
systems report SCSI-2 but allow for more than 8 LUNs, so rather than put
each box into the device_list[] (BLIST_SPARSELUN), I was investigating
how this problem could be solved in an orderly fashion: either by using
REPORT LUNS or by examining the Peripheral Qualifier in the INQUIRY data
(which would require BLIST_SPARSELUN, nonetheless).
I looked at the 2.5 code and found that REPORT LUNS was implemented but
stumbled across the comment. I couln't really make sense out of SCSI-3
and a manual I have for EMC RAID subsystems on the data returned by the
REPORT LUNS command. I then looked at the code, got confused about the
shifts and broke the code with my patch.

> I did not notice your shift changes - I don't see how they work or what
> it fixes.

Looking at it fresh in the morning: no, it doesn't fix anything, it
breaks code which is OK. I learnt my lesson to be more careful.

> If scsi_lun was: 0x1 0x2 0x3 0x4 0 0 0 0
> 
> Current code:
> 
> First loop in scsilun_to_int, with i = 0:
> 
>         lun = 0 | ( (0x1 << 8 | 0x2) << 0) = 0x0102
> 
> Second loop i = 2:
> 
>         lun = 0x0102 | ( (0x3 << 8 | 0x4) << 16) = 0x03040102
> 
> Your patch would give:
> 
> First loop i = 0:
> 
>         lun = 0 | ((0x1 & 0x3f) << 1) | (0x2 << 0) = 0x02
> 
> Second loop i = 2:
> 
>         lun = 0x02 | ( ((0x3 & 0x3f) << 17) | (0x4 << 16) = 0x00060002

You're absolutely right. If i needed an excuse (other than not having
been careful enough), I'd attribute it to the heat.

I'll test my patches more thoroughly in the future.

Sorry,

Josef
-- 
Josef Möllers (Pinguinpfleger bei FSC)
	If failure had no penalty success would not be a prize
						-- T.  Pratchett
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Patrick Mansfield]

On Thu, Oct 28, 2004 at 02:33:21AM +0300, Catalin Muresan wrote:
> 
> 	Hi,
> 
> 	I have the hardware from the subject, the problem i have is that the
> scsi layer doesn't see any luns i made on the raid except LUN 0. here is an
> example:

LUN 0 didn't show up according to your log output and comments below.

> [root@zerg-b root]# cd /proc/scsi/qla2xxx/  
> [root@zerg-b qla2xxx]# cat 0
> QLogic PCI to Fibre Channel Host Adapter for IBM HS20:
>         Firmware version 3.02.30 IPX, Driver version 8.00.00b15-k

> SCSI Device Information:
> scsi-qla0-adapter-node=2000000d60d367e0;
> scsi-qla0-adapter-port=2100000d60d367e0;
> scsi-qla0-target-0=60003930000026b4;
> 
> SCSI LUN Information:
> (Id:Lun)  * - indicates lun is not registered with the OS.
> ( 0: 5): Total reqs 0, Pending reqs 0, flags 0x0*, 0:0:83 00

The adapter seems to think there is a target and lun 0.

Did anything show up on host 1?

> 	you can see that the driver has found the 5th LUN but scsi_scan
> doesn't se it when scanning, scsi_probe_and_add_lun for LUN 0 returns
> SCSI_SCAN_NO_RESPONSE instead of SCSI_SCAN_TARGET_PRESENT. I have added
> {"APPLE", "Xserve RAID", NULL, BLIST_SPARSELUN}, in scsi_devinfo.c so
> scsi_scan will try and perform a sparse scan on the LUNs but that doesn't
> help because scanning LUN 0 fails.

Yep.

> Oct 28 01:59:37 zerg-b kernel: scsi_scan_host_selected: <0:4294967295:4294967295:4294967295>
> Oct 28 01:59:37 zerg-b kernel: scsi scan: INQUIRY to host 0 channel 0 id 0 lun 0
> Oct 28 01:59:37 zerg-b kernel: scsi scan: 1st INQUIRY failed with code 0x10000

The 0x10000 is a DID_NO_CONNECT, generally the adapter can't talk to
the target at all.

It seem like the storage was seen, then went away and then came back.

Do you always get the same behaviour?

Are there other messages in your log? Just post the entire log without
the scsi logging on, from adapter load time to scan completion.

> 	storage is APPLE Xserve RAID, latest firmware with lun masking activated, for this node
> the 5th LUN is allowed. using add-single-device does the trick but i need a
> kernel to boot from FC. here is the output after 

> # echo "scsi add-single-device 0 0 0 5" > /proc/scsi/scsi

You can also use:

	echo "- - -" > /sys/class/scsi_host/host0/scan

-- Patrick Mansfield

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Catalin Muresan]

	Hi,

On Thu, Oct 28, 2004 at 07:37:34AM -0700, Patrick Mansfield wrote:
 > On Thu, Oct 28, 2004 at 02:33:21AM +0300, Catalin Muresan wrote:
 > > 
 > > 	Hi,
 > > 
 > > 	I have the hardware from the subject, the problem i have is that the
 > > scsi layer doesn't see any luns i made on the raid except LUN 0. here is an
 > > example:
 > 
 > LUN 0 didn't show up according to your log output and comments below.

	the storage performs LUN masking so there's only LUN 5 available on
ID 0.

 > > [root@zerg-b root]# cd /proc/scsi/qla2xxx/  
 > > [root@zerg-b qla2xxx]# cat 0
 > > QLogic PCI to Fibre Channel Host Adapter for IBM HS20:
 > >         Firmware version 3.02.30 IPX, Driver version 8.00.00b15-k
 > 
 > > SCSI Device Information:
 > > scsi-qla0-adapter-node=2000000d60d367e0;
 > > scsi-qla0-adapter-port=2100000d60d367e0;
 > > scsi-qla0-target-0=60003930000026b4;
 > > 
 > > SCSI LUN Information:
 > > (Id:Lun)  * - indicates lun is not registered with the OS.
 > > ( 0: 5): Total reqs 0, Pending reqs 0, flags 0x0*, 0:0:83 00
 > 
 > The adapter seems to think there is a target and lun 0.

	the adapter sees id 0 lun 5 ( 0: 5) because it blindly scans every lun

 > Did anything show up on host 1?

	host 1 is not connected to the FC fabric because i have only 1 FC
switch module installed, that's why i didn't include that info - short no.

 > > 	you can see that the driver has found the 5th LUN but scsi_scan
 > > doesn't se it when scanning, scsi_probe_and_add_lun for LUN 0 returns
 > > SCSI_SCAN_NO_RESPONSE instead of SCSI_SCAN_TARGET_PRESENT. I have added
 > > {"APPLE", "Xserve RAID", NULL, BLIST_SPARSELUN}, in scsi_devinfo.c so
 > > scsi_scan will try and perform a sparse scan on the LUNs but that doesn't
 > > help because scanning LUN 0 fails.
 > 
 > Yep.
 > 
 > > Oct 28 01:59:37 zerg-b kernel: scsi_scan_host_selected: <0:4294967295:4294967295:4294967295>
 > > Oct 28 01:59:37 zerg-b kernel: scsi scan: INQUIRY to host 0 channel 0 id 0 lun 0
 > > Oct 28 01:59:37 zerg-b kernel: scsi scan: 1st INQUIRY failed with code 0x10000
 > 
 > The 0x10000 is a DID_NO_CONNECT, generally the adapter can't talk to
 > the target at all.

	I don't know what the SCSI spec has to say about this, but by
looking at the code it should respond with SCSI_SCAN_TARGET_PRESENT.

 > It seem like the storage was seen, then went away and then came back.

	no, it's just that the storage or the controller don't report
anything for masked luns. It looks more and more a storage problem, I'm
gonna try and bug Apple about this. What does the SCSI spec says about this?

 > Do you always get the same behaviour?

	yes.

 > Are there other messages in your log? Just post the entire log without
 > the scsi logging on, from adapter load time to scan completion.

	ok but is like there's no devices attached (it's an older log
because i've rebooted with another kernel that scans every lun on every
target, thus finding the storage)

Oct 27 22:23:22 zerg-b kernel: QLogic Fibre Channel HBA Driver (c02ad498)
Oct 27 22:23:22 zerg-b kernel: ACPI: PCI interrupt 0000:02:02.0[A] -> GSI 18 (level, low) -> IRQ 177
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: Found an ISP2312, irq 177, iobase 0xf8802000
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: Configuring PCI space...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: Configure NVRAM parameters...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: Verifying loaded RISC code...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: Waiting for LIP to complete...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: LOOP UP detected (2 Gbps).
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: Topology - (F_Port), Host Loop address 0xffff
Oct 27 22:23:22 zerg-b kernel: scsi0 : qla2xxx
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.0: 
Oct 27 22:23:22 zerg-b kernel:  QLogic Fibre Channel HBA Driver: 8.00.00b15-k
Oct 27 22:23:22 zerg-b kernel:   QLogic IBM HS20 - 
Oct 27 22:23:22 zerg-b kernel:   ISP2312: PCI-X (133 MHz) @ 0000:02:02.0 hdma-, host#=0, fw=3.02.30 IPX
Oct 27 22:23:22 zerg-b kernel: ACPI: PCI interrupt 0000:02:02.1[B] -> GSI 19 (level, low) -> IRQ 185
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.1: Found an ISP2312, irq 185, iobase 0xf8808000
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.1: Configuring PCI space...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.1: Configure NVRAM parameters...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.1: Verifying loaded RISC code...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.1: Waiting for LIP to complete...
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.1: Cable is unplugged...
Oct 27 22:23:22 zerg-b kernel: scsi1 : qla2xxx
Oct 27 22:23:22 zerg-b kernel: qla2300 0000:02:02.1: 
Oct 27 22:23:22 zerg-b kernel:  QLogic Fibre Channel HBA Driver: 8.00.00b15-k
Oct 27 22:23:22 zerg-b kernel:   QLogic IBM HS20 - 
Oct 27 22:23:22 zerg-b kernel:   ISP2312: PCI-X (133 MHz) @ 0000:02:02.1 hdma-, host#=1, fw=3.02.30 IPX
Oct 27 22:23:22 zerg-b kernel: ACPI: PCI interrupt 0000:00:0f.2[A] -> GSI 7 (level, low) -> IRQ 7

 > > 	storage is APPLE Xserve RAID, latest firmware with lun masking activated, for this node
 > > the 5th LUN is allowed. using add-single-device does the trick but i need a
 > > kernel to boot from FC. here is the output after 
 > 
 > > # echo "scsi add-single-device 0 0 0 5" > /proc/scsi/scsi
 > 
 > You can also use:
 > 
 > 	echo "- - -" > /sys/class/scsi_host/host0/scan

	my goal is to have a kernel with which I can boot from FC, but i see
that can be done only with an ugly patch(for now) or with tricks from
initrd.

 > -- Patrick Mansfield

-- 
Kat

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Patrick Mansfield]

On Thu, Oct 28, 2004 at 06:35:22PM +0300, Catalin Muresan wrote:

Ok, that should have been lun 5.

>  > > Oct 28 01:59:37 zerg-b kernel: scsi_scan_host_selected: <0:4294967295:4294967295:4294967295>
>  > > Oct 28 01:59:37 zerg-b kernel: scsi scan: INQUIRY to host 0 channel 0 id 0 lun 0
>  > > Oct 28 01:59:37 zerg-b kernel: scsi scan: 1st INQUIRY failed with code 0x10000
>  > 
>  > The 0x10000 is a DID_NO_CONNECT, generally the adapter can't talk to
>  > the target at all.
> 
> 	I don't know what the SCSI spec has to say about this, but by
> looking at the code it should respond with SCSI_SCAN_TARGET_PRESENT.

Hmmm ... my interpretation of DID_NO_CONNECT was that the adapter cannot
talk to the target at all.

You should be able to send an INQUIRY to any LUN on a target, the HBA
really shouldn't block it.

Given that, it is correct for the scan to give SCSI_SCAN_TARGET_PRESENT
on a DID_NO_CONNECT.

So, it looks like the qlogic is giving a DID_NO_CONNECT when it should
not. We could hack around this in scsi_scan.c but that might be bad.

James or Andrew what do you think?

Catalin - does the device support REPORT LUNS? Do you have that
configured? It would avoid this problem.

> 	my goal is to have a kernel with which I can boot from FC, but i see
> that can be done only with an ugly patch(for now) or with tricks from
> initrd.

What tricks and ugly patch? I haven't tried boot via qlogic on an HS20.

-- Patrick Mansfield

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Andrew Vasquez]

On Thu, 28 Oct 2004, Patrick Mansfield wrote:

> On Thu, Oct 28, 2004 at 06:35:22PM +0300, Catalin Muresan wrote:
> 
> Ok, that should have been lun 5.
> 
> >  > > Oct 28 01:59:37 zerg-b kernel: scsi_scan_host_selected: <0:4294967295:4294967295:4294967295>
> >  > > Oct 28 01:59:37 zerg-b kernel: scsi scan: INQUIRY to host 0 channel 0 id 0 lun 0
> >  > > Oct 28 01:59:37 zerg-b kernel: scsi scan: 1st INQUIRY failed with code 0x10000
> >  > 
> >  > The 0x10000 is a DID_NO_CONNECT, generally the adapter can't talk to
> >  > the target at all.
> > 
> > 	I don't know what the SCSI spec has to say about this, but by
> > looking at the code it should respond with SCSI_SCAN_TARGET_PRESENT.
> 
> Hmmm ... my interpretation of DID_NO_CONNECT was that the adapter cannot
> talk to the target at all.
> 
> You should be able to send an INQUIRY to any LUN on a target, the HBA
> really shouldn't block it.
> 
> Given that, it is correct for the scan to give SCSI_SCAN_TARGET_PRESENT
> on a DID_NO_CONNECT.
> 

Not wanting to go into the history of the qla2xxx driver's internal
port/lun structures, let me just say that the driver should always
create an internal 'lun 0' object so that the mid-layer can perform
its scan.

> So, it looks like the qlogic is giving a DID_NO_CONNECT when it should
> not. We could hack around this in scsi_scan.c but that might be bad.
> 
> James or Andrew what do you think?
> 

Yuck, the midlayer should not have to change.

> Catalin - does the device support REPORT LUNS? Do you have that
> configured? It would avoid this problem.
> 

Yes, Catalin, could you find that out?  I've only had a small amount
of exposure with the XRAID box and know that there were some quirks
about the device not performing an RFT_ID with the SNS, not sure if
that was addressed.

> > 	my goal is to have a kernel with which I can boot from FC, but i see
> > that can be done only with an ugly patch(for now) or with tricks from
> > initrd.
> 
> What tricks and ugly patch? I haven't tried boot via qlogic on an HS20.
> 

Catalin,  could you enable some debug settings:

In qla_dbg.h, modify the following line:

	/* #define QL_DEBUG_LEVEL_2  */ /* Output error msgs to COM1 */

to read as:

	#define QL_DEBUG_LEVEL_2   /* Output error msgs to COM1 */

In qla_settings.h, modify the following line:

	#define DEBUG_QLA2100           0       /* For Debug of qla2x00 */

to read as:

	#define DEBUG_QLA2100           1       /* For Debug of qla2x00 */

recompile the driver, load, and forward over the relevent
var/log/messages snippet to me.


BTW: Regarding the internal queues and port/lun objects -- most of our
'experimental' work with 2.6 is begin done with our iSCSI driver.
This includes removal of the driver's internal queuing mechanism and
dependencies on these port/lun objects being built internally during
the driver scan.  Porting (actually mostly pruning) to the FC driver
will follow shortly.

--
Andrew Vasquez

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Catalin Muresan]

On Thu, Oct 28, 2004 at 10:21:43AM -0700, Andrew Vasquez wrote:
 > Not wanting to go into the history of the qla2xxx driver's internal
 > port/lun structures, let me just say that the driver should always
 > create an internal 'lun 0' object so that the mid-layer can perform
 > its scan.

	that sound like fixing hardware bugs in software ;)

 > > Catalin - does the device support REPORT LUNS? Do you have that
 > > configured? It would avoid this problem.
 > > 
 > 
 > Yes, Catalin, could you find that out?  I've only had a small amount
 > of exposure with the XRAID box and know that there were some quirks
 > about the device not performing an RFT_ID with the SNS, not sure if
 > that was addressed.

	i don't know how to find that, i'm gonna go and look at the source
and try to figure it out untill you respond.

 > > > 	my goal is to have a kernel with which I can boot from FC, but i see
 > > > that can be done only with an ugly patch(for now) or with tricks from
 > > > initrd.
 > > 
 > > What tricks and ugly patch? I haven't tried boot via qlogic on an HS20.
 > > 
 > 
 > Catalin,  could you enable some debug settings:
 > 
 > In qla_dbg.h, modify the following line:
 > 
 > 	/* #define QL_DEBUG_LEVEL_2  */ /* Output error msgs to COM1 */
 > 
 > to read as:
 > 
 > 	#define QL_DEBUG_LEVEL_2   /* Output error msgs to COM1 */
 > 
 > In qla_settings.h, modify the following line:
 > 
 > 	#define DEBUG_QLA2100           0       /* For Debug of qla2x00 */
 > 
 > to read as:
 > 
 > 	#define DEBUG_QLA2100           1       /* For Debug of qla2x00 */
 > 
 > recompile the driver, load, and forward over the relevent
 > var/log/messages snippet to me.

	will do that asap.

 > BTW: Regarding the internal queues and port/lun objects -- most of our
 > 'experimental' work with 2.6 is begin done with our iSCSI driver.
 > This includes removal of the driver's internal queuing mechanism and
 > dependencies on these port/lun objects being built internally during
 > the driver scan.  Porting (actually mostly pruning) to the FC driver
 > will follow shortly.
 > 
 > --
 > Andrew Vasquez

-- 
Kat

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Patrick Mansfield]

On Fri, Oct 29, 2004 at 11:58:00AM +0300, Catalin Muresan wrote:

>  > > Catalin - does the device support REPORT LUNS? Do you have that
>  > > configured? It would avoid this problem.
>  > > 
>  > 
>  > Yes, Catalin, could you find that out?  I've only had a small amount
>  > of exposure with the XRAID box and know that there were some quirks
>  > about the device not performing an RFT_ID with the SNS, not sure if
>  > that was addressed.
> 
> 	i don't know how to find that, i'm gonna go and look at the source
> and try to figure it out untill you respond.

I forgot a few things:

1) The REPORT LUNS is always on in current 2.6.x kernels, I do not remember
what kernel version first had the change.

2) If LUN 0 gets a DID_NO_CONNECT (and we do return the
SCSI_SCAN_NO_RESPONSE), we won't scan anything, via REPORT LUN or via
a sequential scan.

3) We have a problem for REPORT LUN with sparse lun devices, in short,
you won't get a REPORT LUN scan if no LUN 0 is configured. I thought Kurt
was working on a fix for that, this post:

http://marc.theaimsgroup.com/?t=109545967900002&r=1&w=2

4) You posted as part of your logs:

Oct 28 02:31:38 zerg-b kernel:   Vendor: APPLE     Model: Xserve RAID Rev: 1.21
Oct 28 02:31:38 zerg-b kernel:   Type:   Direct-Access ANSI SCSI revision: 02

For SCSI 2 devices by default the REPORT LUN scan won't be used. Some
storage arrays have an option as to what SCSI level to report, try that or
use the BLIST_REPORTLUN2 devinfo flag. 

If you had configured in only LUNs 0 1 and 5, the sequential scan (with no
sparse lun) will stop scanning after LUN 2 is not seen.

In short you need to first figure out why LUN 0 is getting a
DID_NO_CONNECT, and after that the sequential scan should work OK for your
*current* configuration (only LUN 5 configured), though the REPORT LUN
scan or sparse lun scan will be needed to avoid sparse LUNs as noted
above.

-- Patrick Mansfield

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Catalin Muresan]

	I have the attached patch from Andrew which fixes the qla2300 part
of the problem (attached) and a patch is still needed for the case where
there are LUN0,3 to blacklist the device as BLIST_SPARSELUN.

On Fri, Oct 29, 2004 at 11:06:33AM -0700, Patrick Mansfield wrote:
 > On Fri, Oct 29, 2004 at 11:58:00AM +0300, Catalin Muresan wrote:
 > 
 > >  > > Catalin - does the device support REPORT LUNS? Do you have that
 > >  > > configured? It would avoid this problem.
 > >  > > 
 > >  > 
 > >  > Yes, Catalin, could you find that out?  I've only had a small amount
 > >  > of exposure with the XRAID box and know that there were some quirks
 > >  > about the device not performing an RFT_ID with the SNS, not sure if
 > >  > that was addressed.
 > > 
 > > 	i don't know how to find that, i'm gonna go and look at the source
 > > and try to figure it out untill you respond.
 > 
 > I forgot a few things:
 > 
 > 1) The REPORT LUNS is always on in current 2.6.x kernels, I do not remember
 > what kernel version first had the change.
 > 
 > 2) If LUN 0 gets a DID_NO_CONNECT (and we do return the
 > SCSI_SCAN_NO_RESPONSE), we won't scan anything, via REPORT LUN or via
 > a sequential scan.

	we need to have something on LUN 0 to be able to send REPORT LUN, in
this case we don't.

 > 3) We have a problem for REPORT LUN with sparse lun devices, in short,
 > you won't get a REPORT LUN scan if no LUN 0 is configured. I thought Kurt
 > was working on a fix for that, this post:
 > 
 > http://marc.theaimsgroup.com/?t=109545967900002&r=1&w=2

	here it was the controller which disabled the LUNs if there were no
devices connected to the LUN. Now (after the patch) we can scan the storage
with REPORT LUNS (if supported) or sequential.

 > 4) You posted as part of your logs:
 > 
 > Oct 28 02:31:38 zerg-b kernel:   Vendor: APPLE     Model: Xserve RAID Rev: 1.21
 > Oct 28 02:31:38 zerg-b kernel:   Type:   Direct-Access ANSI SCSI revision: 02
 > 
 > For SCSI 2 devices by default the REPORT LUN scan won't be used. Some
 > storage arrays have an option as to what SCSI level to report, try that or
 > use the BLIST_REPORTLUN2 devinfo flag. 
 > 
 > If you had configured in only LUNs 0 1 and 5, the sequential scan (with no
 > sparse lun) will stop scanning after LUN 2 is not seen.

	that is where the BLIST_SPARSELUN patch comes to help for non-SCSI-3
devices.

 > In short you need to first figure out why LUN 0 is getting a
 > DID_NO_CONNECT, and after that the sequential scan should work OK for your

	it's because the qla2300 does not enable in it's list LUNs that have
no devices attached. I still think that it should configure all available
LUNs if there is a target present, but this is ok.

 > *current* configuration (only LUN 5 configured), though the REPORT LUN
 > scan or sparse lun scan will be needed to avoid sparse LUNs as noted
 > above.

	Do you need me to test some other stuff or we can "close" this
issue?	

 > -- Patrick Mansfield

-- 
 Catalin Muresan
 ASTRAL Internet & Data
 Technology Manager

Re: Apple Xserve RAID and qlogic ISP2312 (qla2300)

[Patrick Mansfield]

[ resending ... not sure if this went out at all ]

On Mon, Nov 01, 2004 at 12:56:11PM +0200, Catalin Muresan wrote:
> 
> 	I have the attached patch from Andrew which fixes the qla2300 part
> of the problem (attached) and a patch is still needed for the case where
> there are LUN0,3 to blacklist the device as BLIST_SPARSELUN.

No attachment ...

>  > 2) If LUN 0 gets a DID_NO_CONNECT (and we do return the
>  > SCSI_SCAN_NO_RESPONSE), we won't scan anything, via REPORT LUN or via
>  > a sequential scan.
> 
> 	we need to have something on LUN 0 to be able to send REPORT LUN, in
> this case we don't.

You don't need storage configured on LUN 0 to send it a REPORT LUN, if
there is a target there, you can (well should) be able to send it a REPORT
LUN.

You do need the driver to allow commands to be issued to LUN 0 (that is
what the patch should have fixed).

SCSI spec says:

	A SCSI device shall support a REPORT LUNS command that is
	addressed to logical unit zero. Support of the REPORT LUNS command
	by logical units other than logical unit zero is optional.

> 	Do you need me to test some other stuff or we can "close" this
> issue?	

In order of best to worst solutions:

If device supports REPORT LUNs:

1) configure the array to report as SCSI-3

2) black list as BLIST_REPORTLUN2

If no REPORT LUNs support:

3) black list as BLIST_SPARSELUN

-- Patrick Mansfield

Report luns [was: Apple Xserve RAID and qlogic ISP2312 (qla2300)]

[Douglas Gilbert]

Patrick Mansfield wrote:
> [ resending ... not sure if this went out at all ]
> 
> On Mon, Nov 01, 2004 at 12:56:11PM +0200, Catalin Muresan wrote:
> 
>>	I have the attached patch from Andrew which fixes the qla2300 part
>>of the problem (attached) and a patch is still needed for the case where
>>there are LUN0,3 to blacklist the device as BLIST_SPARSELUN.
> 
> 
> No attachment ...
> 
> 
>> > 2) If LUN 0 gets a DID_NO_CONNECT (and we do return the
>> > SCSI_SCAN_NO_RESPONSE), we won't scan anything, via REPORT LUN or via
>> > a sequential scan.
>>
>>	we need to have something on LUN 0 to be able to send REPORT LUN, in
>>this case we don't.
> 
> 
> You don't need storage configured on LUN 0 to send it a REPORT LUN, if
> there is a target there, you can (well should) be able to send it a REPORT
> LUN.
> 
> You do need the driver to allow commands to be issued to LUN 0 (that is
> what the patch should have fixed).
> 
> SCSI spec says:
> 
> 	A SCSI device shall support a REPORT LUNS command that is
> 	addressed to logical unit zero. Support of the REPORT LUNS command
> 	by logical units other than logical unit zero is optional.

Patrick,
That would be SAM or SAM-2. In SAM-3 (and SPC-3) that requirement
has been widened to:
"All SCSI devices shall support LUN 0 (i.e. 00000000 00000000h) or
the REPORT LUNS well-known logical unit". [SAM-3 rev 14 section 4.9.3]

Having recently implemented sg_luns (in sg3_utils) I was surprised
to find a "select" field (byte 2) in REPORT LUN's cdb. When it is 0
(as per scsi_scan.c) the returned lun inventory does _not_ include
any well known lus. When select is 1 the returned lun inventory
only includes well known lus (and when select==2 all lus are
returned). Still this is not going to help in SPC-3 since the
device need not respond to lun 0 at all.

So if that fails scsi_scan.c might issue a REPORT LUNS command to
this lun: 0xc101 which is the value of the REPORT LUNS well-known lu.
Note that this is a 16 bit quantity (not 64 bit as I said in a
previous post about luns). For simplicity let us assume that
that it is at the top level. [If a device returns HiSup=1 in its
standard INQUIRY response then it claims to support a 4 level
hierarchy in its luns (with 16 bits in each level).]

SPC-3 defines 3 well known lus:
   - REPORT LUNS
   - ACCESS CONTROLS [0xc102]
   - TARGET LOG PAGES [0xc103]
and each of these supports a small number of (mandatory) SCSI
commands.

So if a modern SCSI enclosure or bridge uses well known lus
rather than the lun 0 hack, how will applications in linux
access them? They are certainly not block devices.

Doug Gilbert

Re: Report luns [was: Apple Xserve RAID and qlogic ISP2312 (qla2300)]

[Luben Tuikov]

Douglas Gilbert wrote:
> So if that fails scsi_scan.c might issue a REPORT LUNS command to
> this lun: 0xc101 which is the value of the REPORT LUNS well-known lu.
 > Note that this is a 16 bit quantity (not 64 bit as I said in a
> previous post about luns). For simplicity let us assume that
> that it is at the top level. [If a device returns HiSup=1 in its
> standard INQUIRY response then it claims to support a 4 level
> hierarchy in its luns (with 16 bits in each level).]

And also supports REPORT LUNS.

So in either case, we send REPORT LUNS to LUN 0 and if
this fails to REPORT LUNS W-LUN (0xC101...).

Since the standard says that depending on the transport
protocol LUNs are either 16 or 64 bit values, how do
we decide this from SCSI Core/app. client?  Wouldn't it
be better to just send 64 bits with bytes 2 to 7 all 0-ed?

So in general SCSI Core always uses 64 bits and the LLDD
decides between 16 and 64 bits, since the transport could
be messier.

> So if a modern SCSI enclosure or bridge uses well known lus
> rather than the lun 0 hack, how will applications in linux
> access them? They are certainly not block devices.

Ah, good question.  No, they cannot be block devices since
they are not mountable.  But they are _addressable_ devices.

If it is decided to not create a new OS type, then those
could be char devices on the bus, and accessed via sg.

If it is decided to create a new OS type, then 'a' would
be appropriate to denote addressable devices, again
being accessed via sg.  Do you think expanders could
be represented by those device types too?

	Luben

Re: Report luns [was: Apple Xserve RAID and qlogic ISP2312 (qla2300)]

[Patrick Mansfield]

> Douglas Gilbert wrote:
> >So if that fails scsi_scan.c might issue a REPORT LUNS command to
> >this lun: 0xc101 which is the value of the REPORT LUNS well-known lu.
> > Note that this is a 16 bit quantity (not 64 bit as I said in a
> >previous post about luns). For simplicity let us assume that
> >that it is at the top level. [If a device returns HiSup=1 in its
> >standard INQUIRY response then it claims to support a 4 level
> >hierarchy in its luns (with 16 bits in each level).]
> 
> And also supports REPORT LUNS.
> 
> So in either case, we send REPORT LUNS to LUN 0 and if
> this fails to REPORT LUNS W-LUN (0xC101...).

Do we also have to change to send the INQUIRY to the well known LUN?

And then someday add black and white list flags and code for them ...

Why is stuff like this even added to the standard??? Why not use LUN 0
(all zeroes) or at worst have one well known LUN like the all f's?

The hierarchical LUN stuff and the various address modes are dumb, just
treat the LUN as a tag/identifier (similiar to a TCP/IP network port) and
leave the interpretation of it up to the target device (or at worst the
transport).

-- Patrick Mansfield

Re: Report luns

[Douglas Gilbert]

Patrick Mansfield wrote:
>>Douglas Gilbert wrote:
>>
>>>So if that fails scsi_scan.c might issue a REPORT LUNS command to
>>>this lun: 0xc101 which is the value of the REPORT LUNS well-known lu.
>>>Note that this is a 16 bit quantity (not 64 bit as I said in a
>>>previous post about luns). For simplicity let us assume that
>>>that it is at the top level. [If a device returns HiSup=1 in its
>>>standard INQUIRY response then it claims to support a 4 level
>>>hierarchy in its luns (with 16 bits in each level).]
>>
>>And also supports REPORT LUNS.
>>
>>So in either case, we send REPORT LUNS to LUN 0 and if
>>this fails to REPORT LUNS W-LUN (0xC101...).
> 
> 
> Do we also have to change to send the INQUIRY to the well known LUN?

Yes, all well known lus have mandatory support for INQUIRY.

> And then someday add black and white list flags and code for them ...

Hope not. If an INQUIRY to lun 0 fails (and perhaps the transport level
doesn't object) then send an INQUIRY to lun 0xc101 . If that succeeds
then send a REPORT LUNS to lun 0xc101 ...

> Why is stuff like this even added to the standard??? Why not use LUN 0
> (all zeroes) or at worst have one well known LUN like the all f's?

All f's is already defined in SAM-3 as "lu not specified", whatever
that means :-)

> The hierarchical LUN stuff and the various address modes are dumb, just
> treat the LUN as a tag/identifier (similiar to a TCP/IP network port) and
> leave the interpretation of it up to the target device (or at worst the
> transport).

As you have worked in the scsi_scan.c area you are well placed
to comment. I have only recently been looking at luns. I also
wonder about the device peripheral type returned by an INQUIRY
sent to a W-LUN (i.e. well known lun). If it was closely associated
with disks (e.g. a RAID) and returned a "direct access" device then
the sd driver wouldn't make much sense of a W-LUN device. Note this
is not a problem while select=0 is set in scsi_scan's REPORT LUN
commands.


An example, inspired by the BCC draft, involving well
known logical units might help. Say we have
a bridge from SAS (or FC, it doesn't matter) to SPI-4 (U320).
One way to implement the bridge is to map the 15 possible
devices on the U320 bus to luns behind a single target in
a SAS domain (or SAS bus, it's the same thing). Lets assume
there are 5 U320 disks connected to the SPI bus and assume
SPI target ids 0 to 4 map to SAS luns 0 to 4.

So now if one sends a REPORT LUNS to lun 0 that U320 disk is
expected to know about its sibling disks!? That is why I used
the term "hack" with respect to lun 0. What really happens
is the device server in the bridge intercepts the REPORT LUNS
command and produces a response reflecting its knowledge of
those 5 U320 disks present. Why play this game? Why not talk
directly to the device server (addressed via a well known lu).
This way an INQUIRY to a W-LUN yields information about
the bridge, while an INQUIRY to lun 0 yields information about
that disk.
If the TARGET LOG PAGES W-LUN is supported then the
bridge can supply logs from its point of view (a SAS
bridge device) while a LOG SENSE command sent to lun 4
reports that U320 disk's logs.


Luben is probably correct in saying manufacturers will
support both lun 0 and the REPORT LUNS well known lu
in advanced products such as bridges. In Linux currently
we can detect the presence of well known lus (e.g.
sg_luns --select=1 /dev/sg3). If anything is found we
have no mechanism to probe them.

Doug Gilbert

Re: Report luns

[Luben Tuikov]

Douglas Gilbert wrote:
> 
> So now if one sends a REPORT LUNS to lun 0 that U320 disk is
> expected to know about its sibling disks!? That is why I used
> the term "hack" with respect to lun 0. What really happens
> is the device server in the bridge intercepts the REPORT LUNS
> command and produces a response reflecting its knowledge of
> those 5 U320 disks present. Why play this game? Why not talk
> directly to the device server (addressed via a well known lu).
> This way an INQUIRY to a W-LUN yields information about
> the bridge, while an INQUIRY to lun 0 yields information about
> that disk.

Very good point!

So we can reverse the logic into sending REPORT LUNS to
the REPORT LUNS W-LUN *first* and if this fails (most probably
in early years), then issue REPORT LUNS to LUN 0.

	Luben