socal

socal

[Aaron Wirtz]

I have a Sun e3500 and I am trying to get the linux socal/fcal drivers 
working.  I have done a bit of reasearch and have tried several different 
linux kernel versions and different socal microcode versions to no avail.  
Does anyone know of a patch that fixes the broken socal/fcal drivers under 
2.4.x or, failing that, does anyone know what version of the linux kernel and 
the Solaris microcode Jakub was using in the past (1999) when he had the 
driver working?

As a side note, all of the socal microcode I have seen so far uses the full 
64kb of data and has none of the null padding at the end that Jakub mentioned 
in his post - this confuses part of the driver code (65536 is too big for an 
unsigned int) so I just cut out the code that adds the null padding for 
microcode <64kb since my microcode has no padding anyway, and the module 
compiles properly.

When I try to modpobe the socal drivers, there are no messages and it shows as 
loaded under lsmod.
When I try to load fcal, it complains that there are no devices.

The same machine runs fine with the solaris socal driver under Solaris 9.

Thanks.

-Aaron Wirtz

Re: socal

[C.Newport]

On Wednesday 31 December 2003 10:29 pm, Aaron Wirtz wrote:
> I have a Sun e3500 and I am trying to get the linux socal/fcal drivers
> working.  I have done a bit of reasearch and have tried several different
> linux kernel versions and different socal microcode versions to no avail.
> Does anyone know of a patch that fixes the broken socal/fcal drivers under
> 2.4.x or, failing that, does anyone know what version of the linux kernel
> and the Solaris microcode Jakub was using in the past (1999) when he had
> the driver working?

Have you tried checking the firmware Fcode ?.
http://www.splack.org/fibrechan.html has the procedure for the soc
cards, the socal stuff is very similar.

Re: socal

[Keith M Wesolowski]

On Wed, Dec 31, 2003 at 02:29:26PM -0800, Aaron Wirtz wrote:

> Does anyone know of a patch that fixes the broken socal/fcal drivers
> under 2.4.x or, failing that, does anyone know what version of the
> linux kernel and the Solaris microcode Jakub was using in the past
> (1999) when he had the driver working?

Microcode, I don't know.  The driver was added in 2.2.4.  That version
doesn't work either.

> As a side note, all of the socal microcode I have seen so far uses
> the full 64kb of data and has none of the null padding at the end
> that Jakub mentioned in his post - this confuses part of the driver
> code (65536 is too big for an unsigned int) so I just cut out the
> code that adds the null padding for

Huh?  The microcode needs to be 64k in size (or less than that if you
chop off the 0s at the end).  If you disassemble the solaris driver
you will see that their ucode is exactly 64k and that the functions
they call to load it expect 64k.  That value fits in an unsigned int
on any sane platform.  Probably you've extracted the microcode
wrongly.

However, you might as well load an ASCII-art rendition of Bill Joy,
since even with the right microcode the driver does not work.  In
fact, I'd recommend it.  Let us know if it helps.

> When I try to load fcal, it complains that there are no devices.
> 
> The same machine runs fine with the solaris socal driver under Solaris 9.

Unless Jakub wants to look at it or you have the manual for this chip
and plenty of patience, forget it.

-- 
Keith M Wesolowski <wesolows@foobazco.org> http://foobazco.org/~wesolows
------(( Project Foobazco Coordinator and Network Administrator ))------
	"May Buddha bless all stubborn people!"
				-- Uliassutai Karakorum Blake

Re: socal

[Aaron Wirtz]

On Wednesday 31 December 2003 15:07, C.Newport wrote:
>
> Have you tried checking the firmware Fcode ?.
> http://www.splack.org/fibrechan.html has the procedure for the soc
> cards, the socal stuff is very similar.

Given that the Solaris driver (and ucode) work with it under Solaris 9, I 
don't think the fcode version is currently an issue.  One thing I had 
considered was that if I have to use an older ucode version, I may have to 
roll back the fcode to match.

-Aaron Wirtz

Re: socal

[Keith M Wesolowski]

On Wed, Dec 31, 2003 at 11:07:09PM +0000, C.Newport wrote:

> Have you tried checking the firmware Fcode ?.
> http://www.splack.org/fibrechan.html has the procedure for the soc
> cards, the socal stuff is very similar.

You cannot update the fcode in the onboard socal devices on Ex500
using ssaadm.  The fcode is contained within the main PROM for these
systems, of which the latest version is 3.2.30.

-- 
Keith M Wesolowski <wesolows@foobazco.org> http://foobazco.org/~wesolows
------(( Project Foobazco Coordinator and Network Administrator ))------
	"May Buddha bless all stubborn people!"
				-- Uliassutai Karakorum Blake

Re: socal

[Pete Zaitcev]

On Wed, Dec 31, 2003 at 03:19:03PM -0800, Aaron Wirtz wrote:
> On Wednesday 31 December 2003 15:07, C.Newport wrote:
> >
> > Have you tried checking the firmware Fcode ?.
> > http://www.splack.org/fibrechan.html has the procedure for the soc
> > cards, the socal stuff is very similar.
> 
> Given that the Solaris driver (and ucode) work with it under Solaris 9, I 
> don't think the fcode version is currently an issue.  One thing I had 
> considered was that if I have to use an older ucode version, I may have to 
> roll back the fcode to match.

This just makes no sense. FCode carries its own ucode,
which has nothing to do with the one a driver carries.
For one thing, FCode always carries a debugging ucode,
which tops at about 1000 IOPS. A driver carries a normal
microcode, which reaches 5500 IOPS with small blocks,
17000 IOPS with null ops (for IP-over-SCSI, for instance).
They are different and do not have to match, even their
ABI may be slightly different.

-- Pete

Re: socal

[Aaron Wirtz]

On Wednesday 31 December 2003 23:18, C.Newport wrote:
> Huh?  The microcode needs to be 64k in size (or less than that if you
> chop off the 0s at the end).  If you disassemble the solaris driver
> you will see that their ucode is exactly 64k and that the functions
> they call to load it expect 64k.  That value fits in an unsigned int
> on any sane platform.  Probably you've extracted the microcode
> wrongly.
> 
> However, you might as well load an ASCII-art rendition of Bill Joy,
> since even with the right microcode the driver does not work.  In
> fact, I'd recommend it.  Let us know if it helps.

lol, yeah, I know the driver is currently broken, I thought maybe if I could 
get my hands on the same version of the ucode that Jakub successfully used in 
the past, that might fix it especially if used in conjunction with Jakub's 
last functional version of the driver. (he did report actually getting it 
working once upon a time)

As for the ucode size, Jakub tried to cut the module size by storing the ucode 
in his driver without padding and just adding the padding when the driver 
loads it.  The ucode when extracted from the Solaris socal drivers I have 
tried is 64k.  Jakub recommended removing the null padding, but the code I 
get doesn't have any null padding.  If I use a full 64k of ucode in the 
compile, the preprocessor, when calculating the constant that specifies the 
offset at which the padding should start overflows because the constant is 
past the end of the buffer. (sorry, don't know what I was thinking blaming it 
on unsigned ints. ;-)  Must have been my x86 heritage...)  This issue I can 
deal with, but what worries me is that this tells me that I have not yet 
found the same version of the socal ucode that Jakub used when he wrote the 
driver.

> Unless Jakub wants to look at it or you have the manual for this chip
> and plenty of patience, forget it.

:-(

I don't have the manual, and I haven't found Jakub, but I was hoping to be 
able to diff the last working driver and start from there.

-Aaron Wirtz

Re: socal

[Pete Zaitcev]

On Wed, Dec 31, 2003 at 02:29:26PM -0800, Aaron Wirtz wrote:
> Does anyone know of a patch that fixes the broken socal/fcal drivers under 
> 2.4.x or, failing that, does anyone know what version of the linux kernel and 
> the Solaris microcode Jakub was using in the past (1999) when he had the 
> driver working?

Versions are entirely immaterial. To fix the driver, you
need to read the code and tinker with it. It's not that hard.
But playing lego with components will get you nowhere,
just forget about it.

-- Pete

Re: socal

[Aaron Wirtz]

On Wednesday 31 December 2003 15:35, Pete Zaitcev wrote:
> On Wed, Dec 31, 2003 at 03:19:03PM -0800, Aaron Wirtz wrote:
> > On Wednesday 31 December 2003 15:07, C.Newport wrote:
> > > Have you tried checking the firmware Fcode ?.
> > > http://www.splack.org/fibrechan.html has the procedure for the soc
> > > cards, the socal stuff is very similar.
> >
> > Given that the Solaris driver (and ucode) work with it under Solaris 9, I
> > don't think the fcode version is currently an issue.  One thing I had
> > considered was that if I have to use an older ucode version, I may have
> > to roll back the fcode to match.
>
> This just makes no sense. FCode carries its own ucode,
> which has nothing to do with the one a driver carries.
> For one thing, FCode always carries a debugging ucode,
> which tops at about 1000 IOPS. A driver carries a normal
> microcode, which reaches 5500 IOPS with small blocks,
> 17000 IOPS with null ops (for IP-over-SCSI, for instance).
> They are different and do not have to match, even their
> ABI may be slightly different.

Oh.  So the ucode is a full replacement for whatever fcode is installed then?  
That would mean that the soc driver would normally use the fcode (because it 
doesn't load the ucode unless you compile it in), while the socal driver 
always uses ucode (because it never worked with the fcode, so you have to 
compile it in).  That means if I get the driver working with the ucode 
compiled in, I don't have to worry about the fcode at all?

-Aaron Wirtz

Re: socal

[Aaron Wirtz]

On Wednesday 31 December 2003 15:43, Pete Zaitcev wrote:
> Versions are entirely immaterial. To fix the driver, you
> need to read the code and tinker with it. It's not that hard.
> But playing lego with components will get you nowhere,
> just forget about it.

I beg to differ.  Versions are very important (at least to me) here:  one of 
them worked, and I want to either use that version until the current one is 
fixed or see what has changed so that a have a place to start working on 
fixing whatever has broken since it last worked.
I have a high suspicion that the reason the current drivers don't work may not 
be the version of the drivers, but the version of the ucode that I am trying 
to use with them.  If the ucode I am using implememnts an API other than what 
whe driver expects, that might be the issue.

-Aaron Wirtz

Re: socal

[C.Newport]

On Wednesday 31 December 2003 11:57 pm, Aaron Wirtz wrote:

> I have a high suspicion that the reason the current drivers don't work may
> not be the version of the drivers, but the version of the ucode that I am
> trying to use with them.  If the ucode I am using implememnts an API other
> than what whe driver expects, that might be the issue.

The fcode and ucode will not have changed significantly.
I would rather suspect that the drivers have not been updated to
keep up with the 2.4.x kernel changes.
drivers/fc4/socal.c was last updated in Feb 1999 which puts it
back in 2.2.x land somewhere.

There are also problems with soc.c in 2.4.x but I have not looked
at this because my SS1000E has other issues with 2.4 anyway
so it is still on 2.2.20

BTW, Pete, is there any chance that the sun4d issues might come
to the top of the todo list sometime ?.
It would be nice to get these boatanchors working, especially in
view of Solaris 8 being the last supported release for them.

Re: socal

[C.Newport]

On Wednesday 31 December 2003 11:43 pm, Aaron Wirtz wrote:

> I don't have the manual, and I haven't found Jakub, but I was hoping to be
> able to diff the last working driver and start from there.

The drivers give an (old?) address of jj (at) ultra.linux.cz
Google shows him posting to lkml recently from jakub (at) redhat.com

HTH.

Re: socal

[Aaron Wirtz]

On Wednesday 31 December 2003 17:26, C.Newport wrote:
> On Wednesday 31 December 2003 11:43 pm, Aaron Wirtz wrote:
> > I don't have the manual, and I haven't found Jakub, but I was hoping to
> > be able to diff the last working driver and start from there.
>
> The drivers give an (old?) address of jj (at) ultra.linux.cz
> Google shows him posting to lkml recently from jakub (at) redhat.com
>
> HTH.

I tried both of those a month ot two ago, but still no response.

-Aaron Wirtz

Re: socal

[C.Newport]

On Thursday 01 January 2004 1:34 am, you wrote:

> I tried both of those a month ot two ago, but still no response.

Just a thought :-

The E3500 can boot from fcal, so the required code is in the OBP.
You could probably hack up a driver by calling the OBP code.
This would most probably be slow, but it is a start in the right 
direction.
The OBP is quite well documented and most of the code is in
Forth, so it should be easy to examine.

Re: socal

[Pete Zaitcev]

On Thu, Jan 01, 2004 at 02:06:56AM +0000, C.Newport wrote:
> The E3500 can boot from fcal, so the required code is in the OBP.
> You could probably hack up a driver by calling the OBP code.

Anything can boot from the SOCAL card, even sun4d. In fact it's
exactly the same FCode, and even more - it's exactly the same binary
image! The build procedure for the Sunfire OBP builds an FCode as
if it were for the card, then incorporates it into OBP as a binary
with a little od(1) and ld(1) magic.

Calling OBP driver is probably not a good idea, it's beyond
slow. It's glacial. Also, its error recovery is not stellar.
I hardened it a bit when I took over from Ping, but there are
limits to what you could do to FCode. I advise against it.

-- Pete

Re: socal

[C.Newport]

On Thursday 01 January 2004 2:41 am, Pete Zaitcev wrote:
> On Thu, Jan 01, 2004 at 02:06:56AM +0000, C.Newport wrote:
> > The E3500 can boot from fcal, so the required code is in the OBP.
> > You could probably hack up a driver by calling the OBP code.
>
> Anything can boot from the SOCAL card, even sun4d. In fact it's
> exactly the same FCode, and even more - it's exactly the same binary
> image! The build procedure for the Sunfire OBP builds an FCode as
> if it were for the card, then incorporates it into OBP as a binary
> with a little od(1) and ld(1) magic.
>
> Calling OBP driver is probably not a good idea, it's beyond
> slow. It's glacial. Also, its error recovery is not stellar.
> I hardened it a bit when I took over from Ping, but there are
> limits to what you could do to FCode. I advise against it.

Agreed, I was thinking in terms of a temporary driver as an aid
to understanding how to build a production version.

Re: socal

[Aaron Wirtz]

On Wednesday 31 December 2003 16:26, Keith M Wesolowski wrote:
> On Wed, Dec 31, 2003 at 04:19:24PM -0800, Aaron Wirtz wrote:
> > here's the procedure I'm using:
> > (socal is the driver copied from solaris)
> >
> > $ nm socal
> > ---unimportant lines removed---
> > 0000360c t socal_transport_poll
> > 000002a0 D socal_ucode
> > 000102a0 D socal_ucode_end
> > 00004f0c t socal_us_els
> > ---unimportant lines removed---
> > $ dd bs=1 skipg2 counte536 if=socal of=socal.ucode
> > 65536+0 records in
> > 65536+0 records out
> > 65536 bytes transferred in 2.512516 seconds (26084 bytes/sec)
> >
> >
> > Then I used Jakub's sed command to make the header.
> > If I hexedit the socal.ucode file I see the same thing as I see in the .h
> > file, so I know that the sed step works fine.
>
> It's wrong.  An offset of 2a0 into the .rodata section is not the same
> as an offset of 2a0 into the binary on disk.  Use readelf and/or
> objdump to figure out where the ucode actually starts.
>
> Also, the stuff you'll get with his script is in bytes, so it should
> be u8 not u32.

I am using bs=1, so it is counting in bytes.

After looking more closely, I realized that you are right; I was using the 
data offset, not the file offset.  After a bit of fiddling, I have a command 
that seems to extract the ucode properly from the solaris binary driver.  If 
anyone wants to incorporate this into a FAQ, be my guest. (Or if I ever get 
the whole thing working, I'll write a HOWTO.)

perl -e'($o,$s,$l)=(split /\s+/,(grep{/\s$ARGV[1]$/}(split /\n/,
`objdump -t $ARGV[0]`))[0])[0,3,4];$o=hex($o)+hex((split /\s+/,(grep{
/\s$s\s/}(split /\n\s*/,`objdump -h $ARGV[0]`))[0])[5]);open I,$ARGV[0];
seek I,$o,0;read I,$d,hex($l);$d=~s/\x00+$//s;open O,">$ARGV[2]";print
O "static __u32 socal_ucode[] __initdata = {\n";for(split "",$d){print
O "0x".unpack("h2",$_).(++$c%16=0?",\n":",")}print O "};\n"' \
socal socal_ucode socal_asm.h

The last three arguments specify input file, the symbol name, and the output 
file respectively.  This will strip null padding as well.  If you want to use 
it for other purposes, removing the "$d=~s/\x00+$//s;" command will leave 
null padding on the data.

I will do a compile and will try the driver on Monday.

Thanks.

-Aaron Wirtz

Re: socal

[Aaron Wirtz]

[-- Attachment #1: Type: text/plain, Size: 570 bytes --]

On Wednesday 31 December 2003 19:10, Aaron Wirtz wrote:
> I will do a compile and will try the driver on Monday.

Well, (as expected) this didn't work.  The machine freezes when the module 
loads.  I at least got the module to compile without complaining with the 
debugging turned on (patch to fix the debugging calls and the ucode size is 
attached).  I also got the ucode extraction working right (new script 
attached; cleaner, fixed endianness and datasize bugs).  I'll have to try the 
2.2.x kernel version next. (to see if it is just a 2.4 issue).

-Aaron Wirtz


[-- Attachment #2: socal.patch --]
[-- Type: text/x-diff, Size: 1932 bytes --]

--- linux-2.4.23-old/drivers/fc4/socal.c	2003-06-13 07:51:33.000000000 -0700
+++ linux-2.4.23/drivers/fc4/socal.c	2004-01-05 14:10:48.000000000 -0800
@@ -82,7 +82,7 @@
 }
 
 #ifdef HAVE_SOCAL_UCODE
-static void socal_bzero(unsigned long xram, int size)
+static void socal_bzero(unsigned long xram, unsigned long size)
 {
 	while (size) {
 		sbus_writel(0, xram);
@@ -229,7 +229,7 @@
 	SOCAL_SETIMASK(s, s->imask & ~(cmd & SOCAL_CMD_REQ_QALL));
 	SOD(("imask %08x %08x\n", s->imask, sbus_readl(s->regs + IMASK)));
 
-	SOD(("Queues available %08x OUT %X\n", cmd, s->regs->reqpr[0]))
+	SOD(("Queues available %08x OUT %X\n", cmd, (u32)(s->regs + REQP)));
 	if (s->port[s->curr_port].fc.state != FC_STATE_OFFLINE) {
 		fcp_queue_empty ((fc_channel *)&(s->port[s->curr_port]));
 		if (((s->req[1].in + 1) & s->req[1].last) != (s->req[1].out))
@@ -447,7 +447,7 @@
 		qno = 1;
 	else
 		qno = 0;
-	SOD(("Putting a FCP packet type %d into hw queue %d\n", fcmd->proto, qno))
+	SOD(("Putting a FCP packet type %d into hw queue %ld\n", fcmd->proto, qno))
 	if (s->imask & (SOCAL_IMASK_REQ_Q0 << qno)) {
 		SOD(("EIO %08x\n", s->imask))
 		return -EIO;
@@ -457,7 +457,7 @@
 	
 	if (cq_next_in == sw_cq->out &&
 	    cq_next_in == (sw_cq->out = sbus_readb(s->regs + REQP + qno))) {
-		SOD(("%d IN %d OUT %d LAST %d\n",
+		SOD(("%ld IN %d OUT %d LAST %d\n",
 		     qno, sw_cq->in,
 		     sw_cq->out, sw_cq->last))
 		SOCAL_SETIMASK(s, s->imask | (SOCAL_IMASK_REQ_Q0 << qno));
@@ -605,7 +605,7 @@
 static inline void socal_download_fw(struct socal *s)
 {
 #ifdef HAVE_SOCAL_UCODE
-	SOD(("Loading %ld bytes from %p to %p\n", sizeof(socal_ucode), socal_ucode, s->xram))
+	SOD(("Loading %ld bytes from %p to %p\n", sizeof(socal_ucode), socal_ucode, (u32 *)s->xram))
 	socal_copy_to_xram(s->xram, socal_ucode, sizeof(socal_ucode));
 	SOD(("Clearing the rest of memory\n"))
 	socal_bzero (s->xram + sizeof(socal_ucode), 65536 - sizeof(socal_ucode));

[-- Attachment #3: getucode.pl --]
[-- Type: text/x-perl, Size: 1065 bytes --]

#!/usr/bin/perl
use warnings;
use strict;

unless(scalar(@ARGV) == 3)
{
    print "Usage: getucode.pl inputfile symbol outputfile\n";
    print "e.g.: getucode.pl socal socal_ucode socal_asm.h\n";
    exit;
}

my @symbols = split(/\n/, `objdump -t $ARGV[0]`);
@symbols = grep{/\s$ARGV[1]$/}(@symbols);
my($offset, $section, $length) = (split(/\s+/, $symbols[0]))[0, 3, 4];

my @sections = split(/\n\s*/, `objdump -h $ARGV[0]`);
@sections = grep{/\s$section\s/}(@sections);
$section = (split(/\s+/, $sections[0]))[5];

$offset = hex($offset) + hex($section);
$length = hex($length);

open INPUT, $ARGV[0];
seek INPUT, $offset, 0;
my $data;
read INPUT, $data, hex($length);
close INPUT;

# this is necessary because socal.c copies the data 4 bytes at a time,
# so the length needs to be a multiple of 4.
while($data =~ s/\x00{4}$//s){}

open OUTPUT, ">$ARGV[2]";
print OUTPUT "static __u8 socal_ucode[] __initdata = {\n";
my $counter = 0;
for(split '',$data)
{
    print OUTPUT '0x'.unpack('H2', $_).(++$counter%16==0?",\n":',');
}
print OUTPUT "};\n";
close OUTPUT;