AIC94xx SAS question.

AIC94xx SAS question.

[Wakko Warner]

This is a general question and probably doesn't matter which kernel
(although the one in question is 2.6.23 vanilla)

I know that this device requires firmware to operate which has to be loaded
when the driver is loaded.

I'd like to have this driver compiled into the kernel.  My question is:

Can either of these be done:
1) Embed the firmware into the driver so that it can be used before the
rootfs id mounted.
2) Can the firmware be loaded in an initr{d,amfs}?  I personnaly despise
initr{d,amfs} and if this option is used, I'd prefer it not to be kernel
specific (as in having the driver for the card in the initr{d,amfs}) this
way a kernel upgrade wouldn't change the initr{d,amfs}.  I know there are
tools for this kind of thing, but I find it quite pointless to have the
driver in 2 places (in the init{r,amfs} and in /lib/modules)  I also realize
that "disks are cheap" blah blah, but that's not the point.

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals
 Got Gas???

Re: AIC94xx SAS question.

[Keith Hopkins]

On 04/22/2008 07:19 AM, Wakko Warner wrote:
> Can either of these be done:
> 1) Embed the firmware into the driver so that it can be used before the
> rootfs id mounted.

I think this is generally not done because firmware tends to be closed-source.

> 2) Can the firmware be loaded in an initr{d,amfs}?

Sure.  openSuSE does it in their mkinitrd.

--Keith

Re: AIC94xx SAS question.

[Wakko Warner]

Keith Hopkins wrote:
> On 04/22/2008 07:19 AM, Wakko Warner wrote:
> >Can either of these be done:
> >1) Embed the firmware into the driver so that it can be used before the
> >rootfs id mounted.
> 
> I think this is generally not done because firmware tends to be 
> closed-source.

I can understand this from the point of the distrobution of the linux
kernel, but it still can be done.  If there were a patch or something that
would convert the file, that's all I care.  This for me would be preferred.

> >2) Can the firmware be loaded in an initr{d,amfs}?
> 
> Sure.  openSuSE does it in their mkinitrd.

I'm not sure if you got the point of #2 or not.  I do *NOT* want the driver
as a module, I want it compiled into the kernel and then load the firmware
from an initramfs.

I think I figured out how to do #2, but if/when I do it is another story.  I
believe that the driver won't bind to the card if the firmware isn't
available, thus I can have it bind to the card at a later time (during
initramfs).  I tried this on my system last night (since it crashed anyway). 
I loaded the module and /sys wasn't mounted.  I mounted /sys and told the
driver to bind my card and it worked.

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals
 Got Gas???

Re: AIC94xx SAS question.

[James Bottomley]

On Sat, 2008-04-26 at 10:56 -0400, Wakko Warner wrote:
> Keith Hopkins wrote:
> > On 04/22/2008 07:19 AM, Wakko Warner wrote:
> > >Can either of these be done:
> > >1) Embed the firmware into the driver so that it can be used before the
> > >rootfs id mounted.
> > 
> > I think this is generally not done because firmware tends to be 
> > closed-source.
> 
> I can understand this from the point of the distrobution of the linux
> kernel, but it still can be done.  If there were a patch or something that
> would convert the file, that's all I care.  This for me would be preferred.

There is certainly a way to do this: all you really need to do is to
render the firmware into hex as a character array, compile the array
with the driver and return a pointer to the first byte at the
request_firmware call.

The practical reason it's no-longer done (apart from the annoyance it
causes debian) is simply that the file is huge and the firmware seems to
change quite a lot.  For the qla2xxx driver (whose firmware, admittedly
is larger than aic94xx) it used to result in a 4MB patch every time
there was a tiny change to the file.

> > >2) Can the firmware be loaded in an initr{d,amfs}?
> > 
> > Sure.  openSuSE does it in their mkinitrd.
> 
> I'm not sure if you got the point of #2 or not.  I do *NOT* want the driver
> as a module, I want it compiled into the kernel and then load the firmware
> from an initramfs.
> 
> I think I figured out how to do #2, but if/when I do it is another story.  I
> believe that the driver won't bind to the card if the firmware isn't
> available, thus I can have it bind to the card at a later time (during
> initramfs).  I tried this on my system last night (since it crashed anyway). 
> I loaded the module and /sys wasn't mounted.  I mounted /sys and told the
> driver to bind my card and it worked.

James

Re: AIC94xx SAS question.

[Wakko Warner]

James Bottomley wrote:
> > I can understand this from the point of the distrobution of the linux
> > kernel, but it still can be done.  If there were a patch or something that
> > would convert the file, that's all I care.  This for me would be preferred.
> 
> There is certainly a way to do this: all you really need to do is to
> render the firmware into hex as a character array, compile the array
> with the driver and return a pointer to the first byte at the
> request_firmware call.

I know I can make a .h file with the firmware in it.  I might beable to do
the other change as well, not sure yet.  I'm not a kernel hacker.

> The practical reason it's no-longer done (apart from the annoyance it
> causes debian) is simply that the file is huge and the firmware seems to
> change quite a lot.  For the qla2xxx driver (whose firmware, admittedly
> is larger than aic94xx) it used to result in a 4MB patch every time
> there was a tiny change to the file.

It would be nice if someone had a patch that would include a .h file to the
driver and the .h can be generated from whatever firmware is wanted.

If I ever decide to redesign my system, I may have a go at this.

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals
 Got Gas???

Re: AIC94xx SAS question.

[Keith Hopkins]

On 04/26/2008 10:56 PM, Wakko Warner wrote:
> Keith Hopkins wrote:
>> On 04/22/2008 07:19 AM, Wakko Warner wrote:
>>> Can either of these be done:
>>> 1) Embed the firmware into the driver so that it can be used before the
>>> rootfs id mounted.
>> I think this is generally not done because firmware tends to be 
>> closed-source.
> 
> I can understand this from the point of the distrobution of the linux
> kernel, but it still can be done.  If there were a patch or something that
> would convert the file, that's all I care.  This for me would be preferred.

Sorry, no idea if it can actually be done.

> 
>>> 2) Can the firmware be loaded in an initr{d,amfs}?
>> Sure.  openSuSE does it in their mkinitrd.
> 
> I'm not sure if you got the point of #2 or not.  I do *NOT* want the driver
> as a module, I want it compiled into the kernel and then load the firmware
> from an initramfs.

I think I got you all right.  My answers were just too short :)

SuSE loads the driver as a module and the firmware in initrd.  I haven't tried initramfs, nor compiling it into the kernel.

I unpacked my current initrd, and in it I found:

lib/firmware/aic94xx-seq.fw
+ some ql firmware that I don't need.

I'm not sure if the driver compiled for SuSE "just knows" to look in /lib/firmware, or how it finds the file.  It might have something to do with the firmware_class module in SuSE.
I do remember that I needed to load 'edd' before 94xx, or it would not init the firmware.

As for compiling the driver in the kernel, and putting the firmware in initrd/initramfs, you would have to force the init of the 94xx module to be after the initrd/initramfs was initialized, loaded and mounted.  I have not checked to see if that is currently possible.

--Keith

Re: AIC94xx SAS question.

[Raoul Bhatia [IPAX]]

just for the records

Keith Hopkins wrote:
> SuSE loads the driver as a module and the firmware in initrd.  I haven't
> tried initramfs, nor compiling it into the kernel.
> 
> I unpacked my current initrd, and in it I found:
> 
> lib/firmware/aic94xx-seq.fw
> + some ql firmware that I don't need.
> 
> I'm not sure if the driver compiled for SuSE "just knows" to look in
> /lib/firmware, or how it finds the file.  It might have something to do
> with the firmware_class module in SuSE.
> I do remember that I needed to load 'edd' before 94xx, or it would not
> init the firmware.

this is a default behaviour of this and many other drivers.

> As for compiling the driver in the kernel, and putting the firmware in
> initrd/initramfs, you would have to force the init of the 94xx module to
> be after the initrd/initramfs was initialized, loaded and mounted.  I
> have not checked to see if that is currently possible.

in the end, there is no big deal in putting the firmware into the
initramfs. some distributions may work out of the box, some might
require a little tuning of an initramfs hook.

which leads me to my question: why are you absolutly against an
initramfs image? properly configured - and that is not a big deal
in current versions of mkinitramfs and current distributions - there
are few possibilities to break the setup.

if you require assistance, i can help.

cheers,
raoul
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia@ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office@ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________

Re: AIC94xx SAS question.

[Wakko Warner]

Raoul Bhatia [IPAX] wrote:
> Keith Hopkins wrote:
> > SuSE loads the driver as a module and the firmware in initrd.  I haven't
> > tried initramfs, nor compiling it into the kernel.
> > 
> > I unpacked my current initrd, and in it I found:
> > 
> > lib/firmware/aic94xx-seq.fw
> > + some ql firmware that I don't need.
> > 
> > I'm not sure if the driver compiled for SuSE "just knows" to look in
> > /lib/firmware, or how it finds the file.  It might have something to do
> > with the firmware_class module in SuSE.
> > I do remember that I needed to load 'edd' before 94xx, or it would not
> > init the firmware.
> 
> this is a default behaviour of this and many other drivers.

Which part are you refering to as being default?  Requiring edd or something
else?

> > As for compiling the driver in the kernel, and putting the firmware in
> > initrd/initramfs, you would have to force the init of the 94xx module to
> > be after the initrd/initramfs was initialized, loaded and mounted.  I
> > have not checked to see if that is currently possible.
> 
> in the end, there is no big deal in putting the firmware into the
> initramfs. some distributions may work out of the box, some might
> require a little tuning of an initramfs hook.

I compile kernels specifically for each machine that I have (which is not
that often and not that many).  Some mount NFS root, others have different
drivers required to mount / and other CPU options as well.  Thus I do not
need the extra overhead and complexity of an initramfs.

> which leads me to my question: why are you absolutly against an
> initramfs image? properly configured - and that is not a big deal
> in current versions of mkinitramfs and current distributions - there
> are few possibilities to break the setup.

I've always hated the idea that one would require this just to get their /
mounted.  Granted there are some setups out there that absolutely require
it.  I have one such setup at work, but that system was designed to run out
of ram anyway.

See comment about initramfs above.  Now if I were attempting to mount
NFSroot over wireless, I would have to use an initramfs.  I see no point in
using initramfs for something that can simply be compiled into the kernel. 
I am aware of possible hardware changes (IE going from ata, scsi, etc to
sata or sas) that would require a recompile and I prefer this again over
initramfs.

As for the reason for this thread, the AIC94xx chip requires non-free
firmware.  Right now, my / is on SCSI and is compiled in.  I had considered
re-doing my system and doing away with the SCSI hard drives.  That'd mean
that I'd need the driver in the kernel (Key word is "I").  So how do I go
about getting the firmware to the driver.  My 2 options, if doable, is to
somehow build the firmware into the kernel myself or build an initramfs with
just enough tools to load the firmware into the driver and mount my /.

I still like the "There's more than 1 way to do it" and I'll probably do a
different way for anything =)

> if you require assistance, i can help.

I appreciate the offer.  I think I know what needs to be done if/when I
decide to redesign my system.

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals
 Got Gas???

Re: AIC94xx SAS question.

[Raoul Bhatia [IPAX]]

Wakko Warner wrote:
>>> lib/firmware/aic94xx-seq.fw
...
>>> I'm not sure if the driver compiled for SuSE "just knows" to look in
>>> /lib/firmware, or how it finds the file.  It might have something to do
>>> with the firmware_class module in SuSE.
>>> I do remember that I needed to load 'edd' before 94xx, or it would not
>>> init the firmware.
>> this is a default behaviour of this and many other drivers.
>
> Which part are you refering to as being default?  Requiring edd or
something
> else?

the driver finding the firmware by itself. in my hurry, i missed the
"edd" part ;)

>>> As for compiling the driver in the kernel, and putting the firmware in
>>> initrd/initramfs, you would have to force the init of the 94xx module to
>>> be after the initrd/initramfs was initialized, loaded and mounted.  I
>>> have not checked to see if that is currently possible.
>> in the end, there is no big deal in putting the firmware into the
>> initramfs. some distributions may work out of the box, some might
>> require a little tuning of an initramfs hook.
>
> I compile kernels specifically for each machine that I have (which is not
> that often and not that many).  Some mount NFS root, others have different
> drivers required to mount / and other CPU options as well.  Thus I do not
> need the extra overhead and complexity of an initramfs.

i used to do this too. and on special systems, i still am in favor of
compiling my very specific kernel - maybe even without module loading
support.

but for 99% of the servers i am taking care of, i abandoned this in
favour of simply having to use apt to upgrade these systems.

given my available time vs the time the compilation for the different
systems would need (ok, i know that you can set up some build
environment which compiles a kernel for every type of configuration you
want) i happily accept the loss in performance, etc. that comes with
it ;)

>> which leads me to my question: why are you absolutly against an
>> initramfs image? properly configured - and that is not a big deal
>> in current versions of mkinitramfs and current distributions - there
>> are few possibilities to break the setup.
>
> I've always hated the idea that one would require this just to get their /
> mounted.  Granted there are some setups out there that absolutely require
> it.  I have one such setup at work, but that system was designed to
run out
> of ram anyway.
>
> See comment about initramfs above.  Now if I were attempting to mount
> NFSroot over wireless, I would have to use an initramfs.  I see no
point in
> using initramfs for something that can simply be compiled into the
kernel.
> I am aware of possible hardware changes (IE going from ata, scsi, etc to
> sata or sas) that would require a recompile and I prefer this again over
> initramfs.

i used to think the same. plus i thought about the extra performance of
disabling module support/interfaces plus the additional security which
you gain when there is no module-support compiled into the kernel.

see above for my conclusion.

> As for the reason for this thread, the AIC94xx chip requires non-free
> firmware.  Right now, my / is on SCSI and is compiled in.  I had
considered
> re-doing my system and doing away with the SCSI hard drives.  That'd mean
> that I'd need the driver in the kernel (Key word is "I").  So how do I go
> about getting the firmware to the driver.  My 2 options, if doable, is to
> somehow build the firmware into the kernel myself or build an
initramfs with
> just enough tools to load the firmware into the driver and mount my /.

i know the problem with aic94xx - we initially struggled too because the
aic94xx was the first driver we needed the firmware for to mount /. i'm
still in favour of initramfs as this, imho, made testing easier too ;)

> I still like the "There's more than 1 way to do it" and I'll probably do a
> different way for anything =)

good - feel free to post your patches/changes/etc. ;)

cheers,
raoul
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia@ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office@ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________

Re: AIC94xx SAS question.

[Wakko Warner]

Keith Hopkins wrote:
> On 04/26/2008 10:56 PM, Wakko Warner wrote:
> >I can understand this from the point of the distrobution of the linux
> >kernel, but it still can be done.  If there were a patch or something that
> >would convert the file, that's all I care.  This for me would be preferred.
> 
> Sorry, no idea if it can actually be done.

I believe it can be.  I don't think the driver really cares if the firmware
came from firmware loader or a binary blob compiled into the kernel (so long
as the code is there to do this).

> >I'm not sure if you got the point of #2 or not.  I do *NOT* want the driver
> >as a module, I want it compiled into the kernel and then load the firmware
> >from an initramfs.
> 
> I think I got you all right.  My answers were just too short :)

I'd say so! =)

> SuSE loads the driver as a module and the firmware in initrd.  I haven't 
> tried initramfs, nor compiling it into the kernel.
> 
> I unpacked my current initrd, and in it I found:
> 
> lib/firmware/aic94xx-seq.fw
> + some ql firmware that I don't need.
> 
> I'm not sure if the driver compiled for SuSE "just knows" to look in 
> /lib/firmware, or how it finds the file.  It might have something to do 
> with the firmware_class module in SuSE.
> I do remember that I needed to load 'edd' before 94xx, or it would not init 
> the firmware.

[wakko@vegeta:/home/wakko] lsmod|egrep "edd|aic"
aic94xx                77436  6 
libsas                 46728  1 aic94xx
firmware_class         10880  1 aic94xx
scsi_transport_sas     35456  2 aic94xx,libsas
[wakko@vegeta:/home/wakko] 

Apparently, I didn't have to.  I still use the old hotplug scripts (I
personally hate udev)

> As for compiling the driver in the kernel, and putting the firmware in 
> initrd/initramfs, you would have to force the init of the 94xx module to be 
> after the initrd/initramfs was initialized, loaded and mounted.  I have not 
> checked to see if that is currently possible.

I'm not 100% sure how it would work, I know that if there's no firmware
available at module load time, it'll appear to hang for x number of seconds
waiting for the firmware.  I believe the same will happen if compiled in,
but I've not read the code nor would I understand if I did.  As I said in
another email, I "oopsed" (not a kernel oops!!) and loaded the module w/o
/sys mounted.  I later mounted /sys and told the driver to bind to the PCI
ID of the card.  It happily loaded the firmware that time.  I believe the
same would be true for compiled in.  In this case, a static initramfs
(static meaning no kernel modules and doesn't require updating every kernel
update) would work.  It would make firmware updates easier, but how often
would you update the firmware and not the kernel?

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals
 Got Gas???