Need to match driver to microcode?

Need to match driver to microcode?

[grarpamp]

For the same core number, is it necessary to update
the b43 driver when a new microcode is added? Or
does the API/addressing into the microcode stay
the same across microcode updates?

ie: Were any updates to the driver needed to support
the span from say 478.104 to 784.2 of the microcode
for any of the cores already present in 478.104?

I'm not seeing anything obvious to that effect in the
short summaries here:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/net/wireless/b43

Need to match driver to microcode?

[Larry Finger]

On 02/05/2014 04:58 PM, grarpamp wrote:
> For the same core number, is it necessary to update
> the b43 driver when a new microcode is added? Or
> does the API/addressing into the microcode stay
> the same across microcode updates?
>
> ie: Were any updates to the driver needed to support
> the span from say 478.104 to 784.2 of the microcode
> for any of the cores already present in 478.104?
>
> I'm not seeing anything obvious to that effect in the
> short summaries here:
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/net/wireless/b43

Throughout the life of the b43 driver, there have been changes in the microcode 
that required changes in the driver; however all of them predate version 
478.104. With that said, I have not tested anything newer than 666.x. Where did 
you get version 784.2?

Larry

Need to match driver to microcode?

[grarpamp]

On Wed, Feb 5, 2014 at 6:57 PM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
> Throughout the life of the b43 driver, there have been changes in the
> microcode that required changes in the driver; however all of them predate
> version 478.104.

So from 478.104 through 666.2 b43 driver stayed
the same in relation to simply cutting, dropping
in and working with any of the 7 new microcodes
post 478.104?

Can you link to a pre 478.104 example from the repo?
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/net/wireless/b43


> With that said, I have not tested anything newer than
> 666.x. Where did you get version 784.2?

You added it to fwcutter here... :)

http://bues.ch/gitweb?p=b43-tools.git;a=commit;h=f9ea2d59b33b552994a9c7ad3ca8fa1ef9992b95

Need to match driver to microcode?

[Larry Finger]

On 02/05/2014 06:54 PM, grarpamp wrote:
> On Wed, Feb 5, 2014 at 6:57 PM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
>> Throughout the life of the b43 driver, there have been changes in the
>> microcode that required changes in the driver; however all of them predate
>> version 478.104.
>
> So from 478.104 through 666.2 b43 driver stayed
> the same in relation to simply cutting, dropping
> in and working with any of the 7 new microcodes
> post 478.104?

If you had one of the newer cores, then you needed newer firmware to get the 
correct file, but the older ones worked. However, I should never trust my 
memory. The TX and RX headers changed with firmware version 598.314. The change 
in the driver was introduced with commit 17030f4 in such a way that the older 
firmware versions still worked.

> Can you link to a pre 478.104 example from the repo?
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/net/wireless/b43

The two commits are at the bottom of the page 
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/net/wireless/b43?ofs=200.
>
>> With that said, I have not tested anything newer than
>> 666.x. Where did you get version 784.2?
>
> You added it to fwcutter here... :)
>
> http://bues.ch/gitweb?p=b43-tools.git;a=commit;h=f9ea2d59b33b552994a9c7ad3ca8fa1ef9992b95

Ah, I know it by a different number.

Larry

Need to match driver to microcode?

[grarpamp]

On Wed, Feb 5, 2014 at 8:50 PM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
> If you had one of the newer cores, then you needed newer firmware to get the
> correct file, but the older ones worked. However, I should never trust my
> memory.

Correct my understanding... 'cores' refers to physical hardware
devices/revisions from Broadcom? So if I 'had newer cores', I'd need
to find newer 'firmware aka: microcode' to support them. Because
'older ones aka: firmware files' would not in fact work for me
since they would not have support for the new physical core
I posess? ie: I see newer files from say Linksys have more
ucodeN.fw N'umbers' available inside them than their older files..

> The TX and RX headers changed with firmware version 598.314.

598.314 is not now in fwcutter, was it at one time present and
then removed?

> change in the driver was introduced with commit 17030f4 in such a way that
> the older firmware versions still worked.
>
> The two commits are at the bottom of the page
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/net/wireless/b43?ofs=200.

Looking them over.

> Ah, I know it by a different number.

What number is that?
How does something like '6.30.102.9 (r366174)' on the wrapper
relate to its corresponding internal '784.2'? What are those
strings each describing?

And is there a supposed changelog, perhaps but
not necessarily from Broadcom, for what changed
in the different firmwares that we find?

Thanks.

Need to match driver to microcode?

[Rafał Miłecki]

2014-02-06 grarpamp <grarpamp@gmail.com>:
>> Ah, I know it by a different number.
>
> What number is that?
> How does something like '6.30.102.9 (r366174)' on the wrapper
> relate to its corresponding internal '784.2'? What are those
> strings each describing?

Broadcom's closed source driver embeds firmware in the driver. They
don't request it from the user space (ex. from /lib/firmware/).
However they develop driver and firmware independently, so they use
different numbering for them.

You can't really say what firmware version is embedded in the driver
until you load this firmware to the WiFi card and read the version
back.

What we do is downloading closed source driver (that was compiled for
some completely different system/architecture) and extract firmware
from it (which is a tiny part of the total size). We extract it to the
/lib/firmware/ and then b43 can use it.

In the past there were only 2 noticeable changes in the driver <->
firmware communication. It results in some 3 different code paths in
the b43 (we didn't drop support for the old firmwares yet).

-- 
Rafa?

Need to match driver to microcode?

[grarpamp]

On Thu, Feb 6, 2014 at 1:10 AM, Rafa? Mi?ecki <zajec5@gmail.com> wrote:
>> How does something like '6.30.102.9 (r366174)' on the wrapper
>> relate to its corresponding internal '784.2'? What are those
>> strings each describing?
>
> Broadcom's closed source driver embeds firmware in the driver. They
> don't request it from the user space (ex. from /lib/firmware/).
> However they develop driver and firmware independently, so they use
> different numbering for them.
>
> You can't really say what firmware version is embedded in the driver
> until you load this firmware to the WiFi card and read the version
> back.
>
> What we do is downloading closed source driver (that was compiled for
> some completely different system/architecture) and extract firmware
> from it (which is a tiny part of the total size). We extract it to the
> /lib/firmware/ and then b43 can use it.

I should say i understand Broadcom makes a closed source
linux driver:
http://www.broadcom.com/support/802.11/linux_sta.php
which I don't think includes the card firmware, but I'll
try to unpack it again and look closer. Even if it does
i don't see any reference on b43 site to fw ever being
cut out of it and used with b43/insmod, only to blobs
from GPL being used that way.

And I know the firmware the open b43 driver uses comes
from unpacking GPL releases from people like Linksys.

What I didn't understand was if I run strings on wl_apsta.o
I get the first number, and b43 prints the second number
when loading the firmware. I didn't get what those different
numbers in the same .o/.fw file represented. And I'm
not sure if the first number is present in the loaded firmware
to also be printed on load.

It sounds like the 784.2 is Broadcom microcode rev
and the 6.30.102.9 (rN) is their driver rev (which seemed
a bit odd to find the latter in wl_apsta.o
which i don't think functions as a driver to Linksys,
only as a firmware blob). It could be just a build chain
note or artifact. I suppose apsta could be a driver
object too, but haven't looked that far into the GPL kits.

Now I get what those two numbers are, driver and firmware,
respectively I think.

> In the past there were only 2 noticeable changes in the driver <->
> firmware communication. It results in some 3 different code paths in
> the b43 (we didn't drop support for the old firmwares yet).

Ok, so it sounds like it's not necessarily always just matter of
adding a newer firmware like 7mm.nnn to fwcutter_list.h and
then insmod for b43. There could be other things todo there.

Need to match driver to microcode?

[Rafał Miłecki]

2014-02-06 9:19 GMT+01:00 grarpamp <grarpamp@gmail.com>:
> On Thu, Feb 6, 2014 at 1:10 AM, Rafa? Mi?ecki <zajec5@gmail.com> wrote:
>>> How does something like '6.30.102.9 (r366174)' on the wrapper
>>> relate to its corresponding internal '784.2'? What are those
>>> strings each describing?
>>
>> Broadcom's closed source driver embeds firmware in the driver. They
>> don't request it from the user space (ex. from /lib/firmware/).
>> However they develop driver and firmware independently, so they use
>> different numbering for them.
>>
>> You can't really say what firmware version is embedded in the driver
>> until you load this firmware to the WiFi card and read the version
>> back.
>>
>> What we do is downloading closed source driver (that was compiled for
>> some completely different system/architecture) and extract firmware
>> from it (which is a tiny part of the total size). We extract it to the
>> /lib/firmware/ and then b43 can use it.
>
> I should say i understand Broadcom makes a closed source
> linux driver:
> http://www.broadcom.com/support/802.11/linux_sta.php
> which I don't think includes the card firmware, but I'll
> try to unpack it again and look closer. Even if it does
> i don't see any reference on b43 site to fw ever being
> cut out of it and used with b43/insmod, only to blobs
> from GPL being used that way.

It has to include firmware, it's just (as I said) compiled into the binary.

We don't extract firmware from Linux STA because it wasn't available
for a long time, it was only recently updated and you can usually find
newer firmware in routers (MIPS/ARM) drivers.


> What I didn't understand was if I run strings on wl_apsta.o
> I get the first number, and b43 prints the second number
> when loading the firmware. I didn't get what those different
> numbers in the same .o/.fw file represented. And I'm
> not sure if the first number is present in the loaded firmware
> to also be printed on load.

I told you. The other number (784.2, etc.) is what firmware reports.
You have to upload firmware to the WiFi card and read it back. Since
the whole firmware in a blob (embedded in the driver), above number
(784.2) isn't visible as a string.

Need to match driver to microcode?

[Larry Finger]

On 02/05/2014 10:34 PM, grarpamp wrote:
> On Wed, Feb 5, 2014 at 8:50 PM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
>> If you had one of the newer cores, then you needed newer firmware to get the
>> correct file, but the older ones worked. However, I should never trust my
>> memory.
>
> Correct my understanding... 'cores' refers to physical hardware
> devices/revisions from Broadcom? So if I 'had newer cores', I'd need
> to find newer 'firmware aka: microcode' to support them. Because
> 'older ones aka: firmware files' would not in fact work for me
> since they would not have support for the new physical core
> I posess? ie: I see newer files from say Linksys have more
> ucodeN.fw N'umbers' available inside them than their older files..

The Broadcom devices consist of a number of different individual units with an 
interconnect. These units are the cores. The firmware files needed are 
determined by the revision number of the 802.11 or PHY core. As Broadcom 
develops new versions of the PHY core, the revision numbers get incremented. If 
you have a PHY core newer than anything supported by b43, it would not matter if 
the firmware for that chip is available or not, b43 would not work. Yes, there 
are newer versions of ucodeN, but it does not matter unless someone does the 
reverse engineering to see what is needed to make the device work.

>> The TX and RX headers changed with firmware version 598.314.
>
> 598.314 is not now in fwcutter, was it at one time present and
> then removed?

No, someone noticed that the descriptors changed with that version. We either 
never found a file containing that version, or a newer revision was found first.

>> change in the driver was introduced with commit 17030f4 in such a way that
>> the older firmware versions still worked.
>>
>> The two commits are at the bottom of the page
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/net/wireless/b43?ofs=200.
>
> Looking them over.
>
>> Ah, I know it by a different number.
>
> What number is that?

6.30.163.46

> How does something like '6.30.102.9 (r366174)' on the wrapper
> relate to its corresponding internal '784.2'? What are those
> strings each describing?

There is no one to one relationship. Both are internal Broadcom designations.

> And is there a supposed changelog, perhaps but
> not necessarily from Broadcom, for what changed
> in the different firmwares that we find?

No. The firmware is just a black box.

Larry

Need to match driver to microcode?

[grarpamp]

On Thu, Feb 6, 2014 at 2:22 AM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
> The Broadcom devices consist of a number of different individual units with
> an interconnect. These units are the cores. The firmware files needed are
> determined by the revision number of the 802.11 or PHY core. As Broadcom
> develops new versions of the PHY core, the revision numbers get incremented.
> If you have a PHY core newer than anything supported by b43, it would not
> matter if the firmware for that chip is available or not, b43 would not
> work.

I think I see this as ucodeN N based conditionals in the b43 driver.
Guess the driver probes the card to find the core rev Ns then
picks the matching Ns out of the big insmodded set of all N and
pushes them to the card.

I should look for a way to print the individual core revs to
the [name][N].fw files picked by b43 when loaded. Would make
comparing any fwcutter output changes from different apsta.o's
for the card you have a bit more clear. Should I expect which,
if any, of the x's labeled below to match the N's above?

Linux version 3.12.9
b43-phy0: Broadcom 43xx WLAN found (core revision x)
b43-phy0: Found PHY: Analog x, Type x (LP), Revision x
b43-phy0: Loading firmware version xxx.x (xxxx-xx-xx xx:xx:xx)

> Yes, there are newer versions of ucodeN, but it does not matter unless
> someone does the reverse engineering to see what is needed to make the
> device work.


>>> Ah, I know it by a different number.
>> What number is that?
> 6.30.163.46

that apsta file surrounds todays 784.2...

> There is no one to one relationship. Both are internal Broadcom
> designations.

... got it thx.

> No. The firmware is just a black box.

Hopefully these makers will someday stop software
secrets. They sell primary hardware, not sw.

Need to match driver to microcode?

[Larry Finger]

On 02/06/2014 03:01 AM, grarpamp wrote:
> On Thu, Feb 6, 2014 at 2:22 AM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
>
>> There is no one to one relationship. Both are internal Broadcom
>> designations.
>
> ... got it thx.
>
>> No. The firmware is just a black box.
>
> Hopefully these makers will someday stop software
> secrets. They sell primary hardware, not sw.

The firmware source would reveal trade secrets about the chip construction that 
a competitor could use. For that reason, I don't expect many of the vendors to 
publish those sources.

Most vendors license their binary firmware files for redistribution, and they 
are available in the linux-firmware git repo. That is where the distros get 
firmware. Broadcom does the same for devices driven by brcmsmac and brcmfmac.

For the b43-driven devices, the rules are different. Broadcom will not allow 
their firmware files to be redistributed under any circumstances. Any web site 
that contains files such as ucodeN, etc. is illegal and the operator of that 
site could be sued for illegally distributing proprietary material!

To get the firmware we have, we take advantage of a loophole. Because Broadcom 
uses Linux for the OS in their routers, the GPL requires them to make their 
contributions available for redistribution. Of course, they only provide binary 
blobs for their driver, but the firmware is contained within those binaries. 
Thus we need to find a binary driver that is redistributable, determine where 
the firmware is contained within that binary, and modify fwcutter to extract the 
firmware files from that blob.

Not all of Broadcom's drivers are suitable. For example, it has never seemed 
fruitful to download 300-500 MB of stuff to extract approximately 1 MB of 
firmware. My repository contains files that are no larger than ~10 MB. Windows 
drivers are stripped to the point that the firmware locations are not easily 
determined, thus I avoid them.

Larry

Need to match driver to microcode?

[grarpamp]

On Thu, Feb 6, 2014 at 11:54 AM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
> Because Broadcom uses Linux for the OS in their routers

Didn't know they made consumer routers or published any
GPL packages, I think you meant people like Linksys do, not
Broadcom. I'll search broadcom site more for this though.

> Not all of Broadcom's drivers are suitable. For example, it has never seemed
> fruitful to download 300-500 MB of stuff to extract approximately 1 MB of
> firmware. My repository contains files that are no larger than ~10 MB.

If it's critical download/space issue for devs to check for new firmware
revs, I'll download and unpack any links to big GPL zipfiles that people
send me. The 10MB was first reached with some microcode 5xx and
up, now it's at least 20MB, but the links are in fwcutter_list.h for people.

As usual, 7z compresses apsta.o the best to under 20% original
size, so under 6MB for all those redistributable apsta.o's.

> Windows drivers are stripped to the point that the firmware locations are
> not easily determined, thus I avoid them.

Wondered about that since all I've seen are the GPL apsta.o ones
that objdump probably only works with (never tried to find any windows
drivers with fw inside to try to mklist them).

Anyway, my question was mostly about what the fw/mic xxx.xx
and driver x.x.x.x number relation was (none really), and whether
any new fw was always simply cuttable, loadable and expected
to work as such (not necessarily). I get that now. Thanks.

Need to match driver to microcode?

[Chris Adams]

Once upon a time, grarpamp <grarpamp@gmail.com> said:
> On Thu, Feb 6, 2014 at 11:54 AM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
> > Because Broadcom uses Linux for the OS in their routers
> 
> Didn't know they made consumer routers or published any
> GPL packages, I think you meant people like Linksys do, not
> Broadcom. I'll search broadcom site more for this though.

I don't think Broadcom sells anything complete devices directly, but
Linksys, Netgear, D-Link, etc. use Broadcom chips inside.  Broadcom has
reference designs, with Linux code ready for customization, and the
retail vendors brand it, tweak some features if they want, box it up,
and sell it.

Broadcom makes lots of chips used in a wide variety of consumer
electronics devices, including network gear (such as routers, APs,
switches), DVRs, Blu-Ray players, and TVs.

-- 
Chris Adams <cma@cmadams.net>