Re: ROM Patching (was: [PATCH] bluetooth: remove wrong dependency for BT_ATH3K)

Re: ROM Patching (was: [PATCH] bluetooth: remove wrong dependency for BT_ATH3K)

[Luis R. Rodriguez]

CC'ing linux-wireless as ROM patching is mentioned and in so far as
802.11 mobile is concerned this is a pretty frequent practice there
as well so figured we'd tie in the conversations.

On Tue, Jul 30, 2013 at 03:05:18PM -0700, Marcel Holtmann wrote:
> Hi Luis,
> 
> >> This brings an interesting question: shouldn't the firmware download
> >> part be isolated from the USB driver? After all, I want to
> >> communicate with a UART bluetooth chip.
> > 
> > There are a few BT firmware upload modules (last I checked at least 2),
> > I believe it should be possible to stuff all that code a shared
> > module or even as FreeBSD does it -- treat fw uploading in userspace,
> > however just keep in mind for quirks [0]. So patches welcomed.
> 
> it really depends on what kind of patching or firmware download has to be done.
> 
> For all the ROM patching via HCI commands, we explicitly added support
> for setup stage

This stage is what is I was referring to.

> and exposed a standard way of getting in front of
> Bluetooth core init sequence.

Nice.

> An example here would be the Intel
> Bluetooth devices that require ROM patching. This can be useful for
> many Bluetooth devices that are generally USB transport standard
> compliant, but need a few extra commands of vendor setup.

Interesting. I believe for 802.11 we just throw in a bunch of ROM
patching onto the 802.11 driver_firwmare.fw file and then assume
the driver can do the right thing. I however haven't worked on that so I
am curious if someone more familiar with this can provide details of
how that happens. I do know that the 802.11 mobile drivers get tons of
firmware updates because of this exact architectural choice and do
wonder how we can easily keep up with the firmware updates in meaninful
way to linux-firwmare.

How often do the ROM patches get updated? And do the HCI commands
trigger a userspace event?

> We are planning to take one extra step and split this into a
> mini-driver approach similar to what has been done for usbnet, but we
> are not there yet.

Neat. Perhaps we need something that we can share with 802.11 or other
hardare I highly doubt we're the only ones patching ROM. Don't we even
patch up core CPUs? I'm wondering if firmware_class could be expanded to
support serialized ROM patching. The biggest hurdle I see with splititng
ROM patching from a single firmware is serializing that, addressing
revision dependencies and of course kernel dependencies.

I've thrown this on the list of topics for the next wireless summit
at New Orleans.

> However the ROM patching drivers need to be in the kernel since
> otherwise they will race with the core init sequence.

Sure and depending on the architecture -- if this is kicked off to
userspace helpers or not then we may need to consider dbus in
kernel thing to help with speed / races, dependenecies / async
loading, -EPROBE_DEFER, etc.

> There are also just firmware download drivers that do nothing else
> than just loading the firmware. And here it can be done via userspace
> or kernel space. In the Bluetooth world we have seen both, but
> generally the kernel ones stayed around while the userspace ones had a
> hard time to work with udev, libusb, libusb1 etc.

Ah I see. Perhaps we can address this once we get some form of dbus in
kernel.

> For UART based drivers it is a little bit different since we had to
> bring up the line discipline from userspace anyway and configure all
> the UART parameters. For these drivers the firmware download or ROM
> patching has been done normally via userspace since there is full
> exclusive access before the Bluetooth subsystem knows about the
> device.

Is this something that dbus in kernel can help clean up a bit?

> With the newly introduced setup stage, the drivers could actually
> share the setup handling for UART and USB based devices. Not sure if
> any vendor is going for this approach.

Interesting... Is the architecture documented somewhere? Is this
upstream already?

  Luis

Re: ROM Patching (was: [PATCH] bluetooth: remove wrong dependency for BT_ATH3K)

[Greg Kroah-Hartman]

On Tue, Jul 30, 2013 at 03:48:09PM -0700, Luis R. Rodriguez wrote:
> > We are planning to take one extra step and split this into a
> > mini-driver approach similar to what has been done for usbnet, but we
> > are not there yet.
> 
> Neat. Perhaps we need something that we can share with 802.11 or other
> hardare I highly doubt we're the only ones patching ROM. Don't we even
> patch up core CPUs? I'm wondering if firmware_class could be expanded to
> support serialized ROM patching. The biggest hurdle I see with splititng
> ROM patching from a single firmware is serializing that, addressing
> revision dependencies and of course kernel dependencies.

What does the firmware_class need to do here that it doesn't do today?

> > However the ROM patching drivers need to be in the kernel since
> > otherwise they will race with the core init sequence.
> 
> Sure and depending on the architecture -- if this is kicked off to
> userspace helpers or not then we may need to consider dbus in
> kernel thing to help with speed / races, dependenecies / async
> loading, -EPROBE_DEFER, etc.

How would dbus in the kernel change anything here?

> > There are also just firmware download drivers that do nothing else
> > than just loading the firmware. And here it can be done via userspace
> > or kernel space. In the Bluetooth world we have seen both, but
> > generally the kernel ones stayed around while the userspace ones had a
> > hard time to work with udev, libusb, libusb1 etc.
> 
> Ah I see. Perhaps we can address this once we get some form of dbus in
> kernel.

The kernel directly loads firmware from the filesystem now, no userspace
helpers are involved, so no need for udev, libusb, etc.  And as such, no
need for any "dbus" in the kernel to do this either.

> > For UART based drivers it is a little bit different since we had to
> > bring up the line discipline from userspace anyway and configure all
> > the UART parameters. For these drivers the firmware download or ROM
> > patching has been done normally via userspace since there is full
> > exclusive access before the Bluetooth subsystem knows about the
> > device.
> 
> Is this something that dbus in kernel can help clean up a bit?

I don't understand how it could, please explain.

Also note, for those not knowing, we have been working on dbus in the
kernel (google "kdbus" for the github repo), but it is all
outward-facing (i.e. userspace using the kdbus code to interact with
other processes with a dbus-like protocol, not for in-kernel dbus
things, although adding it wouldn't be that hard, just really strange.

thanks,

greg k-h

Re: ROM Patching (was: [PATCH] bluetooth: remove wrong dependency for BT_ATH3K)

[Luis R. Rodriguez]

On Tue, Jul 30, 2013 at 04:55:08PM -0700, Greg Kroah-Hartman wrote:
> On Tue, Jul 30, 2013 at 03:48:09PM -0700, Luis R. Rodriguez wrote:
> > > We are planning to take one extra step and split this into a
> > > mini-driver approach similar to what has been done for usbnet, but we
> > > are not there yet.
> > 
> > Neat. Perhaps we need something that we can share with 802.11 or other
> > hardare I highly doubt we're the only ones patching ROM. Don't we even
> > patch up core CPUs? I'm wondering if firmware_class could be expanded to
> > support serialized ROM patching. The biggest hurdle I see with splititng
> > ROM patching from a single firmware is serializing that, addressing
> > revision dependencies and of course kernel dependencies.
> 
> What does the firmware_class need to do here that it doesn't do today?

It will depend on the requirements for ROM patches and if or not
firmware can be split up into patches rather than getting a full
firmware.fw update for any single patch update. If ROM patches
get split up then how I can imagine driver code firmware getting
to become a pain in the ass to maintain and nasty. It'd seem
better to build relationships between these and possible patch
depdendencies on ROM and let firmware_class do the management
of that.

> > > However the ROM patching drivers need to be in the kernel since
> > > otherwise they will race with the core init sequence.
> > 
> > Sure and depending on the architecture -- if this is kicked off to
> > userspace helpers or not then we may need to consider dbus in
> > kernel thing to help with speed / races, dependenecies / async
> > loading, -EPROBE_DEFER, etc.
> 
> How would dbus in the kernel change anything here?
> The kernel directly loads firmware from the filesystem now, no userspace
> helpers are involved, so no need for udev, libusb, etc.  And as such, no
> need for any "dbus" in the kernel to do this either.
> I don't understand how it could, please explain.

Yeah nevermind :D too much coffee today made my brain fart.

> Also note, for those not knowing, we have been working on dbus in the
> kernel (google "kdbus" for the github repo), but it is all
> outward-facing (i.e. userspace using the kdbus code to interact with
> other processes with a dbus-like protocol, not for in-kernel dbus
> things, although adding it wouldn't be that hard, just really strange.

Thanks!

  Luis

Re: ROM Patching (was: [PATCH] bluetooth: remove wrong dependency for BT_ATH3K)

[Johannes Berg]

On Tue, 2013-07-30 at 15:48 -0700, Luis R. Rodriguez wrote:

> Neat. Perhaps we need something that we can share with 802.11 or other
> hardare I highly doubt we're the only ones patching ROM. Don't we even
> patch up core CPUs? I'm wondering if firmware_class could be expanded to
> support serialized ROM patching. The biggest hurdle I see with splititng
> ROM patching from a single firmware is serializing that, addressing
> revision dependencies and of course kernel dependencies.

I don't really see what you're trying to get it so I arbitrarily picked
this ...

I think "ROM patching" like in BT and 802.11 are two completely
different things. In BT at least you have a common transport to the
device, a more-or-less common command set (HCI) etc. None of that exists
for wireless (today) so trying to find common ground between firmware
(patches) for different 802.11 hardware seems completely futile?

johannes

Re: ROM Patching (was: [PATCH] bluetooth: remove wrong dependency for BT_ATH3K)

[Luis R. Rodriguez]

On Tue, Jul 30, 2013 at 11:46 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2013-07-30 at 15:48 -0700, Luis R. Rodriguez wrote:
>
>> Neat. Perhaps we need something that we can share with 802.11 or other
>> hardare I highly doubt we're the only ones patching ROM. Don't we even
>> patch up core CPUs? I'm wondering if firmware_class could be expanded to
>> support serialized ROM patching. The biggest hurdle I see with splititng
>> ROM patching from a single firmware is serializing that, addressing
>> revision dependencies and of course kernel dependencies.
>
> I don't really see what you're trying to get it so I arbitrarily picked
> this ...
>
> I think "ROM patching" like in BT and 802.11 are two completely
> different things. In BT at least you have a common transport to the
> device, a more-or-less common command set (HCI) etc.

Sure.

> None of that exists
> for wireless (today) so trying to find common ground between firmware
> (patches) for different 802.11 hardware seems completely futile?

So long as we can gaurantee mobile drivers get fw updated as much as
is really needed in the industry great.

  Luis