[RFC] Online firmware upgrade in non-embedded systems

[RFC] Online firmware upgrade in non-embedded systems

[Ben Hutchings]

Network and disk controllers normally have at least some firmware in
flash to support their use as boot devices.  Solarflare's current
network adapters can have as many as 4 distinct firmware partitions: one
for each PHY, one for the management CPU and one for net-boot.  Any or
all of these may need to be updated in the field, to fix bugs or to add
features.  (Furthermore, the net-boot firmware is based on gPXE and is
therefore modifiable by any user.)

Currently the sfc network driver is optionally combined with an MTD
driver (CONFIG_SFC_MTD) which exposes all upgradable firmware and
configuration partitions in flash.  This works nicely in kernels with
MTD enabled, but since MTD is mainly used in embedded systems with
on-board flash it is often disabled in distribution kernels and custom
kernels alike.  This leaves users of sfc unable to upgrade firmware
without rebuilding the kernel or booting some other distribution.  The
lack of widespread MTD support is a regular cause of support requests.

There are two main alternatives I'm aware of:

- Use the ethtool ETHTOOL_SEEPROM command with multiple magic numbers
for different partitions. bnx2x does something a little like this for
PHY firmware upgrades. The ethtool core passes in a page at a time so
this won't work well with flash block sizes greater than PAGE_SIZE.

- Use the ethtool ETHTOOL_FLASHDEV command. It's intended for precisely
this purpose, but the driver is supposed to implement it using
request_firmware(). That seems to require any firmware update program to
integrate with udev or to copy firmware images under /lib/firmware even
though they are one-time updates. That doesn't make a lot of sense to
me. Am I missing something?

Of course these are both specific to network devices; it seems deisrable
to have a more general convention for online firmware upgrades.  MTDs
clearly are more generally applicable, and pretty much every computer
does have flash storage for firmware and boot configuration, so perhaps
it should be treated as more of a standard feature?

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [RFC] Online firmware upgrade in non-embedded systems

[Carl-Daniel Hailfinger]

[adding flashrom@flashrom.org to CC, senders will be whitelisted after a
short delay]

On 28.09.2010 19:59, Ben Hutchings wrote:
> Network and disk controllers normally have at least some firmware in
> flash to support their use as boot devices. [...]
>
> Currently the sfc network driver is optionally combined with an MTD
> driver (CONFIG_SFC_MTD) which exposes all upgradable firmware and
> configuration partitions in flash.  This works nicely in kernels with
> MTD enabled, but since MTD is mainly used in embedded systems with
> on-board flash it is often disabled in distribution kernels and custom
> kernels alike.  This leaves users of sfc unable to upgrade firmware
> without rebuilding the kernel or booting some other distribution.  The
> lack of widespread MTD support is a regular cause of support requests.
>
> There are two main alternatives I'm aware of:
>
> - Use the ethtool ETHTOOL_SEEPROM [...]
>
> - Use the ethtool ETHTOOL_FLASHDEV command [..]
>
> Of course these are both specific to network devices; it seems deisrable
> to have a more general convention for online firmware upgrades.  MTDs
> clearly are more generally applicable, and pretty much every computer
> does have flash storage for firmware and boot configuration, so perhaps
> it should be treated as more of a standard feature?
>   

Given that the flashrom utility <http://www.flashrom.org/> (GPLv2)
supports flashing many network cards, SATA/PATA controllers, graphics
cards, and of course the main system firmware/BIOS/EFI, and it does that
from userspace without any kernel support, using flashrom for those
purposes makes sense IMHO.

flashrom works fine under Linux 2.4, 2.6 and pretty much every other OS
out there. There is a focus on x86, but MIPS, PowerPC and other
architectures are supported as well. flashrom is independent of the MTD
framework, so you don't need to boot a special kernel for a firmware
upgrade.

That said, writing a flashrom driver for your hardware is pretty easy.
You could even use existing kernel code for your device as a template
for the flashrom driver since the license is compatible.

If you have any questions, please ask.

Regards,
Carl-Daniel

Re: [RFC] Online firmware upgrade in non-embedded systems

[Artem Bityutskiy]

On Tue, 2010-09-28 at 18:59 +0100, Ben Hutchings wrote:
> Currently the sfc network driver is optionally combined with an MTD
> driver (CONFIG_SFC_MTD) which exposes all upgradable firmware and
> configuration partitions in flash.  This works nicely in kernels with
> MTD enabled, but since MTD is mainly used in embedded systems with
> on-board flash it is often disabled in distribution kernels and custom
> kernels alike.  This leaves users of sfc unable to upgrade firmware
> without rebuilding the kernel or booting some other distribution.  The
> lack of widespread MTD support is a regular cause of support requests.

At least Fedora does have MTD enabled. But I guess commercial
distributions like RHEL might have it disabled (but I did not check),
and I think I could guess the reason for this.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [RFC] Online firmware upgrade in non-embedded systems

[Ben Hutchings]

On Wed, 2010-09-29 at 00:41 +0200, Carl-Daniel Hailfinger wrote:
> [adding flashrom@flashrom.org to CC, senders will be whitelisted after a
> short delay]
> 
> On 28.09.2010 19:59, Ben Hutchings wrote:
> > Network and disk controllers normally have at least some firmware in
> > flash to support their use as boot devices. [...]
> >
> > Currently the sfc network driver is optionally combined with an MTD
> > driver (CONFIG_SFC_MTD) which exposes all upgradable firmware and
> > configuration partitions in flash.  This works nicely in kernels with
> > MTD enabled, but since MTD is mainly used in embedded systems with
> > on-board flash it is often disabled in distribution kernels and custom
> > kernels alike.  This leaves users of sfc unable to upgrade firmware
> > without rebuilding the kernel or booting some other distribution.  The
> > lack of widespread MTD support is a regular cause of support requests.
> >
> > There are two main alternatives I'm aware of:
> >
> > - Use the ethtool ETHTOOL_SEEPROM [...]
> >
> > - Use the ethtool ETHTOOL_FLASHDEV command [..]
> >
> > Of course these are both specific to network devices; it seems deisrable
> > to have a more general convention for online firmware upgrades.  MTDs
> > clearly are more generally applicable, and pretty much every computer
> > does have flash storage for firmware and boot configuration, so perhaps
> > it should be treated as more of a standard feature?
> >   
> 
> Given that the flashrom utility <http://www.flashrom.org/> (GPLv2)
> supports flashing many network cards, SATA/PATA controllers, graphics
> cards, and of course the main system firmware/BIOS/EFI, and it does that
> from userspace without any kernel support,
[...]

I'm looking for a clean solution, not a hack.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [RFC] Online firmware upgrade in non-embedded systems

[Ben Hutchings]

On Wed, 2010-09-29 at 15:34 +0300, Artem Bityutskiy wrote:
> On Tue, 2010-09-28 at 18:59 +0100, Ben Hutchings wrote:
> > Currently the sfc network driver is optionally combined with an MTD
> > driver (CONFIG_SFC_MTD) which exposes all upgradable firmware and
> > configuration partitions in flash.  This works nicely in kernels with
> > MTD enabled, but since MTD is mainly used in embedded systems with
> > on-board flash it is often disabled in distribution kernels and custom
> > kernels alike.  This leaves users of sfc unable to upgrade firmware
> > without rebuilding the kernel or booting some other distribution.  The
> > lack of widespread MTD support is a regular cause of support requests.
> 
> At least Fedora does have MTD enabled. But I guess commercial
> distributions like RHEL might have it disabled (but I did not check),
> and I think I could guess the reason for this.

RHEL 4 and 5 have it; the first RHEL 6 beta did not but this was changed
at my request.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [RFC] Online firmware upgrade in non-embedded systems

[Carl-Daniel Hailfinger]

On 29.09.2010 14:35, Ben Hutchings wrote:
> On Wed, 2010-09-29 at 00:41 +0200, Carl-Daniel Hailfinger wrote:
>   
>> [adding flashrom@flashrom.org to CC, senders will be whitelisted after a
>> short delay]
>>
>> On 28.09.2010 19:59, Ben Hutchings wrote:
>>     
>>> Network and disk controllers normally have at least some firmware in
>>> flash to support their use as boot devices. [...]
>>>   
>>>       
>> Given that the flashrom utility <http://www.flashrom.org/> (GPLv2)
>> supports flashing many network cards, SATA/PATA controllers, graphics
>> cards, and of course the main system firmware/BIOS/EFI, and it does that
>> from userspace without any kernel support,
>>     
> [...]
>
> I'm looking for a clean solution, not a hack.
>   

What would qualify as a clean solution? And is cross-platform code one
of your goals?


Regards,
Carl-Daniel

Re: [RFC] Online firmware upgrade in non-embedded systems

[Ben Hutchings]

On Wed, 2010-09-29 at 14:45 +0200, Carl-Daniel Hailfinger wrote:
> On 29.09.2010 14:35, Ben Hutchings wrote:
> > On Wed, 2010-09-29 at 00:41 +0200, Carl-Daniel Hailfinger wrote:
> >   
> >> [adding flashrom@flashrom.org to CC, senders will be whitelisted after a
> >> short delay]
> >>
> >> On 28.09.2010 19:59, Ben Hutchings wrote:
> >>     
> >>> Network and disk controllers normally have at least some firmware in
> >>> flash to support their use as boot devices. [...]
> >>>   
> >>>       
> >> Given that the flashrom utility <http://www.flashrom.org/> (GPLv2)
> >> supports flashing many network cards, SATA/PATA controllers, graphics
> >> cards, and of course the main system firmware/BIOS/EFI, and it does that
> >> from userspace without any kernel support,
> >>     
> > [...]
> >
> > I'm looking for a clean solution, not a hack.
> >   
> 
> What would qualify as a clean solution?

One where hardware access is mediated by the kernel, and doesn't involve
unloading or potentially conflicting with the driver for that hardware.

> And is cross-platform code one of your goals?

Not at this level.  At the application level, yes, but we already have a
working application so I'm not interested in using flashrom for that.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [RFC] Online firmware upgrade in non-embedded systems

[Carl-Daniel Hailfinger]

On 29.09.2010 15:10, Ben Hutchings wrote:
> On Wed, 2010-09-29 at 14:45 +0200, Carl-Daniel Hailfinger wrote:
>   
>> On 29.09.2010 14:35, Ben Hutchings wrote:
>>     
>>> On Wed, 2010-09-29 at 00:41 +0200, Carl-Daniel Hailfinger wrote:
>>>   
>>>       
>>>> [adding flashrom@flashrom.org to CC, senders will be whitelisted after a
>>>> short delay]
>>>>
>>>> On 28.09.2010 19:59, Ben Hutchings wrote:
>>>>     
>>>>         
>>>>> Network and disk controllers normally have at least some firmware in
>>>>> flash to support their use as boot devices. [...]
>>>>>   
>>>>>       
>>>>>           
>>>> Given that the flashrom utility <http://www.flashrom.org/> (GPLv2)
>>>> supports flashing many network cards, SATA/PATA controllers, graphics
>>>> cards, and of course the main system firmware/BIOS/EFI, and it does that
>>>> from userspace without any kernel support,
>>>>     
>>>>         
>>> [...]
>>>
>>> I'm looking for a clean solution, not a hack.
>>>   
>>>       
>> What would qualify as a clean solution?
>>     
>
> One where hardware access is mediated by the kernel, and doesn't involve
> unloading or potentially conflicting with the driver for that hardware.
>   

flashrom can ask the kernel driver to "please stop accessing flash". No
unloading needed, no conflict in place.


>> And is cross-platform code one of your goals?
>>     
>
> Not at this level.  At the application level, yes, but we already have a
> working application so I'm not interested in using flashrom for that.
>   

I see. Just because I'm interested in how other flashing applications
solve this: Does that application work on *BSD as well? And could you
tell me the name of the app so I can take a look at it? Thanks.

Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/