kbl_guc and bxt_guc firmware missing from linux-firmware

kbl_guc and bxt_guc firmware missing from linux-firmware

[Seth Forshee]

Hi,

I've noted that kbl_guc_ver9_14.bin and bxt_guc_ver8_7.bin are not in
linux-firmware despite being available here:

 https://01.org/linuxgraphics/downloads/firmware

Is there some reason they haven't been pusehd out to linux-firmware,
e.g. are they not yet stable or something like that? Any reason we
shouldn't ship those files in Ubuntu's linux-firmware?

Frankly, the practice of adding MODULE_FIRMWARE statements to i915 for
files which aren't in linux-firmware has become a significant annoyance
to me as Ubuntu's linux-firmware package maintainer. I'm getting a
steady stream of bug reports from users about the "Possible missing
firmware ..." messages from mkinitramfs, to the point that I've started
removing the MODULE_FIRMWARE statements for those files from our
kernels.

Thanks,
Seth
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: kbl_guc and bxt_guc firmware missing from linux-firmware

[Jani Nikula]

On Tue, 14 Feb 2017, Seth Forshee <seth.forshee@canonical.com> wrote:
> Hi,
>
> I've noted that kbl_guc_ver9_14.bin and bxt_guc_ver8_7.bin are not in
> linux-firmware despite being available here:
>
>  https://01.org/linuxgraphics/downloads/firmware
>
> Is there some reason they haven't been pusehd out to linux-firmware,
> e.g. are they not yet stable or something like that? Any reason we
> shouldn't ship those files in Ubuntu's linux-firmware?

None that I know of. Rodrigo, please send the pull request to
linux-firmware if you haven't already.

> Frankly, the practice of adding MODULE_FIRMWARE statements to i915 for
> files which aren't in linux-firmware has become a significant annoyance
> to me as Ubuntu's linux-firmware package maintainer. I'm getting a
> steady stream of bug reports from users about the "Possible missing
> firmware ..." messages from mkinitramfs, to the point that I've started
> removing the MODULE_FIRMWARE statements for those files from our
> kernels.

Admittedly we've probably done that even before the firmware has hit
01.org, and we should fix this. We don't have a long history of dealing
with firmware blobs, and your feedback is appreciated.

I think my main question is, what are the downsides of requesting a
firmware even if it hasn't been declared using a MODULE_FIRMWARE
statement? I don't see anything preventing that. Is MODULE_FIRMWARE
purely informational, to help ensure the packaging gets it right?

We do need to have the firmware loading code in place long before we
actually publish the firmware, to test the stuff. Could we get away with
adding the MODULE_FIRMWARE statements only after the blobs have hit
linux-firmware?


BR,
Jani.

-- 
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: kbl_guc and bxt_guc firmware missing from linux-firmware

[Seth Forshee]

On Wed, Feb 15, 2017 at 12:28:42PM +0200, Jani Nikula wrote:
> On Tue, 14 Feb 2017, Seth Forshee <seth.forshee@canonical.com> wrote:
> > Hi,
> >
> > I've noted that kbl_guc_ver9_14.bin and bxt_guc_ver8_7.bin are not in
> > linux-firmware despite being available here:
> >
> >  https://01.org/linuxgraphics/downloads/firmware
> >
> > Is there some reason they haven't been pusehd out to linux-firmware,
> > e.g. are they not yet stable or something like that? Any reason we
> > shouldn't ship those files in Ubuntu's linux-firmware?
> 
> None that I know of. Rodrigo, please send the pull request to
> linux-firmware if you haven't already.
> 
> > Frankly, the practice of adding MODULE_FIRMWARE statements to i915 for
> > files which aren't in linux-firmware has become a significant annoyance
> > to me as Ubuntu's linux-firmware package maintainer. I'm getting a
> > steady stream of bug reports from users about the "Possible missing
> > firmware ..." messages from mkinitramfs, to the point that I've started
> > removing the MODULE_FIRMWARE statements for those files from our
> > kernels.
> 
> Admittedly we've probably done that even before the firmware has hit
> 01.org, and we should fix this. We don't have a long history of dealing
> with firmware blobs, and your feedback is appreciated.
> 
> I think my main question is, what are the downsides of requesting a
> firmware even if it hasn't been declared using a MODULE_FIRMWARE
> statement? I don't see anything preventing that. Is MODULE_FIRMWARE
> purely informational, to help ensure the packaging gets it right?
> 
> We do need to have the firmware loading code in place long before we
> actually publish the firmware, to test the stuff. Could we get away with
> adding the MODULE_FIRMWARE statements only after the blobs have hit
> linux-firmware?

The only downside I know of is this. mkinitramfs uses the modinfo
firmware field to copy firmware into the initrd along with the kernel
module, so when i915.ko is copied to the initrd that firmware would not
be copied along with it. There may be other tools using that modinfo
field that I'm not aware of though.

Maybe it would be good to have something like MODULE_OPTIONAL_FIRMWARE
to identify firmware that isn't required but will be used by the driver
if available. Then mkinitramfs can try to copy those files along with
the module but know that there's no need to produce a warning if it's
not present.

Thanks,
Seth
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: kbl_guc and bxt_guc firmware missing from linux-firmware

[Vivi, Rodrigo]

On Wed, 2017-02-15 at 07:01 -0600, Seth Forshee wrote:
> On Wed, Feb 15, 2017 at 12:28:42PM +0200, Jani Nikula wrote:
> > On Tue, 14 Feb 2017, Seth Forshee <seth.forshee@canonical.com> wrote:
> > > Hi,
> > >
> > > I've noted that kbl_guc_ver9_14.bin and bxt_guc_ver8_7.bin are not in
> > > linux-firmware despite being available here:
> > >
> > >  https://01.org/linuxgraphics/downloads/firmware
> > >
> > > Is there some reason they haven't been pusehd out to linux-firmware,
> > > e.g. are they not yet stable or something like that? Any reason we
> > > shouldn't ship those files in Ubuntu's linux-firmware?
> > 
> > None that I know of. Rodrigo, please send the pull request to
> > linux-firmware if you haven't already.

Anusha already did the pull-request. 

We are waiting on Kyle now.


> > 
> > > Frankly, the practice of adding MODULE_FIRMWARE statements to i915 for
> > > files which aren't in linux-firmware has become a significant annoyance
> > > to me as Ubuntu's linux-firmware package maintainer. I'm getting a
> > > steady stream of bug reports from users about the "Possible missing
> > > firmware ..." messages from mkinitramfs, to the point that I've started
> > > removing the MODULE_FIRMWARE statements for those files from our
> > > kernels.
> > 
> > Admittedly we've probably done that even before the firmware has hit
> > 01.org, and we should fix this. We don't have a long history of dealing
> > with firmware blobs, and your feedback is appreciated.
> > 
> > I think my main question is, what are the downsides of requesting a
> > firmware even if it hasn't been declared using a MODULE_FIRMWARE
> > statement? I don't see anything preventing that. Is MODULE_FIRMWARE
> > purely informational, to help ensure the packaging gets it right?
> > 
> > We do need to have the firmware loading code in place long before we
> > actually publish the firmware, to test the stuff. Could we get away with
> > adding the MODULE_FIRMWARE statements only after the blobs have hit
> > linux-firmware?

I believe we can improve a bit and only merge the final loading patch
after our QA gave the ok to release at 01.org and propagate to
linux-firmware.git. But this for the regular case where the patch only
add a certain version.

One thing that hit us hard on this was the first time we had huc and the
rebasing of all loading patches was a painful so we had to merge that
soon so we could really test and keep some level of CI and regression
tests running before really releasing the firmware.

> 
> The only downside I know of is this. mkinitramfs uses the modinfo
> firmware field to copy firmware into the initrd along with the kernel
> module, so when i915.ko is copied to the initrd that firmware would not
> be copied along with it. There may be other tools using that modinfo
> field that I'm not aware of though.
> 
> Maybe it would be good to have something like MODULE_OPTIONAL_FIRMWARE
> to identify firmware that isn't required but will be used by the driver
> if available. Then mkinitramfs can try to copy those files along with
> the module but know that there's no need to produce a warning if it's
> not present.
> 
> Thanks,
> Seth

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: kbl_guc and bxt_guc firmware missing from linux-firmware

[Jani Nikula]

On Wed, 15 Feb 2017, Seth Forshee <seth.forshee@canonical.com> wrote:
> Maybe it would be good to have something like MODULE_OPTIONAL_FIRMWARE
> to identify firmware that isn't required but will be used by the driver
> if available. Then mkinitramfs can try to copy those files along with
> the module but know that there's no need to produce a warning if it's
> not present.

The documentation for the current MODULE_FIRMWARE already says,
"Optional firmware file (or files) needed by the module format is simply
firmware file name."

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: kbl_guc and bxt_guc firmware missing from linux-firmware

[Seth Forshee]

On Wed, Mar 01, 2017 at 03:23:06PM +0200, Jani Nikula wrote:
> On Wed, 15 Feb 2017, Seth Forshee <seth.forshee@canonical.com> wrote:
> > Maybe it would be good to have something like MODULE_OPTIONAL_FIRMWARE
> > to identify firmware that isn't required but will be used by the driver
> > if available. Then mkinitramfs can try to copy those files along with
> > the module but know that there's no need to produce a warning if it's
> > not present.
> 
> The documentation for the current MODULE_FIRMWARE already says,
> "Optional firmware file (or files) needed by the module format is simply
> firmware file name."

That may have been the intention. But since it is the only thing
available it has been used for any firmware files, including many which
will cause device probe to fail if not present. Userspace tools seem to
treat the firmware files in modinfo to mean, "firmware files without
which the hardware may fail to function or may not function properly,"
because in many cases that is true. I'd argue that this is the de facto
definition of MODULE_FIRMWARE, and that the comment is at best
misleading.

Seth
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: kbl_guc and bxt_guc firmware missing from linux-firmware

[Jani Nikula]

On Wed, 01 Mar 2017, Seth Forshee <seth.forshee@canonical.com> wrote:
> On Wed, Mar 01, 2017 at 03:23:06PM +0200, Jani Nikula wrote:
>> On Wed, 15 Feb 2017, Seth Forshee <seth.forshee@canonical.com> wrote:
>> > Maybe it would be good to have something like MODULE_OPTIONAL_FIRMWARE
>> > to identify firmware that isn't required but will be used by the driver
>> > if available. Then mkinitramfs can try to copy those files along with
>> > the module but know that there's no need to produce a warning if it's
>> > not present.
>> 
>> The documentation for the current MODULE_FIRMWARE already says,
>> "Optional firmware file (or files) needed by the module format is simply
>> firmware file name."
>
> That may have been the intention. But since it is the only thing
> available it has been used for any firmware files, including many which
> will cause device probe to fail if not present. Userspace tools seem to
> treat the firmware files in modinfo to mean, "firmware files without
> which the hardware may fail to function or may not function properly,"
> because in many cases that is true. I'd argue that this is the de facto
> definition of MODULE_FIRMWARE, and that the comment is at best
> misleading.

Just a quick follow-up. We may choose to improve the MODULE_FIRMWARE
mechanism in the long run, but for the time being we're committed to
only adding the MODULE_FIRMWARE statements once the blobs have hit the
linux-firmware repository. For example, we'll queue a fix to v4.11-rc3
(or -rc4 at the latest) removing one such statement.

Thanks again for your feedback.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx