firmware requirements

firmware requirements

[Dave Airlie]

I'd just like to remind everyone of the firmware requirements for
vendors that control their firmware and the driver upstreams:

https://docs.kernel.org/driver-api/firmware/firmware-usage-guidelines.html

Intel VPU it seems like you are not currently shipping upstream
firmware, and might have tied your fw and userspace together.

I'm cc'ing the AMD XDNA driver as it recently landed and I'd like them
to confirm they are following the above requirements.

The main reason we don't allow userspace/fw direct linkage is if a
user deploys two containers with two different userspace drivers in
them on the same hardware, what is the kernel driver supposed to do?

Firmware should be abstracted in the kernel if it is not possible to
build proper fw APIs for userspace to use directly, by proper I mean
forward and backwards compatible.

Thanks,
Dave.

Re: firmware requirements

[Jacek Lawrynowicz]

Hi,

On 2/19/2025 10:01 PM, Dave Airlie wrote:
> I'd just like to remind everyone of the firmware requirements for
> vendors that control their firmware and the driver upstreams:
> 
> https://docs.kernel.org/driver-api/firmware/firmware-usage-guidelines.html
> 
> Intel VPU it seems like you are not currently shipping upstream
> firmware, and might have tied your fw and userspace together.

Yep, this is correct :/

> I'm cc'ing the AMD XDNA driver as it recently landed and I'd like them
> to confirm they are following the above requirements.
> 
> The main reason we don't allow userspace/fw direct linkage is if a
> user deploys two containers with two different userspace drivers in
> them on the same hardware, what is the kernel driver supposed to do?

This makes sense, but I didn't see anything in the firmware usage guidelines about needing user-space and firmware to be compatible.
It is focused on making sure the kernel driver works well with the firmware.
Our intel_vpu driver sticks to the basics with the firmware ABI, so we've got backward and forward compatibility covered from the initial release.

Now, when it comes to user space and firmware, that's a whole other story. We've made some headway, but there's still a lot to do.
Our firmware is pretty massive (like 10 times bigger than your average GPU firmware) which makes things trickier than what other drivers deal with.
That's why we didn't put the firmware in linux-firmware right away, but we'll get the next release in there.

We will continue working on separating user space from the firmware more to make things smoother.

> Firmware should be abstracted in the kernel if it is not possible to
> build proper fw APIs for userspace to use directly, by proper I mean
> forward and backwards compatible.

This dependents on a project. In our case only user space should provide this compatibility.
We don't even parse command buffers at the moment in kernel space.

For your reference, we release updates for both the user space and firmware every couple of weeks here: https://github.com/intel/linux-npu-driver/releases

Jacek

Re: firmware requirements

[Simona Vetter]

Hi Jacek!

Bit late reply, was sick last week and still recovering from missed mails.

On Thu, Feb 20, 2025 at 11:50:10AM +0100, Jacek Lawrynowicz wrote:
> On 2/19/2025 10:01 PM, Dave Airlie wrote:
> > I'd just like to remind everyone of the firmware requirements for
> > vendors that control their firmware and the driver upstreams:
> > 
> > https://docs.kernel.org/driver-api/firmware/firmware-usage-guidelines.html
> > 
> > Intel VPU it seems like you are not currently shipping upstream
> > firmware, and might have tied your fw and userspace together.
> 
> Yep, this is correct :/
> 
> > I'm cc'ing the AMD XDNA driver as it recently landed and I'd like them
> > to confirm they are following the above requirements.
> > 
> > The main reason we don't allow userspace/fw direct linkage is if a
> > user deploys two containers with two different userspace drivers in
> > them on the same hardware, what is the kernel driver supposed to do?
> 
> This makes sense, but I didn't see anything in the firmware usage

Well, when Dave wrote that documentation section and I reviewed it we
figured that's clear from the rules we have. There's two rules:

- firmware is not allowed to break the kernel. This is not limited to
  "does the kernel driver keep loading", but fully extends to anything the
  kernel driver does or needs to fullfill its job.
- the kernel is not allowed to break userspace. This is not limited to the
  uapi structures, but anything they point at or implicitly reference, any
  implementation details userspace relies on, any other hidden semantics
  that aren't entirely transparent. Everything really that can result in a
  bug report from users.

If the firmware upgrade still works with the kernel, but changes the
how the overall uapi works, then the firmware broke the kernel.

I think the two individiual steps above are very clear already, but I
guess that both together mean that firmware isn't allowed to break
userspace isn't clear. Can you please send a patch to add that as another
very explicit bullet to the existing list in the fw guidelines?

> guidelines about needing user-space and firmware to be compatible.
> It is focused on making sure the kernel driver works well with the
> firmware.
> Our intel_vpu driver sticks to the basics with the firmware ABI, so
> we've got backward and forward compatibility covered from the initial
> release.
> 
> Now, when it comes to user space and firmware, that's a whole other
> story. We've made some headway, but there's still a lot to do.
> Our firmware is pretty massive (like 10 times bigger than your average
> GPU firmware) which makes things trickier than what other drivers deal
> with.
> That's why we didn't put the firmware in linux-firmware right away, but
> we'll get the next release in there.
> 
> We will continue working on separating user space from the firmware more
> to make things smoother.
> 
> > Firmware should be abstracted in the kernel if it is not possible to
> > build proper fw APIs for userspace to use directly, by proper I mean
> > forward and backwards compatible.
> 
> This dependents on a project. In our case only user space should provide
> this compatibility.
> We don't even parse command buffers at the moment in kernel space.
> 
> For your reference, we release updates for both the user space and
> firmware every couple of weeks here:
> https://github.com/intel/linux-npu-driver/releases

Yeah you need to fix this. With containers and other packages runtimes
there's really no connection between the base os image, and the userspace
you're running. Which means you really cannot assume that on any given
system there's only one abi version across the firmware and userspace
libraries. So even without upstreams "no breaking uapi" guarantee this
does not work in production.

Cheers, Sima
-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: firmware requirements

[Jacek Lawrynowicz]

Hi,

On 3/4/2025 6:06 PM, Simona Vetter wrote:
> Hi Jacek!
> 
> Bit late reply, was sick last week and still recovering from missed mails.
> 
> On Thu, Feb 20, 2025 at 11:50:10AM +0100, Jacek Lawrynowicz wrote:
>> On 2/19/2025 10:01 PM, Dave Airlie wrote:
>>> I'd just like to remind everyone of the firmware requirements for
>>> vendors that control their firmware and the driver upstreams:
>>>
>>> https://docs.kernel.org/driver-api/firmware/firmware-usage-guidelines.html
>>>
>>> Intel VPU it seems like you are not currently shipping upstream
>>> firmware, and might have tied your fw and userspace together.
>>
>> Yep, this is correct :/
>>
>>> I'm cc'ing the AMD XDNA driver as it recently landed and I'd like them
>>> to confirm they are following the above requirements.
>>>
>>> The main reason we don't allow userspace/fw direct linkage is if a
>>> user deploys two containers with two different userspace drivers in
>>> them on the same hardware, what is the kernel driver supposed to do?
>>
>> This makes sense, but I didn't see anything in the firmware usage
> 
> Well, when Dave wrote that documentation section and I reviewed it we
> figured that's clear from the rules we have. There's two rules:
> 
> - firmware is not allowed to break the kernel. This is not limited to
>   "does the kernel driver keep loading", but fully extends to anything the
>   kernel driver does or needs to fullfill its job.
> - the kernel is not allowed to break userspace. This is not limited to the
>   uapi structures, but anything they point at or implicitly reference, any
>   implementation details userspace relies on, any other hidden semantics
>   that aren't entirely transparent. Everything really that can result in a
>   bug report from users.
> 
> If the firmware upgrade still works with the kernel, but changes the
> how the overall uapi works, then the firmware broke the kernel.
> 
> I think the two individiual steps above are very clear already, but I
> guess that both together mean that firmware isn't allowed to break
> userspace isn't clear. Can you please send a patch to add that as another
> very explicit bullet to the existing list in the fw guidelines?

Sure, I've sent the patch.

>> guidelines about needing user-space and firmware to be compatible.
>> It is focused on making sure the kernel driver works well with the
>> firmware.
>> Our intel_vpu driver sticks to the basics with the firmware ABI, so
>> we've got backward and forward compatibility covered from the initial
>> release.
>>
>> Now, when it comes to user space and firmware, that's a whole other
>> story. We've made some headway, but there's still a lot to do.
>> Our firmware is pretty massive (like 10 times bigger than your average
>> GPU firmware) which makes things trickier than what other drivers deal
>> with.
>> That's why we didn't put the firmware in linux-firmware right away, but
>> we'll get the next release in there.
>>
>> We will continue working on separating user space from the firmware more
>> to make things smoother.
>>
>>> Firmware should be abstracted in the kernel if it is not possible to
>>> build proper fw APIs for userspace to use directly, by proper I mean
>>> forward and backwards compatible.
>>
>> This dependents on a project. In our case only user space should provide
>> this compatibility.
>> We don't even parse command buffers at the moment in kernel space.
>>
>> For your reference, we release updates for both the user space and
>> firmware every couple of weeks here:
>> https://github.com/intel/linux-npu-driver/releases
> 
> Yeah you need to fix this. With containers and other packages runtimes
> there's really no connection between the base os image, and the userspace
> you're running. Which means you really cannot assume that on any given
> system there's only one abi version across the firmware and userspace
> libraries. So even without upstreams "no breaking uapi" guarantee this
> does not work in production.

We should be able to provide backward compatibility, so new FW will work with all past userspace versions.
The base OS just has to use up-to-date FW. This should satisfy FW requirements, right?

Regards,
Jacek

Re: firmware requirements

[Simona Vetter]

On Fri, 14 Mar 2025 at 11:16, Jacek Lawrynowicz
<jacek.lawrynowicz@linux.intel.com> wrote:
>
> Hi,
>
> On 3/4/2025 6:06 PM, Simona Vetter wrote:
> > Hi Jacek!
> >
> > Bit late reply, was sick last week and still recovering from missed mails.
> >
> > On Thu, Feb 20, 2025 at 11:50:10AM +0100, Jacek Lawrynowicz wrote:
> >> On 2/19/2025 10:01 PM, Dave Airlie wrote:
> >>> I'd just like to remind everyone of the firmware requirements for
> >>> vendors that control their firmware and the driver upstreams:
> >>>
> >>> https://docs.kernel.org/driver-api/firmware/firmware-usage-guidelines.html
> >>>
> >>> Intel VPU it seems like you are not currently shipping upstream
> >>> firmware, and might have tied your fw and userspace together.
> >>
> >> Yep, this is correct :/
> >>
> >>> I'm cc'ing the AMD XDNA driver as it recently landed and I'd like them
> >>> to confirm they are following the above requirements.
> >>>
> >>> The main reason we don't allow userspace/fw direct linkage is if a
> >>> user deploys two containers with two different userspace drivers in
> >>> them on the same hardware, what is the kernel driver supposed to do?
> >>
> >> This makes sense, but I didn't see anything in the firmware usage
> >
> > Well, when Dave wrote that documentation section and I reviewed it we
> > figured that's clear from the rules we have. There's two rules:
> >
> > - firmware is not allowed to break the kernel. This is not limited to
> >   "does the kernel driver keep loading", but fully extends to anything the
> >   kernel driver does or needs to fullfill its job.
> > - the kernel is not allowed to break userspace. This is not limited to the
> >   uapi structures, but anything they point at or implicitly reference, any
> >   implementation details userspace relies on, any other hidden semantics
> >   that aren't entirely transparent. Everything really that can result in a
> >   bug report from users.
> >
> > If the firmware upgrade still works with the kernel, but changes the
> > how the overall uapi works, then the firmware broke the kernel.
> >
> > I think the two individiual steps above are very clear already, but I
> > guess that both together mean that firmware isn't allowed to break
> > userspace isn't clear. Can you please send a patch to add that as another
> > very explicit bullet to the existing list in the fw guidelines?
>
> Sure, I've sent the patch.

Thanks a lot!

> >> guidelines about needing user-space and firmware to be compatible.
> >> It is focused on making sure the kernel driver works well with the
> >> firmware.
> >> Our intel_vpu driver sticks to the basics with the firmware ABI, so
> >> we've got backward and forward compatibility covered from the initial
> >> release.
> >>
> >> Now, when it comes to user space and firmware, that's a whole other
> >> story. We've made some headway, but there's still a lot to do.
> >> Our firmware is pretty massive (like 10 times bigger than your average
> >> GPU firmware) which makes things trickier than what other drivers deal
> >> with.
> >> That's why we didn't put the firmware in linux-firmware right away, but
> >> we'll get the next release in there.
> >>
> >> We will continue working on separating user space from the firmware more
> >> to make things smoother.
> >>
> >>> Firmware should be abstracted in the kernel if it is not possible to
> >>> build proper fw APIs for userspace to use directly, by proper I mean
> >>> forward and backwards compatible.
> >>
> >> This dependents on a project. In our case only user space should provide
> >> this compatibility.
> >> We don't even parse command buffers at the moment in kernel space.
> >>
> >> For your reference, we release updates for both the user space and
> >> firmware every couple of weeks here:
> >> https://github.com/intel/linux-npu-driver/releases
> >
> > Yeah you need to fix this. With containers and other packages runtimes
> > there's really no connection between the base os image, and the userspace
> > you're running. Which means you really cannot assume that on any given
> > system there's only one abi version across the firmware and userspace
> > libraries. So even without upstreams "no breaking uapi" guarantee this
> > does not work in production.
>
> We should be able to provide backward compatibility, so new FW will work with all past userspace versions.
> The base OS just has to use up-to-date FW. This should satisfy FW requirements, right?

Yeah new userspace not working without newer firmware is ok. It's not
great for userspace and ideally you limit this to exceptional cases
only or make those additional features an optional requirement, but
it's not a concern for upstream. It's old userspace breaking on newer
firmware that would not be ok. So I think you're good. Would be good
to update the wording on the website to reflect that too.

Cheers, Sima
-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: firmware requirements

[Hou, Lizhi]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

[AMD Official Use Only - AMD Internal Distribution Only]

Hi Dave,

The AMD XDNA driver will follow the guideline requirements.

Thanks,
Lizhi

From: Dave Airlie <airlied@gmail.com>
Date: Wednesday, February 19, 2025 at 1:02 PM
To: dri-devel <dri-devel@lists.freedesktop.org>, Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com>, Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>, Ma, Min <min.ma@amd.com>, Hou, Lizhi <lizhi.hou@amd.com>
Subject: firmware requirements
I'd just like to remind everyone of the firmware requirements for
vendors that control their firmware and the driver upstreams:

https://docs.kernel.org/driver-api/firmware/firmware-usage-guidelines.html

Intel VPU it seems like you are not currently shipping upstream
firmware, and might have tied your fw and userspace together.

I'm cc'ing the AMD XDNA driver as it recently landed and I'd like them
to confirm they are following the above requirements.

The main reason we don't allow userspace/fw direct linkage is if a
user deploys two containers with two different userspace drivers in
them on the same hardware, what is the kernel driver supposed to do?

Firmware should be abstracted in the kernel if it is not possible to
build proper fw APIs for userspace to use directly, by proper I mean
forward and backwards compatible.

Thanks,
Dave.

[-- Attachment #2: Type: text/html, Size: 4050 bytes --]