[PATCH 1/2] schemas: Add firmware node schema

[PATCH 1/2] schemas: Add firmware node schema

[Simon Glass]

Add a motivation and purpose for this new proposed node.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 dtschema/schemas/firmware.yaml | 83 ++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
 create mode 100644 dtschema/schemas/firmware.yaml

diff --git a/dtschema/schemas/firmware.yaml b/dtschema/schemas/firmware.yaml
new file mode 100644
index 0000000..4439a70
--- /dev/null
+++ b/dtschema/schemas/firmware.yaml
@@ -0,0 +1,83 @@
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-clause
+# Copyright 2023 Google LLC
+#
+
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/firmware.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: /firmware Node
+
+maintainers:
+  - Simon Glass <sjg@chromium.org>
+
+description: |
+  The '/firmware' node does not represent a real device, but serves as a place
+  for recording information about the main firmware used on the device, such as
+  a map of its contents. This is used by the Operating System (OS), user space
+  programs and possibly other firmware components. Data in the '/firmware' node
+  does not itself represent the hardware.
+
+  Properties in this node should be common to (and used by) at least two
+  firmware projects, such as U-Boot and TF-A. Project-specific subnodes can be
+  used for properties which are specific to a single project.
+
+  Purpose of '/firmware' node
+  ---------------------------
+
+  Firmware has traditionally been fairly opaque to the OS, with the OS taking
+  no interest in its contents, version, layout or how it might be updated. This
+  is less than ideal, since firmware is an important part of the system and
+  visibility into its operation is every bit as important as visbility into the
+  OS and user-space programs within the system.
+
+  The traditional approach has been to let firmware deal with firmware, and the
+  OS deal with everything else. Updating firmware has been handled by firmware.
+  For example, the UEFI spec defines a way for the OS to post a 'capsule' which
+  is discovered next time the system boots, permitting firmware updates. But
+  firmware updates in firmware are highly problematic. They require a reboot
+  and a sometimes-lengthy wait with a strange-looking interface unfamiliar
+  to most users. It seems better to make the update as transparent as possible
+  to the user. As an example of that, ChromeOS has full knowledge of the
+  firmware version and layout, updates it in the background from user space and
+  instantly selects the new firmware when the user reboots or logs out.
+
+  A common objection to considering the system holistically is that some parts
+  of the system are inaccessible to the OS, such as a secure enclave. However
+  this does not preclude providing visibility into what is present in that
+  enclave. Firmware-version information is still useful. Firmware updates are
+  still needed and can still be initiated from user space.
+
+  Another objection is that firmware should provide an interface to the OS,
+  while keeping its structure private. This thinking is largely driven by
+  extrapolating from how firmware has been handled in the 'BIOS' days.
+  It should be considered a degenerate case rather than the norm. As complexity
+  increases, it creates an artificial boundary between two pieces of the whole.
+  Mechanisms then need to be invented to cross this unnecessary chasm. An
+  example of this is Intel's Dynamic Platform and Thermal Framework (DPTF),
+  which consists of user-space, OS and firmware components all working towards
+  a shared goal. We need a standard description of these cross-system pieces.
+
+  In order to 'teach the OS about firmware', we need a place to put this
+  information. That is the purpose of this node.
+
+  In an Open Source world the entire model of firmware needs to adjust to be
+  more open, more visible and managed just like any other part of the system.
+  The major goal is to standardise how firmware is presented to the OS and user
+  space, so that common utilities can be used to manage the entire system,
+  including the firmware. For example, fwupd can look in this node for
+  information on how to update the firmware, similar to how VBE works. [1]
+  It is likely that other purposes will come to light over time.
+
+  [1] https://github.com/fwupd/fwupd/tree/main/plugins/vbe
+
+properties:
+  $nodename:
+    const: firmware
+
+  "#address-cells": true
+  "#size-cells": true
+
+additionalProperties:
+  type: object
-- 
2.41.0.255.g8b1d071c50-goog

[PATCH 2/2] schemas: Add a schema for binman

[Simon Glass]

I am unsure whether to add this with a generic name, such as 'layout',
but for now am using /firmware/binman to avoid conflicts with any other
firmware-layout schema that others might be working on.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 dtschema/schemas/firmware/binman.yaml       | 51 ++++++++++++++++++
 dtschema/schemas/firmware/binman/entry.yaml | 57 +++++++++++++++++++++
 2 files changed, 108 insertions(+)
 create mode 100644 dtschema/schemas/firmware/binman.yaml
 create mode 100644 dtschema/schemas/firmware/binman/entry.yaml

diff --git a/dtschema/schemas/firmware/binman.yaml b/dtschema/schemas/firmware/binman.yaml
new file mode 100644
index 0000000..4b1ecf6
--- /dev/null
+++ b/dtschema/schemas/firmware/binman.yaml
@@ -0,0 +1,51 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright 2023 Google LLC
+
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/firmware/binman.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Binman firmware layout
+
+maintainers:
+  - Simon Glass <sjg@chromium.org>
+
+description: |
+  The binman node provides a layout for firmware, used when packaging firmware
+  from multiple projects. For now it just supports a very simple set of
+  features, as a starting point for discussion.
+
+  Documentation for Binman is available at:
+
+  https://u-boot.readthedocs.io/en/latest/develop/package/binman.html
+
+  with the current image-description format at:
+
+  https://u-boot.readthedocs.io/en/latest/develop/package/binman.html#image-description-format
+
+properties:
+  $nodename:
+    const: binman
+
+required:
+  - compatible
+
+additionalProperties: false
+
+examples:
+  - |
+  - |
+    firmware {
+      binman {
+        compatible = "u-boot,binman";
+
+        u-boot {
+          size = <0xa0000>;
+        };
+
+        atf-bl31 {
+          offset = <0x100000>;
+        };
+      };
+    };
diff --git a/dtschema/schemas/firmware/binman/entry.yaml b/dtschema/schemas/firmware/binman/entry.yaml
new file mode 100644
index 0000000..16b84c5
--- /dev/null
+++ b/dtschema/schemas/firmware/binman/entry.yaml
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright 2023 Google LLC
+
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/firmware/binman/entry.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Binman entry
+
+maintainers:
+  - Simon Glass <sjg@chromium.org>
+
+description: |
+  The entry node specifies a single entry in the firmware.
+
+  Entries have a specific type, such as "u-boot" or "atf-bl31". If the type
+  is missing, the name is used as the type.
+
+  Note: This definition is intended to be hierarchical, so that entries can
+  appear in other entries. Schema for that is TBD.
+
+properties:
+  $nodename:
+    pattern: "^[-a-z]+(-[0-9]+)?$"
+
+  type:
+    $ref: /schemas/types.yaml#/definitions/string
+
+  offset:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: |
+      Provides the offset of this entry from the start of its parent section.
+
+  size:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: |
+      Provides the size of this entry in bytes.
+
+additionalProperties: false
+
+examples:
+  - |
+    firmware {
+      binman {
+        compatible = "u-boot,binman";
+
+        u-boot {
+          size = <0xa0000>;
+        };
+
+        second-area {
+          type = "atf-bl31";
+          offset = <0x100000>;
+        };
+      };
+    };
-- 
2.41.0.255.g8b1d071c50-goog

Re: [PATCH 1/2] schemas: Add firmware node schema

[Rob Herring]

On Tue, Jul 11, 2023 at 3:18 PM Simon Glass <sjg@chromium.org> wrote:
>
> Add a motivation and purpose for this new proposed node.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
>  dtschema/schemas/firmware.yaml | 83 ++++++++++++++++++++++++++++++++++
>  1 file changed, 83 insertions(+)
>  create mode 100644 dtschema/schemas/firmware.yaml
>
> diff --git a/dtschema/schemas/firmware.yaml b/dtschema/schemas/firmware.yaml
> new file mode 100644
> index 0000000..4439a70
> --- /dev/null
> +++ b/dtschema/schemas/firmware.yaml
> @@ -0,0 +1,83 @@
> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-clause
> +# Copyright 2023 Google LLC
> +#
> +
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/firmware.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: /firmware Node
> +
> +maintainers:
> +  - Simon Glass <sjg@chromium.org>
> +
> +description: |
> +  The '/firmware' node does not represent a real device, but serves as a place
> +  for recording information about the main firmware used on the device, such as
> +  a map of its contents. This is used by the Operating System (OS), user space
> +  programs and possibly other firmware components. Data in the '/firmware' node
> +  does not itself represent the hardware.
> +
> +  Properties in this node should be common to (and used by) at least two
> +  firmware projects, such as U-Boot and TF-A. Project-specific subnodes can be
> +  used for properties which are specific to a single project.
> +
> +  Purpose of '/firmware' node
> +  ---------------------------
> +
> +  Firmware has traditionally been fairly opaque to the OS, with the OS taking
> +  no interest in its contents, version, layout or how it might be updated. This
> +  is less than ideal, since firmware is an important part of the system and
> +  visibility into its operation is every bit as important as visbility into the
> +  OS and user-space programs within the system.
> +
> +  The traditional approach has been to let firmware deal with firmware, and the
> +  OS deal with everything else. Updating firmware has been handled by firmware.
> +  For example, the UEFI spec defines a way for the OS to post a 'capsule' which
> +  is discovered next time the system boots, permitting firmware updates. But
> +  firmware updates in firmware are highly problematic. They require a reboot
> +  and a sometimes-lengthy wait with a strange-looking interface unfamiliar
> +  to most users. It seems better to make the update as transparent as possible
> +  to the user. As an example of that, ChromeOS has full knowledge of the
> +  firmware version and layout, updates it in the background from user space and
> +  instantly selects the new firmware when the user reboots or logs out.

Perhaps if OS based firmware updates are useful, then UEFI should gain
that capability rather than inventing some way to do it with DT. Seems
like a worthy goal, just needs wider review IMO.

> +  A common objection to considering the system holistically is that some parts
> +  of the system are inaccessible to the OS, such as a secure enclave. However
> +  this does not preclude providing visibility into what is present in that
> +  enclave. Firmware-version information is still useful. Firmware updates are
> +  still needed and can still be initiated from user space.
> +
> +  Another objection is that firmware should provide an interface to the OS,
> +  while keeping its structure private. This thinking is largely driven by
> +  extrapolating from how firmware has been handled in the 'BIOS' days.

It's also the case that the OS may not have direct access to the h/w needed.

> +  It should be considered a degenerate case rather than the norm. As complexity
> +  increases, it creates an artificial boundary between two pieces of the whole.
> +  Mechanisms then need to be invented to cross this unnecessary chasm. An
> +  example of this is Intel's Dynamic Platform and Thermal Framework (DPTF),
> +  which consists of user-space, OS and firmware components all working towards
> +  a shared goal. We need a standard description of these cross-system pieces.
> +
> +  In order to 'teach the OS about firmware', we need a place to put this
> +  information. That is the purpose of this node.
> +
> +  In an Open Source world the entire model of firmware needs to adjust to be
> +  more open, more visible and managed just like any other part of the system.
> +  The major goal is to standardise how firmware is presented to the OS and user
> +  space, so that common utilities can be used to manage the entire system,
> +  including the firmware. For example, fwupd can look in this node for
> +  information on how to update the firmware, similar to how VBE works. [1]
> +  It is likely that other purposes will come to light over time.

It's good we're documenting /firmware, but your use seems different to
what's already in place. Generally, /firmware has been for providers
which are implemented by firmware and are not on any bus. SCMI for
example. PSCI is another example, but it predated using /firmware so
it's typically just put at the top level (and also PSCI should have
been made discoverable like any SMCCC interface).

> +
> +  [1] https://github.com/fwupd/fwupd/tree/main/plugins/vbe
> +
> +properties:
> +  $nodename:
> +    const: firmware
> +
> +  "#address-cells": true
> +  "#size-cells": true

What address space is this? It's not memory mapped because you don't
have 'ranges'.

Rob

Re: [PATCH 1/2] schemas: Add firmware node schema

[Simon Glass]

Hi Rob,

On Fri, 14 Jul 2023 at 10:58, Rob Herring <robh@kernel.org> wrote:
>
> On Tue, Jul 11, 2023 at 3:18 PM Simon Glass <sjg@chromium.org> wrote:
> >
> > Add a motivation and purpose for this new proposed node.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> >
> >  dtschema/schemas/firmware.yaml | 83 ++++++++++++++++++++++++++++++++++
> >  1 file changed, 83 insertions(+)
> >  create mode 100644 dtschema/schemas/firmware.yaml
> >
> > diff --git a/dtschema/schemas/firmware.yaml b/dtschema/schemas/firmware.yaml
> > new file mode 100644
> > index 0000000..4439a70
> > --- /dev/null
> > +++ b/dtschema/schemas/firmware.yaml
> > @@ -0,0 +1,83 @@
> > +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-clause
> > +# Copyright 2023 Google LLC
> > +#
> > +
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/firmware.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: /firmware Node
> > +
> > +maintainers:
> > +  - Simon Glass <sjg@chromium.org>
> > +
> > +description: |
> > +  The '/firmware' node does not represent a real device, but serves as a place
> > +  for recording information about the main firmware used on the device, such as
> > +  a map of its contents. This is used by the Operating System (OS), user space
> > +  programs and possibly other firmware components. Data in the '/firmware' node
> > +  does not itself represent the hardware.
> > +
> > +  Properties in this node should be common to (and used by) at least two
> > +  firmware projects, such as U-Boot and TF-A. Project-specific subnodes can be
> > +  used for properties which are specific to a single project.
> > +
> > +  Purpose of '/firmware' node
> > +  ---------------------------
> > +
> > +  Firmware has traditionally been fairly opaque to the OS, with the OS taking
> > +  no interest in its contents, version, layout or how it might be updated. This
> > +  is less than ideal, since firmware is an important part of the system and
> > +  visibility into its operation is every bit as important as visbility into the
> > +  OS and user-space programs within the system.
> > +
> > +  The traditional approach has been to let firmware deal with firmware, and the
> > +  OS deal with everything else. Updating firmware has been handled by firmware.
> > +  For example, the UEFI spec defines a way for the OS to post a 'capsule' which
> > +  is discovered next time the system boots, permitting firmware updates. But
> > +  firmware updates in firmware are highly problematic. They require a reboot
> > +  and a sometimes-lengthy wait with a strange-looking interface unfamiliar
> > +  to most users. It seems better to make the update as transparent as possible
> > +  to the user. As an example of that, ChromeOS has full knowledge of the
> > +  firmware version and layout, updates it in the background from user space and
> > +  instantly selects the new firmware when the user reboots or logs out.
>
> Perhaps if OS based firmware updates are useful, then UEFI should gain
> that capability rather than inventing some way to do it with DT. Seems
> like a worthy goal, just needs wider review IMO.

Perhaps it should, although it would involve changing the spec, etc.
In any case it would be a very strange world if we mandated UEFI
everywhere.

Yes I am looking for wider review, partly since the work to document
all the firmware-image complexity is happening mostly in U-Boot at
present.

>
> > +  A common objection to considering the system holistically is that some parts
> > +  of the system are inaccessible to the OS, such as a secure enclave. However
> > +  this does not preclude providing visibility into what is present in that
> > +  enclave. Firmware-version information is still useful. Firmware updates are
> > +  still needed and can still be initiated from user space.
> > +
> > +  Another objection is that firmware should provide an interface to the OS,
> > +  while keeping its structure private. This thinking is largely driven by
> > +  extrapolating from how firmware has been handled in the 'BIOS' days.
>
> It's also the case that the OS may not have direct access to the h/w needed.

I tried to cover that in the paragraph you quote immediately
above...what is missing?

>
> > +  It should be considered a degenerate case rather than the norm. As complexity
> > +  increases, it creates an artificial boundary between two pieces of the whole.
> > +  Mechanisms then need to be invented to cross this unnecessary chasm. An
> > +  example of this is Intel's Dynamic Platform and Thermal Framework (DPTF),
> > +  which consists of user-space, OS and firmware components all working towards
> > +  a shared goal. We need a standard description of these cross-system pieces.
> > +
> > +  In order to 'teach the OS about firmware', we need a place to put this
> > +  information. That is the purpose of this node.
> > +
> > +  In an Open Source world the entire model of firmware needs to adjust to be
> > +  more open, more visible and managed just like any other part of the system.
> > +  The major goal is to standardise how firmware is presented to the OS and user
> > +  space, so that common utilities can be used to manage the entire system,
> > +  including the firmware. For example, fwupd can look in this node for
> > +  information on how to update the firmware, similar to how VBE works. [1]
> > +  It is likely that other purposes will come to light over time.
>
> It's good we're documenting /firmware, but your use seems different to
> what's already in place. Generally, /firmware has been for providers
> which are implemented by firmware and are not on any bus. SCMI for
> example. PSCI is another example, but it predated using /firmware so
> it's typically just put at the top level (and also PSCI should have
> been made discoverable like any SMCCC interface).

The intent here is that the node is not on any bus.

I'm happy to create a new node if that is better.

>
> > +
> > +  [1] https://github.com/fwupd/fwupd/tree/main/plugins/vbe
> > +
> > +properties:
> > +  $nodename:
> > +    const: firmware
> > +
> > +  "#address-cells": true
> > +  "#size-cells": true
>
> What address space is this? It's not memory mapped because you don't
> have 'ranges'.

Probably this should be 'false' ? I suppose there may be an address
space, for example in memory-mapped SPI flash, but that is not very
common. The firmware could be stored in eMMC or UFS, which are not
memory-mapped.

Regards,
Simon

Re: [PATCH 1/2] schemas: Add firmware node schema

[Simon Glass]

Hi Rob,

On Fri, 14 Jul 2023 at 11:57, Simon Glass <sjg@chromium.org> wrote:
>
> Hi Rob,
>
> On Fri, 14 Jul 2023 at 10:58, Rob Herring <robh@kernel.org> wrote:
> >
> > On Tue, Jul 11, 2023 at 3:18 PM Simon Glass <sjg@chromium.org> wrote:
> > >
> > > Add a motivation and purpose for this new proposed node.
> > >
> > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > ---
> > >
> > >  dtschema/schemas/firmware.yaml | 83 ++++++++++++++++++++++++++++++++++
> > >  1 file changed, 83 insertions(+)
> > >  create mode 100644 dtschema/schemas/firmware.yaml
> > >
> > > diff --git a/dtschema/schemas/firmware.yaml b/dtschema/schemas/firmware.yaml
> > > new file mode 100644
> > > index 0000000..4439a70
> > > --- /dev/null
> > > +++ b/dtschema/schemas/firmware.yaml
> > > @@ -0,0 +1,83 @@
> > > +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-clause
> > > +# Copyright 2023 Google LLC
> > > +#
> > > +
> > > +%YAML 1.2
> > > +---
> > > +$id: http://devicetree.org/schemas/firmware.yaml#
> > > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > > +
> > > +title: /firmware Node
> > > +
> > > +maintainers:
> > > +  - Simon Glass <sjg@chromium.org>
> > > +
> > > +description: |
> > > +  The '/firmware' node does not represent a real device, but serves as a place
> > > +  for recording information about the main firmware used on the device, such as
> > > +  a map of its contents. This is used by the Operating System (OS), user space
> > > +  programs and possibly other firmware components. Data in the '/firmware' node
> > > +  does not itself represent the hardware.
> > > +
> > > +  Properties in this node should be common to (and used by) at least two
> > > +  firmware projects, such as U-Boot and TF-A. Project-specific subnodes can be
> > > +  used for properties which are specific to a single project.
> > > +
> > > +  Purpose of '/firmware' node
> > > +  ---------------------------
> > > +
> > > +  Firmware has traditionally been fairly opaque to the OS, with the OS taking
> > > +  no interest in its contents, version, layout or how it might be updated. This
> > > +  is less than ideal, since firmware is an important part of the system and
> > > +  visibility into its operation is every bit as important as visbility into the
> > > +  OS and user-space programs within the system.
> > > +
> > > +  The traditional approach has been to let firmware deal with firmware, and the
> > > +  OS deal with everything else. Updating firmware has been handled by firmware.
> > > +  For example, the UEFI spec defines a way for the OS to post a 'capsule' which
> > > +  is discovered next time the system boots, permitting firmware updates. But
> > > +  firmware updates in firmware are highly problematic. They require a reboot
> > > +  and a sometimes-lengthy wait with a strange-looking interface unfamiliar
> > > +  to most users. It seems better to make the update as transparent as possible
> > > +  to the user. As an example of that, ChromeOS has full knowledge of the
> > > +  firmware version and layout, updates it in the background from user space and
> > > +  instantly selects the new firmware when the user reboots or logs out.
> >
> > Perhaps if OS based firmware updates are useful, then UEFI should gain
> > that capability rather than inventing some way to do it with DT. Seems
> > like a worthy goal, just needs wider review IMO.
>
> Perhaps it should, although it would involve changing the spec, etc.
> In any case it would be a very strange world if we mandated UEFI
> everywhere.
>
> Yes I am looking for wider review, partly since the work to document
> all the firmware-image complexity is happening mostly in U-Boot at
> present.
>
> >
> > > +  A common objection to considering the system holistically is that some parts
> > > +  of the system are inaccessible to the OS, such as a secure enclave. However
> > > +  this does not preclude providing visibility into what is present in that
> > > +  enclave. Firmware-version information is still useful. Firmware updates are
> > > +  still needed and can still be initiated from user space.
> > > +
> > > +  Another objection is that firmware should provide an interface to the OS,
> > > +  while keeping its structure private. This thinking is largely driven by
> > > +  extrapolating from how firmware has been handled in the 'BIOS' days.
> >
> > It's also the case that the OS may not have direct access to the h/w needed.
>
> I tried to cover that in the paragraph you quote immediately
> above...what is missing?
>
> >
> > > +  It should be considered a degenerate case rather than the norm. As complexity
> > > +  increases, it creates an artificial boundary between two pieces of the whole.
> > > +  Mechanisms then need to be invented to cross this unnecessary chasm. An
> > > +  example of this is Intel's Dynamic Platform and Thermal Framework (DPTF),
> > > +  which consists of user-space, OS and firmware components all working towards
> > > +  a shared goal. We need a standard description of these cross-system pieces.
> > > +
> > > +  In order to 'teach the OS about firmware', we need a place to put this
> > > +  information. That is the purpose of this node.
> > > +
> > > +  In an Open Source world the entire model of firmware needs to adjust to be
> > > +  more open, more visible and managed just like any other part of the system.
> > > +  The major goal is to standardise how firmware is presented to the OS and user
> > > +  space, so that common utilities can be used to manage the entire system,
> > > +  including the firmware. For example, fwupd can look in this node for
> > > +  information on how to update the firmware, similar to how VBE works. [1]
> > > +  It is likely that other purposes will come to light over time.
> >
> > It's good we're documenting /firmware, but your use seems different to
> > what's already in place. Generally, /firmware has been for providers
> > which are implemented by firmware and are not on any bus. SCMI for
> > example. PSCI is another example, but it predated using /firmware so
> > it's typically just put at the top level (and also PSCI should have
> > been made discoverable like any SMCCC interface).
>
> The intent here is that the node is not on any bus.
>
> I'm happy to create a new node if that is better.
>
> >
> > > +
> > > +  [1] https://github.com/fwupd/fwupd/tree/main/plugins/vbe
> > > +
> > > +properties:
> > > +  $nodename:
> > > +    const: firmware
> > > +
> > > +  "#address-cells": true
> > > +  "#size-cells": true
> >
> > What address space is this? It's not memory mapped because you don't
> > have 'ranges'.
>
> Probably this should be 'false' ? I suppose there may be an address
> space, for example in memory-mapped SPI flash, but that is not very
> common. The firmware could be stored in eMMC or UFS, which are not
> memory-mapped.

Are there any other comments on this? Is there a way to tag it so that
other firmware people might see it?

Regards,
Simon

Re: [PATCH 2/2] schemas: Add a schema for binman

[Alper Nebi Yasak]

Hi Simon,

I know I haven't been able to look at binman stuff for a long time, but
I've been occasionally thinking about it through the course of a year
and think binman severely needs a design-wise review before things get
entrenched, even in the most fundamental parts. I do see the
cross-project collaboration intention here, but still...

There's also the issue of binman having multiple different device-trees:
its input, the ones in fdtmaps per image, the ones injected to U-Boot
dtb files per image. I'd say each has different needs, and those
differences have to be figured out before specified upstream. I can only
guess this is about the one that'll be in a u-boot.dtb.

I want to go through binman more thoroughly, but a lot of changes
happened since I last looked at it and I'm a bit slow at writing things,
so won't exactly be soon. That being said, here's some ideas off the top
of my head, for inspiration on both this schema and binman itself.

On 2023-07-12 00:18 +03:00, Simon Glass wrote:
> I am unsure whether to add this with a generic name, such as 'layout',
> but for now am using /firmware/binman to avoid conflicts with any other
> firmware-layout schema that others might be working on.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---

I've been thinking of compatible = "data,<type>" for entries, so
proposing 'data' here. A big ask, but it might be the one schema to
unify them all if we look at things as "description of data" instead of
"firmware layout".

Also consider data layouts in-memory. For example it could be an
alternative to /chosen linux,initrd-start to specify initramfs location.
Or things like keys or logs received from previous stages / other parts?
Weak examples, but maybe might connect better into firmware handoff
things. (Sorry, I don't know much about those.)

I'm also thinking about things like JPEG/BMP files for ACPI BGRT-like
boot splash, unique firmware/configuration/calibration data for drivers,
but they don't exactly need to be in-memory I guess.

> 
>  dtschema/schemas/firmware/binman.yaml       | 51 ++++++++++++++++++
>  dtschema/schemas/firmware/binman/entry.yaml | 57 +++++++++++++++++++++
>  2 files changed, 108 insertions(+)
>  create mode 100644 dtschema/schemas/firmware/binman.yaml
>  create mode 100644 dtschema/schemas/firmware/binman/entry.yaml
> 
> diff --git a/dtschema/schemas/firmware/binman.yaml b/dtschema/schemas/firmware/binman.yaml
> new file mode 100644
> index 0000000..4b1ecf6
> --- /dev/null
> +++ b/dtschema/schemas/firmware/binman.yaml
> @@ -0,0 +1,51 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +# Copyright 2023 Google LLC
> +
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/firmware/binman.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Binman firmware layout
> +
> +maintainers:
> +  - Simon Glass <sjg@chromium.org>
> +
> +description: |
> +  The binman node provides a layout for firmware, used when packaging firmware
> +  from multiple projects. For now it just supports a very simple set of
> +  features, as a starting point for discussion.
> +
> +  Documentation for Binman is available at:
> +
> +  https://u-boot.readthedocs.io/en/latest/develop/package/binman.html
> +
> +  with the current image-description format at:
> +
> +  https://u-boot.readthedocs.io/en/latest/develop/package/binman.html#image-description-format
> +
> +properties:
> +  $nodename:
> +    const: binman

I think having a single /firmware/binman node for a single image *and*
having a "multiple-images" mode under it is quite bad because the
single-image mode is multiple-images with one image. Multiple images
should be the only mode of operation

Furthermore, each image should be able to function independently of
others, so I think to help enforce that maybe there shouldn't be any
formal binman node. Merely instances of images that may happen to be
under one parent node that may happen to be called /firmware/binman. The
images can be compatible = "data,image" or something and we could search
by that.

Or, heh, a single image node that happens to be named /firmware/binman
because we are embedding it into a u-boot.dtb in that image. But perhaps
we shouldn't restrain things like that. A bit contrived, but why should
we be unable to start from an image in SPI and make it load something
from microSD?

Going even further, let me suggest putting the image nodes as children
of the devices whose layout the image describes. Then I imagine we can
have something like a "data driver" (both for U-Boot and Linux) that
reads the entry data from the parent data/block device and makes it
available for other drivers or in sysfs.

> +
> +required:
> +  - compatible
> +
> +additionalProperties: false

Including the binman version or a version number for the spec might be
useful later on.

> +
> +examples:
> +  - |
> +  - |
> +    firmware {
> +      binman {
> +        compatible = "u-boot,binman";

Consider including "image" or "namespace" in the compatible string to
disambiguate. Not sure of the conventions, "binman,*" could work just as
well, though I'm partial to "data,*" as I've suggested.

> +
> +        u-boot {
> +          size = <0xa0000>;
> +        };
> +
> +        atf-bl31 {
> +          offset = <0x100000>;
> +        };
> +      };
> +    };
> diff --git a/dtschema/schemas/firmware/binman/entry.yaml b/dtschema/schemas/firmware/binman/entry.yaml
> new file mode 100644
> index 0000000..16b84c5
> --- /dev/null
> +++ b/dtschema/schemas/firmware/binman/entry.yaml
> @@ -0,0 +1,57 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +# Copyright 2023 Google LLC
> +
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/firmware/binman/entry.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Binman entry
> +
> +maintainers:
> +  - Simon Glass <sjg@chromium.org>
> +
> +description: |
> +  The entry node specifies a single entry in the firmware.
> +
> +  Entries have a specific type, such as "u-boot" or "atf-bl31". If the type
> +  is missing, the name is used as the type.

I really dislike the name being a fallback for the type, it's guiding
people to using types as the names (or even worse, to assuming names can
only be same as the type), leading to ugly combinations of "blob-ext",
"fdt-N", "fit", "section" and so on. Node names should be descriptive
even when you don't have the device-tree source in front of you, think
of "binman extract" and others. Ones like "u-boot" and "atf-bl31" are
fine only because they are very specific entry types.

> +
> +  Note: This definition is intended to be hierarchical, so that entries can
> +  appear in other entries. Schema for that is TBD.

It might be better to redesign things bottom-up -- first raw data, then
a few data types, then sections and images that hold these. Oh, and I
have a feeling section/image can be the same thing if we think hard
enough (e.g. explicit node labels/references for fdtmap).

> +
> +properties:
> +  $nodename:
> +    pattern: "^[-a-z]+(-[0-9]+)?$"
> +
> +  type:
> +    $ref: /schemas/types.yaml#/definitions/string

Consider compatible = "data,bytes" instead, making it required. Or maybe
"data,blob" but that's not what's there on the binman side. (e.g. rename
"blob" to "file", "blob-ext" could be "file" combined with a
globally-available "optional" property.)

> +
> +  offset:
> +    $ref: /schemas/types.yaml#/definitions/uint32
> +    description: |
> +      Provides the offset of this entry from the start of its parent section.
> +
> +  size:
> +    $ref: /schemas/types.yaml#/definitions/uint32
> +    description: |
> +      Provides the size of this entry in bytes.

Can't we use "reg" for these, and even entry@<addr> in the definitions?
I know the latter conflicts with the fdt@SEQ stuff, but that should be a
binman-input-only thing, and I dislike that anyway. Maybe could work if
we put image nodes under block devices...

Or maybe even better, I think we should make it like FIT: allow a "data"
property that has it inside the dtb, or a pair of "data-offset/position"
and "data-size" properties if it's outside. Inlining data inside the dtb
could help us do nice things in binman, like hashes/signatures as entry
types instead of special-casing them.

In fact it could be possible to turn binman images into a FIT 2.0 if we
do some more work on top of that, instead of nesting a FIT inside a
binman image.

> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    firmware {
> +      binman {
> +        compatible = "u-boot,binman";
> +
> +        u-boot {
> +          size = <0xa0000>;
> +        };
> +
> +        second-area {
> +          type = "atf-bl31";
> +          offset = <0x100000>;
> +        };
> +      };
> +    };

Re: [PATCH 2/2] schemas: Add a schema for binman

[Simon Glass]

Hi Alper,

On Thu, 20 Jul 2023 at 09:23, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote:
>
> Hi Simon,
>
> I know I haven't been able to look at binman stuff for a long time, but
> I've been occasionally thinking about it through the course of a year
> and think binman severely needs a design-wise review before things get
> entrenched, even in the most fundamental parts. I do see the
> cross-project collaboration intention here, but still...

Yes, I agree. I was expecting a few more comments but I suppose not
that many people worry too much about open firmware. Many of the
points you've mentioned here have been in my mind as we've moved from
a prototype 6 years ago to what Binman is today, and I know we've
discussed some before on the mailing list. I really appreciated you
collecting your thoughts on this. The whole thing needs a new look and
I hope that the discussion here can help us settle on an industry
standard for this stuff.

So...

>
> There's also the issue of binman having multiple different device-trees:
> its input, the ones in fdtmaps per image, the ones injected to U-Boot
> dtb files per image. I'd say each has different needs, and those
> differences have to be figured out before specified upstream. I can only
> guess this is about the one that'll be in a u-boot.dtb.

Well, there is really only one. The fdtmap is actually the same
schema, except that it mentions only the image that it is embedded in,
i.e. if the fdtmap is for the SPI image, then the fdtmap in SPI flash
only contains the definition for the SPI image, not the MMC image
which is in a different device. The input is the same schema, albeit
that things like 'offset' may be filled in by Binman automatically
when it packs things.

>
> I want to go through binman more thoroughly, but a lot of changes
> happened since I last looked at it and I'm a bit slow at writing things,
> so won't exactly be soon. That being said, here's some ideas off the top
> of my head, for inspiration on both this schema and binman itself.

Do you mean the code? There are definitely some abstractions that are
stretching a bit, but it is mostly holding together for now.

>
> On 2023-07-12 00:18 +03:00, Simon Glass wrote:
> > I am unsure whether to add this with a generic name, such as 'layout',
> > but for now am using /firmware/binman to avoid conflicts with any other
> > firmware-layout schema that others might be working on.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
>
> I've been thinking of compatible = "data,<type>" for entries, so
> proposing 'data' here. A big ask, but it might be the one schema to
> unify them all if we look at things as "description of data" instead of
> "firmware layout".

As I mentioned in the commit message I was a bit unsure about just
coming out with a generic name, but let's do it and see what people
think.

>
> Also consider data layouts in-memory. For example it could be an
> alternative to /chosen linux,initrd-start to specify initramfs location.
> Or things like keys or logs received from previous stages / other parts?
> Weak examples, but maybe might connect better into firmware handoff
> things. (Sorry, I don't know much about those.)

Yes I suspect at the high level those are better handled by firmware
handoff, which is some other work going on. But I would like this to
be a standard for firmware images, so BInman could perhaps plug into
the handoff size of things too,

>
> I'm also thinking about things like JPEG/BMP files for ACPI BGRT-like
> boot splash, unique firmware/configuration/calibration data for drivers,
> but they don't exactly need to be in-memory I guess.

Sure,

>
> >
> >  dtschema/schemas/firmware/binman.yaml       | 51 ++++++++++++++++++
> >  dtschema/schemas/firmware/binman/entry.yaml | 57 +++++++++++++++++++++
> >  2 files changed, 108 insertions(+)
> >  create mode 100644 dtschema/schemas/firmware/binman.yaml
> >  create mode 100644 dtschema/schemas/firmware/binman/entry.yaml
> >
> > diff --git a/dtschema/schemas/firmware/binman.yaml b/dtschema/schemas/firmware/binman.yaml
> > new file mode 100644
> > index 0000000..4b1ecf6
> > --- /dev/null
> > +++ b/dtschema/schemas/firmware/binman.yaml
> > @@ -0,0 +1,51 @@
> > +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> > +# Copyright 2023 Google LLC
> > +
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/firmware/binman.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Binman firmware layout
> > +
> > +maintainers:
> > +  - Simon Glass <sjg@chromium.org>
> > +
> > +description: |
> > +  The binman node provides a layout for firmware, used when packaging firmware
> > +  from multiple projects. For now it just supports a very simple set of
> > +  features, as a starting point for discussion.
> > +
> > +  Documentation for Binman is available at:
> > +
> > +  https://u-boot.readthedocs.io/en/latest/develop/package/binman.html
> > +
> > +  with the current image-description format at:
> > +
> > +  https://u-boot.readthedocs.io/en/latest/develop/package/binman.html#image-description-format
> > +
> > +properties:
> > +  $nodename:
> > +    const: binman
>
> I think having a single /firmware/binman node for a single image *and*
> having a "multiple-images" mode under it is quite bad because the
> single-image mode is multiple-images with one image. Multiple images
> should be the only mode of operation

Yes, I think that is pretty clear now. Most boards have ended up with
more than a single image, which I had not envisaged at the start.

>
> Furthermore, each image should be able to function independently of
> others, so I think to help enforce that maybe there shouldn't be any
> formal binman node. Merely instances of images that may happen to be
> under one parent node that may happen to be called /firmware/binman. The
> images can be compatible = "data,image" or something and we could search
> by that.

An image is a section with some extra properties. But yes that seems OK to me.

>
> Or, heh, a single image node that happens to be named /firmware/binman
> because we are embedding it into a u-boot.dtb in that image. But perhaps
> we shouldn't restrain things like that. A bit contrived, but why should
> we be unable to start from an image in SPI and make it load something
> from microSD?

Well, you can do that, although U-Boot currently has no mechanism to
connect the binman definition to the media. We need use cases for this
stuff, IMO, since designing something before it is needed by a
particular board is just a pain.

>
> Going even further, let me suggest putting the image nodes as children
> of the devices whose layout the image describes. Then I imagine we can
> have something like a "data driver" (both for U-Boot and Linux) that
> reads the entry data from the parent data/block device and makes it
> available for other drivers or in sysfs.

Yes, we have that on x86 actually (grep for "fwstore_spi" in U-Boot).
I'm sure people will weigh in on this, but I feel it is a natural
thing to do. Of course, we may have the same image structure which can
appear on SD or eMMC, so perhaps we should have a property in the
device, like data,image-desc = <&binman_image_node> so that the image
desc can be in the /firmware node.

>
> > +
> > +required:
> > +  - compatible
> > +
> > +additionalProperties: false
>
> Including the binman version or a version number for the spec might be
> useful later on.

I worry a little about that. Devicetree schemas are supposed to be
backwards-compatible. But you are right in that we may add new
features later and that may involve new properties. As an example, the
'image-pos' property was added to Binman long after the 'offset'
property. Perhaps we need a way for readers to see what level of
support is present in the file?

But for now I am going to leave it out, since I'm not sure what to do
here. If you have a proposal, please let me know.

>
> > +
> > +examples:
> > +  - |
> > +  - |
> > +    firmware {
> > +      binman {
> > +        compatible = "u-boot,binman";
>
> Consider including "image" or "namespace" in the compatible string to
> disambiguate. Not sure of the conventions, "binman,*" could work just as
> well, though I'm partial to "data,*" as I've suggested.

OK I can try 'data,image' and see what people think.

>
> > +
> > +        u-boot {
> > +          size = <0xa0000>;
> > +        };
> > +
> > +        atf-bl31 {
> > +          offset = <0x100000>;
> > +        };
> > +      };
> > +    };
> > diff --git a/dtschema/schemas/firmware/binman/entry.yaml b/dtschema/schemas/firmware/binman/entry.yaml
> > new file mode 100644
> > index 0000000..16b84c5
> > --- /dev/null
> > +++ b/dtschema/schemas/firmware/binman/entry.yaml
> > @@ -0,0 +1,57 @@
> > +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> > +# Copyright 2023 Google LLC
> > +
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/firmware/binman/entry.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Binman entry
> > +
> > +maintainers:
> > +  - Simon Glass <sjg@chromium.org>
> > +
> > +description: |
> > +  The entry node specifies a single entry in the firmware.
> > +
> > +  Entries have a specific type, such as "u-boot" or "atf-bl31". If the type
> > +  is missing, the name is used as the type.
>
> I really dislike the name being a fallback for the type, it's guiding
> people to using types as the names (or even worse, to assuming names can
> only be same as the type), leading to ugly combinations of "blob-ext",
> "fdt-N", "fit", "section" and so on. Node names should be descriptive
> even when you don't have the device-tree source in front of you, think
> of "binman extract" and others. Ones like "u-boot" and "atf-bl31" are
> fine only because they are very specific entry types.

Well this doesn't have to be in the schema itself, since Binman can
state that it handles fixing up these things. I really like the
name-as-type thing since it simplifies so much of the documentation,
test dts files and makes it much easier for people to understand
initially.

>
> > +
> > +  Note: This definition is intended to be hierarchical, so that entries can
> > +  appear in other entries. Schema for that is TBD.
>
> It might be better to redesign things bottom-up -- first raw data, then
> a few data types, then sections and images that hold these. Oh, and I
> have a feeling section/image can be the same thing if we think hard
> enough (e.g. explicit node labels/references for fdtmap).

Yes section and image are essentially the same, as you can see in the
code. But we do need top-level nodes which represent images.

>
> > +
> > +properties:
> > +  $nodename:
> > +    pattern: "^[-a-z]+(-[0-9]+)?$"
> > +
> > +  type:
> > +    $ref: /schemas/types.yaml#/definitions/string
>
> Consider compatible = "data,bytes" instead, making it required. Or maybe
> "data,blob" but that's not what's there on the binman side. (e.g. rename
> "blob" to "file", "blob-ext" could be "file" combined with a
> globally-available "optional" property.)

I don't like 'file' at all, sorry. Where it comes from is not the
point here; we are worried about what it is. Since we have no
information on its format, it is a blob, IMO. Binman has an "optional"
property which means that the entry itself is optional.

For now I just have a generic entry, with no definition of the
available types. For a first series I very much want to avoid getting
into this level of detail. It is easy to add stuff later.

>
> > +
> > +  offset:
> > +    $ref: /schemas/types.yaml#/definitions/uint32
> > +    description: |
> > +      Provides the offset of this entry from the start of its parent section.
> > +
> > +  size:
> > +    $ref: /schemas/types.yaml#/definitions/uint32
> > +    description: |
> > +      Provides the size of this entry in bytes.
>
> Can't we use "reg" for these, and even entry@<addr> in the definitions?
> I know the latter conflicts with the fdt@SEQ stuff, but that should be a
> binman-input-only thing, and I dislike that anyway. Maybe could work if
> we put image nodes under block devices...

That was an early decision, based on the fact that generally you don't
specify the offset or size, so they are optional; reg does not work
well with an optional address or size. Also, we have image-pos as the
absolute position (which again I have elected to worry about later
once we have an agreed initial schema). Does reg mean an absolute
address or a relative one ('ranges', anyone...nooooo!!!)? To me it is
stretching the meaning.

That said, I can see some benefit in writing a 'reg' property when all
the entries are packed:

reg = <offset size>

i.e. the offset is relative to its containing section. That would
simplify some of the 'replace' logic, since offset and size can be
left in there.

>
> Or maybe even better, I think we should make it like FIT: allow a "data"
> property that has it inside the dtb, or a pair of "data-offset/position"
> and "data-size" properties if it's outside.

Eh? The point of the entry is to declare the position of actual data.
If the data is elsewhere, then the entry will be too.

> Inlining data inside the dtb
> could help us do nice things in binman, like hashes/signatures as entry
> types instead of special-casing them.

We already do that though, right? See testHash() for example. This is
a powerful feature of a firmware-layout description / Binman, IMO,
since all the metadata you need is right there.

>
> In fact it could be possible to turn binman images into a FIT 2.0 if we
> do some more work on top of that, instead of nesting a FIT inside a
> binman image.

Well binman needs to produce things which are not FITs. Bear in mind
that the output can be a binary file in any format. It doesn't
necessarily have to have any metadata in it at all, certainly not a
FIT header.

>
> > +
> > +additionalProperties: false
> > +
> > +examples:
> > +  - |
> > +    firmware {
> > +      binman {
> > +        compatible = "u-boot,binman";
> > +
> > +        u-boot {
> > +          size = <0xa0000>;
> > +        };
> > +
> > +        second-area {
> > +          type = "atf-bl31";
> > +          offset = <0x100000>;
> > +        };
> > +      };
> > +    };

Thanks again for your encouraging comments. I'll do a v2 with the above changes.

Regards,
Simon