Devicetree Compiler
 help / color / mirror / Atom feed
* [PATCH v2 00/15] dtc: Merge the fragments of a dts file after parsing
@ 2026-11-04 21:17 Simon Glass
  2026-11-04 21:17 ` [PATCH v2 01/15] tests: Check each error when merging a dts file Simon Glass
                   ` (14 more replies)
  0 siblings, 15 replies; 22+ messages in thread
From: Simon Glass @ 2026-11-04 21:17 UTC (permalink / raw)
  To: Devicetree Compiler; +Cc: Herve Codina, David Gibson, Simon Glass

The parser merges each top-level fragment of a dts file into the tree as
soon as it is parsed, so nothing can look at a fragment on its own. Here
a fragment is a top-level part of the source, such as / { ... }; or
&label { ... };, not an overlay's fragment@N node.

This causes several problems:

- A repeated node or property name gives a check failure in the first
  fragment, but silently overwrites when it is in a later fragment
- Deletes cannot be dealt with consistently
- Checks which need the whole tree give false positives for a plugin,
  yet checks which do not need it cannot run before the merge

The ultimate aim is to run most checks on each fragment before it is
merged, and only those which need the whole tree afterwards. For this to
work, we need the merge to come after dt_from_source() and
build_dt_info() have returned.

As a first step towards keeping the fragments apart until after parsing,
move the merge logic out of the grammar actions into livetree.c, one
action at a time, then have the parser return a list of fragments and
merge them from main(). The output only changes for &{label} { ... } in
a plugin, where label is in the plugin itself, which is now merged in
the same way as &label { ... }. The only changes to the errors are the
final message when a reference cannot be resolved, that a syntax error
later in the file now stops dtc before any earlier reference errors are
reported, and that reference errors now come after any other errors from
the parser.

Most of the merge errors have no tests, so the series starts by adding a
test for each one, checking the message and its position in the source.
Every line of the new merge code is then covered by the tests. At each
commit, make check passes and dtc gives the same output for all 4246
.dts and .dtso files in Linux v7.3-12284-g3799eeb906c9, with the same
messages apart from the changes above.

This series only moves the merge. It still happens straight after the
input is read, before any checks, so none of the problems above are
fixed yet, and all fragments still come from source rather than a dtbo.

Following this, the plan is to apply deletes while merging, deciding
what a delete in a plugin should do, and then to run some checks on each
fragment.

Link: https://lore.kernel.org/devicetree-compiler/auK0ERvxGCJGSc3p@gractus.seuss/

Changes in v2:
- Reword four comments in the test files
- Add merge-root-plugin.dts to check / { ... } against &{/} { ... }
- Add merge-label-base.dts to check a label which is not in the plugin
- Add merge-label-braces.dts to check &{label} in a plugin
- Add merge-label-order.dts to check the order of &{label} and &label
- Add new patch for srcpos_string()
- Replace FRAGMENT_ROOT and FRAGMENT_REF with FRAGMENT_NODE
- Use a NULL ref for the root node
- Drop path_ref in favour of checking the ref for a leading '/'
- Add a constructor for each kind of fragment, taking all its fields
- Add the list to struct dt_info later, when it is first used
- Change the subject from 'Add a list' to 'Add a struct'
- Use dt_ref instead of fragment_ref
- Return true on success from merge_delete_node()
- Check the fragment type with assert()
- Use dt_ref instead of fragment_ref
- Return true on success from merge_omit_if_no_ref()
- Check the fragment type with assert()
- Rename merge_label_ref() to merge_ref()
- Build the fragment in one step, with build_fragment_ref()
- Return true on success from merge_ref()
- Check the fragment type with assert()
- Add new patch for &{label} in a plugin
- Combine merge_path_ref() into merge_ref()
- Combine the rules for label and path references, using dt_ref
- Combine merge_labelled_ref() into merge_ref()
- Use dt_ref instead of fragment_ref
- Use dt_ref instead of fragment_ref
- Return true on success from merge_first_ref()
- Use the new constructors and return true on success
- Use dt_ref instead of fragment_ref
- Combine the label and path rules earlier, in the path-reference patch
- Return true from check_relative_ref() if the reference is OK
- Pass the list to build_dt_info(), adding it to struct dt_info here
- Return true on success from merge_fragments()
- Explain the aim of the series and what it does not do
- Update for merge_fragments() returning true on success

Simon Glass (15):
  tests: Check each error when merging a dts file
  srcpos: Allow srcpos_string() to accept NULL
  dtc: Add a struct for the top-level fragments of a dts file
  dtc: Move the /delete-node/ handling out of the grammar
  dtc: Move the /omit-if-no-ref/ handling out of the grammar
  dtc: Move is_ref_relative() to livetree.c
  dtc: Move label-reference handling out of the grammar
  dtc: Merge &{label} in a plugin as with &label
  dtc: Move path-reference handling out of the grammar
  dtc: Move labelled-reference handling out of the grammar
  dtc: Move first-reference handling out of the grammar
  dtc: Add merge_fragment() to merge any fragment
  dtc: Add helpers for the errors when merging a fragment
  dtc: Merge the fragments of a dts file after parsing
  dtc: Merge the fragments of a dts file in main()

 dtc-parser.y                       | 109 ++---------
 dtc.c                              |   2 +
 dtc.h                              |  55 +++++-
 flattree.c                         |   2 +-
 fstree.c                           |   2 +-
 livetree.c                         | 294 ++++++++++++++++++++++++++++-
 srcpos.c                           |   3 +
 tests/merge-delete-missing.dts     |   9 +
 tests/merge-errors.dts             |  14 ++
 tests/merge-first-ref-relative.dts |  10 +
 tests/merge-first-ref.dts          |   9 +
 tests/merge-label-base.dts         |  10 +
 tests/merge-label-braces.dts       |  12 ++
 tests/merge-label-order.dts        |  17 ++
 tests/merge-labelled-plugin.dts    |  12 ++
 tests/merge-labelled-relative.dts  |  14 ++
 tests/merge-omit-missing.dts       |   9 +
 tests/merge-parse-error.dts        |  17 ++
 tests/merge-path-ref-missing.dts   |  11 ++
 tests/merge-path-relative.dts      |  12 ++
 tests/merge-root-plugin.dts        |  12 ++
 tests/run_tests.sh                 |  70 ++++++-
 treesource.c                       |   5 +-
 23 files changed, 613 insertions(+), 97 deletions(-)
 create mode 100644 tests/merge-delete-missing.dts
 create mode 100644 tests/merge-errors.dts
 create mode 100644 tests/merge-first-ref-relative.dts
 create mode 100644 tests/merge-first-ref.dts
 create mode 100644 tests/merge-label-base.dts
 create mode 100644 tests/merge-label-braces.dts
 create mode 100644 tests/merge-label-order.dts
 create mode 100644 tests/merge-labelled-plugin.dts
 create mode 100644 tests/merge-labelled-relative.dts
 create mode 100644 tests/merge-omit-missing.dts
 create mode 100644 tests/merge-parse-error.dts
 create mode 100644 tests/merge-path-ref-missing.dts
 create mode 100644 tests/merge-path-relative.dts
 create mode 100644 tests/merge-root-plugin.dts

---
base-commit: 54e547173bf0f2a4cc35eba8b5738caf38b9230f
branch: dt-forest2

-- 
2.43.0


^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2026-11-06 17:31 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-04 21:17 [PATCH v2 00/15] dtc: Merge the fragments of a dts file after parsing Simon Glass
2026-11-04 21:17 ` [PATCH v2 01/15] tests: Check each error when merging a dts file Simon Glass
2026-11-04 21:17 ` [PATCH v2 02/15] srcpos: Allow srcpos_string() to accept NULL Simon Glass
2026-11-04 21:17 ` [PATCH v2 03/15] dtc: Add a struct for the top-level fragments of a dts file Simon Glass
2026-11-04 21:17 ` [PATCH v2 04/15] dtc: Move the /delete-node/ handling out of the grammar Simon Glass
2026-11-04 21:17 ` [PATCH v2 05/15] dtc: Move the /omit-if-no-ref/ " Simon Glass
2026-11-04 21:17 ` [PATCH v2 06/15] dtc: Move is_ref_relative() to livetree.c Simon Glass
2026-11-04 21:17 ` [PATCH v2 07/15] dtc: Move label-reference handling out of the grammar Simon Glass
2026-11-05  9:07   ` Herve Codina
2026-11-05 21:49     ` Simon Glass
2026-11-06  6:56       ` Herve Codina
2026-11-06 13:05         ` Simon Glass
2026-11-06 15:55           ` Herve Codina
2026-11-06 17:30             ` Simon Glass
2026-11-04 21:17 ` [PATCH v2 08/15] dtc: Merge &{label} in a plugin as with &label Simon Glass
2026-11-04 21:17 ` [PATCH v2 09/15] dtc: Move path-reference handling out of the grammar Simon Glass
2026-11-04 21:17 ` [PATCH v2 10/15] dtc: Move labelled-reference " Simon Glass
2026-11-04 21:17 ` [PATCH v2 11/15] dtc: Move first-reference " Simon Glass
2026-11-04 21:17 ` [PATCH v2 12/15] dtc: Add merge_fragment() to merge any fragment Simon Glass
2026-11-04 21:17 ` [PATCH v2 13/15] dtc: Add helpers for the errors when merging a fragment Simon Glass
2026-11-04 21:17 ` [PATCH v2 14/15] dtc: Merge the fragments of a dts file after parsing Simon Glass
2026-11-04 21:17 ` [PATCH v2 15/15] dtc: Merge the fragments of a dts file in main() Simon Glass

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox