Re: Device tree representation of (hotplug) connectors: discussion at ELCE

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 24804 bytes --]

On Mon, Sep 08, 2025 at 02:51:55PM +0200, Herve Codina wrote:
> Hi David,
> 
> On Mon, 8 Sep 2025 14:36:06 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Thu, Sep 04, 2025 at 11:15:44AM +0530, Ayush Singh wrote:
> > > On 9/4/25 10:53, David Gibson wrote:  
> > > > On Tue, Sep 02, 2025 at 10:57:10AM +0200, Luca Ceresoli wrote:  
> > [snip]
> > > > 1) Connector local labels/symbols/aliases
> > > > 
> > > > This is not a new idea - both the export-symbols proposal and my
> > > > ancient connector proposal had this in one form or another.  I think
> > > > something along these lines is almost essential.  Things that plug
> > > > into connectors almost always require references to several host board
> > > > resources (interrupt controller, gpio, ...).  In order to be pluggable
> > > > on multiple host boards you want to refer to those symbolically.  In
> > > > order to support multiple instances of the same connector type, you
> > > > need those symbols to refer to different things fordifferent connector
> > > > instances.
> 
> Some of the resources can be "translated" using a nexus node.
> It works for interrupt, gpio, pwm, ...

Yes, I expect that will be necessary.

> I fact, it should work for all references that use a phandle with a resource
> number. I mean any reference in the form 'ref = <&ctrl 123>' to reference the
> resource 123 of the controller.

Well, as long as they have a way of defining a nexus / map.  That's
true of interrupts of course, but I'm not really familiar with the
more modern instances.

> From the addon, &ctrl need to be resolved. Using a nexus node at connector level
> this becomes "ref = <&connector 10>;" and considering the connector itself in the

Right, this requires at least 'connector' to be resolvable as an
external resource.  If that's possible, I'm not sure that allowing
multiple symbols to be resolved to the base tree is notably more
complex.

> base tree as a nexus node allows to perform the translation.
>    connector {
>       gpio-map = <10 &ctrl 123>;
>    };
> 
> but this don't work for a reference that uses only a phandle without any
> number (pinctrl for instance) or for busses where devices need to be added.

That might be true in practice, but there's nothing inherently true
about this - it just depends on what the bindings define in terms of
nexus / remapping options.

> For busses where devices could be added by the add-on, bus extensions were
> introduced (i2c-bus-extension).

I think bus-reg is a more natural way to do this for any bus without
needing to set up bindings for each new bus type.

> In any cases, some symbols need to be referenced from the addon.

Well, certain base tree nodes need to be referred to by the addon, so
"symbols" in some sense.  That doesn't necessarily have to be done by
the existing symbols mechanism.

> > > > Whhat I think is a mistake is trying to tie this too closely to the
> > > > existing __symbols__ structure.  Those have an ugly encoding that
> > > > requires tortured processing in a way that's not natural for dtb
> > > > handling.  Plus they already kinda-sorta duplicate old-school aliases
> > > > in an odd way.
> > > > 
> > > > You want some sort of string => node mapping on the connector side,
> > > > and a way to mark portions of properties on the plugin side as being
> > > > resolved to some string reference.  But we can take the opportunity to
> > > > design a better way of doing that than the ugly one we have now.  
> > > 
> > > 
> > > Isn't export-symbols exactly this. We do take inspiration from __symbols__.
> > > However, in case of export-symbols, its string => phandle mapping (as
> > > opposed to string => string in __symbols__).  
> > 
> > As far as the specific It kind of is, yes, and that aspect I like.
> > What I'm not convinced by is how export-symbols is proposed to fit in
> > with and be used by surrounding logic.  export-symbols has been
> > designed to fit in with the existing (ugly) overlay mechanism.  I
> > think that's putting the cart before the horse.  Instead work out how
> > to logically define connectors first - which will involve information
> > equivalent to export-symbols - then work out how to update or replace
> > the overlay mechanism to work with that.
> 
> I think that a connector is something with a bunch of resources provided
> by the board where the connector is soldered. Those resources are wired
> to the connector and defined by the connector pinout.
> 
>      3v3   ------- Pin 0
>   i2c_scl  ------- Pin 1
>   i2c_sda  ------- Pin 2
>     gpio A ------- Pin 3
>     gpio B ------- Pin 4
>      gnd   ------- Pin 5
> 
> IMHO, this need to be described and defined in the base board and an addon can
> only reference resources wired and described by the connector node.

Yes, that's exactly what I'm proposing too.

> Now, questions are:
>   - 1) How to describe a connector?
>   - 2) How to reference resources provided at connector level from an add-on?

Right.

> Our current approach was:
> ---- base board DT ----
>   connector0 {
> 	gpio-map = <0 &gpio0 12>, /* gpio A wired to gpio 12 of gpio0 controller */
>                    <1 &gpio2 10;  /* gpio B wired to gpio 10 of gpio2 controller */
>         i2c-one {
> 		compatible = "i2c-bus-extension";
> 		i2c-parent = <i2c5>; /* i2c-one wired to i2c5 controller */
> 	};
> 
> 	i2c-two {
> 		compatible = "i2c-bus-extension";
> 		i2c-parent = <i2c6>; /* i2c-two wired to i2c6 controller */
> 	};
> 
> 	/*
>          * From the addon we need to reference:
>          *    - The connector itself,
>          *    - Maybe some pinctrl related to signals wired to the connector,
>          *    - In some cases the i2c bus (HDMI, ddc-i2c-bus = <&i2c-two>;)
>          * 
>          * This was solved introducing the controversial export-symbols node.
>          */

I think the type of connector should also be named on both sides (with
'compatible' or something like it).

>   };
> 
> ---- addon board DT ----
>    {
> 	some-node {
> 		compatible = "foo,bar";
> 		reset-gpios = <&connector 0>; /* gpio A used as a reset gpio */
> 		ddc-i2c-bus = <&i2c-two>;
>         }
> 
>         i2c-one {
> 		eeprom@10 {
> 			compatible = "baz,eeprom"
> 			reg = 10; 
> 		};
> 	};
>    };
> 
> The addon board DT can only be applied at a connector node.

Right.  This is not how overlays work now.  By the nature of how
they're built they apply global updates to the base tree.  That means
we need to spec a new way of describing addons that *is* restricted to
a particular connector slot (or slots, as Geert points out).  Since we
have that opportunity, we should _not_ try to make it a minimal
extension to existing overlay format, but define a new and better
encoding, designed to meet the problems you're looking to address.

> It described the addon board connected to this connector.
> 
> > 
> > > I suppose export-symbols could follow aliase conventions, but that still is
> > > a string => string mapping, which seems worse to me than a phandle (since
> > > phandle size is constant).
> > > 
> > >   
> > > > 
> > > > 2) Extend dtb itself
> > > > 
> > > > A maor thing that makes current symbols and fixups ugly is the fact
> > > > that they are encoded into properties in the device tree itself,
> > > > despite being logically at a different semantic level.  Obviously you
> > > > *can* do that, but it's not natural.  It would make more sense to add
> > > > fixup tags into the dtb format itself.  
> > > 
> > > Having something akin to fixup in dtb format itself would be nice.  
> > 
> > Yes, it would.
> 
> What could be the modification expected at dtb to support the connector
> use case?

I don't entirely understand the question.  It kind of depends what's
needed.  But certainly FDT_REF_PHANDLE and/or FDT_REF_PATH tags seem
reasonable to me.  A new symbols/aliases/whatever section in the dtb
(separate from the structure block) could also be considered.

> Also what could be the modification expected at dts to described the
> connector?

One thing I'm hoping is that the new structure can be defined in such
a way that not much dts extension is needed.

> > > > 3) bus-reg / bus-ranges
> > > > 
> > > > One thing that makes connector plugins a bit awkward is that they
> > > > often need to add things to multiple buses on the host system (MMIO &
> > > > i2c for a simple case).  This means that once resolved the plugin
> > > > isn't neatly a single subtree.  That's one factor making removal
> 
> It can be a single subtree if decoupling is present at connector node available
> in the base device tree.

Right - allowing that decoupling is exactly what I'm proposing bus-reg
for.  Note that the case of an addon that plugs into multiple
connectors that Geert pointed out complicates this again.

> All resources wired to the connector should be described in the connector node
> A add-on board should only see resources provided and described at the connector
> node or translated by something (nexus, export-symbols) present in the connector
> node.

Agreed.

> > > > really awkward.  Here's an idea I had a while ago to allow plugins to
> > > > be a single subtree, by extending what's allowed in the tree content:
> > > > 
> > > > Currently a node can only really have a presence on its immediate
> > > > parent bus, as encoded in the 'reg' and 'ranges' properties.
> > > > 'bus-reg' and 'bus-ranges' would extend that having a similar format
> > > > to 'reg' and 'ranges' but adding a phandle for each entry saying which
> > > > bus it lives on - somewhat similar to interrupt-map.
> > > > 
> > > > For example, here's an MMIO bus bridge of some sort, which has control
> > > > registers on I2C:
> > > > 
> > > > 	mmio-bus@... {
> > > > 		#address-cells = < 2 >;
> > > > 		#size-cells = < 2 >;
> > > > 		bridge@XXXX {
> > > > 			ranges = <...>;
> > > > 			bus-reg = <&i2c0 0x407>
> > > > 		}
> > > > 	}
> > > > 	i2c0: i2c@... {
> > > > 		#address-cells = < 1 >;
> > > > 		#size-cells = < 0 >;
> > > > 	}
> > > > 
> > > > In a sense this extends the device tree to a device DAG.
> > > > 
> > > > Obviously this does need changes at the OS device core level, but it
> > > > gives you a lot of flexibility having done so.  
> > > 
> > > There is an i2c-bus-extension [1] and spi-bus-extension proposal to do the
> > > same. But, if we can figure out a common way for all buses, that would be
> > > great.
> 
> Exactly, this is the purpose of bus extensions.

Right, but redefining it for each bus type seems silly.

> Also, I don't thing that the 'ranges' property should be used for that purpose.
> The 'ranges' property is used to translate addresses from child addresses space
> to parent addresses space.

The rationale for bus-ranges is that the add-on board could re-expose
one of the buses on the connector (say MMIO for simplicity) to several
chips physically included on the addon board.  We don't want those
sub-chips to need different device nodes depending on whether they're
on an addon board or wired directly onto a main board.  bus-ranges
would allow the root of the connector to create a subtree in which
regular MMIO (or whatever) devices can be described, and then routed
via the connector onto the bus on the main board.

> For instance, in case of i2c, where is the address translation?

I think i2c multiplexers can sometimes do that.  But in any case,
ranges can be used for 1:1 translation as well.

> The address of a child (device) is its I2C address. This address is
> device in its datasheet. There is no reason to have this address depending
> on the I2C bus this child is connected to.

Maybe not for I2C, but other buses can certainly allow translation, so
the flexibility is needed.

> In your example, the bridge@XXXX is not related to any hardware components.

I'm not sure what you mean by that.

> If is allows to physically perform I2C transaction from a MMIO device
> connected to MMIO bus, this is a traduction I2C controller with a bunch of
> registers to configure an perform I2C transactions.

Yes, absolutely.  The i2c in bus-reg there is not supposed to indicate
that the bridge is used to access i2c devices.  Rather it's saying
that the bridge itself has configuration registers accessible via i2c.

If a device does have its own i2c bus hanging off, then as you say
there should not be a 'ranges' property.  It would have its own i2c
controller driver.

bus-ranges could be used when a connector has pins on multiple MMIO
buses, and an addon could have sub-devices on either one of them.

Note that bus-ranges translated between i2c buses isn't theoretically
impossible: that would represent a case with some sort of I2C "relay"
device that forwards I2C commands from one side to the other altering
the address on the way.  I don't know if such devices exist in
practice, but they're certainly theoretically possible (and could be
used, e.g. for allowing 10-bit I2C devices to be accessed from an old
7-bit I2C bus).

> In that case, this "bridge" looks like all I2C we already support in the
> kernel.
> Having the additional indirection with the "bridge" between the MMIO bus
> and the i2c@... node seems not a hardware representation of the system.
> 
> Did I miss something?

I think so.  That's representing the bridge itself being configured
via I2C, not allowing I2C access to devices _on_ the bus.

> > Heh, right.  That reinforces my thought that this could be a good
> > idea.
> > 
> > [Btw, the theoretically correct IEEE1275 way to do this is change
> >  addressing across the whole tree.  e.g. set #address-cells = <3>,
> >  with the first cell being, say, 0 for mmio, 1 for i2c, 2 for SPI,
> >  then the remaining cells are the address within that bus.  So, this
> >  sort of thing is technically possible in old-school OF trees.  That
> >  would become pretty unmanageable though.  The idea of bus-reg is to
> >  encode the same information in a less awkward way]
> > 
> > > [1]:
> > > https://lore.kernel.org/all/20250618082313.549140-1-herve.codina@bootlin.com/
> > > 
> > > [2]: https://lore.kernel.org/all/20250729-spi-bus-extension-v1-0-b20c73f2161a@beagleboard.org/
> > > 
> > >   
> > > > 4) You don't necessarily need to build a "full" device tree
> > > > 
> > > > Flattened device trees (as opposed to original IEEE1275 device trees)
> > > > - by design - allow certain information to be omitted.  The most
> > > > common example is that for introspectable buses, like PCI, it's normal
> > > > to have the DT only include a node for the host bridge, with devices
> > > > under it being discovered by their own bus specific methods.  That's
> > > > discovery is handled by the bus/bridge driver.
> > > > 
> > > > Connectors usually aren't introspectable, but it's still possible to
> > > > use an approach like this where the connector driver's discovery
> > > > method is "look at a different device tree".  So, for example,
> > > > 
> > > > Board device tree:
> > > > 
> > > > / {
> > > > 	compatible = "board-with-foo-connector";
> > > > 	. . .
> > > > 	mmio@... {
> > > > 		foo-connector@... {
> > > > 			compatible = "foo-connector";
> > > > 			ranges = < ... >;
> > > > 		}
> > > > 	}
> > > > }
> 
> I would expect a description of resources wired to the connector
> available at the foo-connector node.

Possibly yes.  Here I was assuming a case where the resources
presented by a "foo-connector" are fixed and described in a
foo-connector binding.  But an approach which explicitly describes the
exposed resources could certainly have advantages.

> > > > 
> > > > Foo device tree:
> > > > 
> > > > / {
> > > > 	compatible = "foo-device";
> > > > 	foo-port-id = < 0x1234 >;
> > > > 	component@... {
> > > > 		reg = < ... >;
> > > > 	}
> > > > }
> > > > 
> > > > Obviously a "foo device tree" would have different conventions than a
> > > > board device tree.  It wouldn't have /cpus, /memory, /chosen - but it
> > > > could have its own "magic" nodes that make sense for the properties of
> > > > the specific connector type.
> 
> I agree with the fact that /cpus, /memory, ... wouldn't be present at this
> node. 
> 
> Can you provide an example of the "magic" node and what do you have in mind
> to store this information in DTB?

One obvious one is somewhere to store a mapping / renaming of
resources for local use on the connector.  Another might be for
metadata, like an ID code, if this connector type supports it.

> > > > Again, that would require work in the device core part of the OS.  The
> > > > bonus is that runtime addition and removal is now trivial.  No hacking
> > > > of the base device tree is needed, and so doesn't need to be reverted.
> > > > The connector driver just adds/removes the reference to its own
> > > > private tree.
> 
> Here also, I don't see exactly what you have in mind. Can you provide some
> details and example?

What I'm suggesting is that the "main" DT managed by the kernel would
never include devices from the addon board.  Instead the connector
driver would locally a store a different device tree representing just
the things under the connector.  It would present a bus to the kernel
device core that would "discover" devices on the bus based on that
local device tree.

> > > > This would, of course, need some way to refer to board resources
> > > > (interrupt controller, gpio controller) etc.  I think that can be
> > > > assembled using some of the previous ideas, though.  
> > > 
> > > I would need to wrap my head around this a bit, specially in context of
> > > chaining connectors. It does seem like it will still require the points you
> > > mentioned above to be present in one form or another, i.e. some way to
> > > extend busses to different nodes/trees and connector (even a chained one)
> > > local symbols/aliases.  
> > 
> > Yes, it would still require those mappings.  I don't think chained
> > connectors introduce a lot of extra complication.  An intermediate
> > connector would need to be able to "re-export" things it got from its
> > parent connector to its child connector(s) - renaming them if
> > necessary.
> > 
> > I think those two elements would be enough to make something that's
> > useful in at least a few cases.  However, the pretty common case of a
> > connector with pins from multiple different parent buses would need
> > bus-reg or something similar.  Or else the nasty multiplexed encoding
> > described above.
> > 
> > I say, "nasty" and in many cases it would be, but I think there may be
> > some cases where that approach does make sense: e.g. a connector that
> > has several logically separate address spaces but which always travel
> > together and are typically handled by a common bridge device.  PCI is
> > a case of this, if you squint - a host bridge provides a config space
> > bus, and an MMIO bus and a PIO bus.  Also sometimes some sort of
> > interrupt controller / interrupt routing, complicated by the fact that
> > there are several different models for that across PCI and PCI-E.
> 
> To move forward on the topic, some concrete example would help to
> understand how to describe link, how the information is stored in DTB.
> 
> This would help us in moving in the right direction.
> 
> 
> If the issue with export-symbols is having it described as a DTS node, this
> could be changed with a new DTS keyword.
> 
> /export/ <symbol_name_to_resolve> <resolved_symbol_name>
>   - <symbol_name_tp_resolved>: Symbol name seen by a user
>   - <resolved_symbol_name>: Symbol used once resolved.

I might have some quibbles with the details, but I'd be open to
something like that.

> <resolved_symbol_name> can be restricted to phandle values.

Because we're operating at the dts level here, whether it's a phandle
is not actually relevant.  We can treat this as a dts label name (or
optionally path) - how that's encoded in the dtb we get to choose. A
phandle is the most obvious approach.

> Mutiple /export/ can be available inside a node in order to give a list
> of exported symbols.

Sure.

> For instance with the connector example I previously mentioned.
> We can replace the export-symbols node by the following:
> 
> ---- base board DT ----
>   conn0: connector0 {
> 	gpio-map = <0 &gpio0 12>, /* gpio A wired to gpio 12 of gpio0 controller */
>                    <1 &gpio2 10;  /* gpio B wired to gpio 10 of gpio2 controller */
> 
>         i2c_one: i2c-one {
> 		compatible = "i2c-bus-extension";
> 		i2c-parent = <i2c5>; /* i2c wired to i2c5 controller */

I think bus-reg or bus-ranges is a more natural way to do this than
per-bus bindings, but that's not particularly relevant to the rest of
the proposal.

> 	};
> 
> 	i2c_two: i2c-two {
> 		compatible = "i2c-bus-extension";
> 		i2c-parent = <i2c6>; /* i2c wired to i2c6 controller */
> 	};
> 
> 	/export/ connector &conn0
> 	/export/ i2cA &i2c_one
> 	/export/ i2cB &i2c_two

Since dtc is explicitly aware of this, we could even allow relative
paths here, say
	/export/ connector &.;


>   };
> 
> A reference to connector (&connector) from the addon will be resolve
> to a reference to &conn0 (phandle of the connector0 node.

To handle the addon with multiple connectors we might want an
(optional) remapping / renaming on the addon side as well.  Or maybe
simpler, we could allow namespacing the references on the addon side.

> 
> ---- addon board DT, applied at connector0 node ----
>    {
> 	some-node {
> 		compatible = "foo,bar";
> 		reset-gpios = <&connector 0>; /* gpioA used as a reset gpio */
> 		ddc-i2c-bus = <&i2cB> /* Resolved thanks to /export/ */
> 		
>         }
> 
>         i2c-one {
> 		/*
>                  * A device added on the i2c-one bus wire to the connector.
>                  * /export/ not involved. i2c-one is a node (bus extension) available
>                  * in the DT node where this addon board DT is applied.
>                  * 
>                  */
> 		eeprom@10 {
> 			compatible = "baz,eeprom"
> 			reg = 10; 
> 		};
> 	};
>    };
> 
> 
> Now, what is expected in term of DTB format?
> 
> I think we need:

>  - Base DT: List of exported symbols per nodes (/export/)

Yes.  I'm not necessarily opposed to encoding this as a regular
property (basically, 'exported-symbols').  It's more the fixup
information that is horrible as regular properties.  Connectors should
also probably have a property marking them *as* a connector and giving
the type.

If we didn't want a regular property we could encode this as an
FDT_EXPORT tag.  These would go after FDT_BEGIN_NODE, before FDT_PROP
and each include a (local-name, phandle) tuple.  local-name could be a
string table reference.

>  - Addon DT: List of unresolved symbols and their location

Not sure we need this as a singular list, see below.

>  - Addon DT: A way to avoid collision in phandle values

I'd suggest maybe three new tags for this, these would appear
immediately after the FDT_PROP tag to which they're relevant.

	FDT_REF_PHANDLE		(offset, name)

This would replace the 4 bytes at `offset` in the previous property
with the phandle of the node referred to by `name` (which must match
something in the connector's exported list).

	FDT_REF_PATH		(offset, name)

Similar but would insert the full path name of the referenced node in
the property.

	FDT_REF_LOCAL		(offset)

This indicates that that offset in the property contains a phandle
value pointing to something within the addon - when we adjust addon
phandles to avoid collisions, it needs to be updated too.

The addon should also include information on the type of connector(s)
it wants to plug into.  That should be a stringlist, like
'compatible', to allow for backwards compatible future extensions.


Or... allowing for addons plugging into multiple connectors, possibly
each of those `name`s needs to be extended to (connector, name).
`connector` would be either a name or index saying which connector
we're referring to (numbering/naming controlled by the addon) with
name being the name of the symbol within that connector (naming
controlled by the base board / connector type).

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]