resend-- RFC: representation of CPUs/threads in device trees

resend-- RFC: representation of CPUs/threads in device trees

[Yoder Stuart-B08248]

There is no standard binding for how hardware threads
(which in most cases share an MMU) for the Power 
architecture are represented in device trees.

The IBM SPAPR uses the ibm,ppc-interrupt-server#s.
property, which is an array with 1 element per thread.  The
number is used in addressing by the interrupt controller.

The question of how/if a CPU or thread's PIR value is
represented is also relevant.  Some CPUs have an unmodifiable
PIR number space that is distinct from the number space
addressed from the interrupt controller.

The proposal below outlines a general approach.  If there
is consensus, this would eventually get written up in a
more formal form and added to the ePAPR.

Regards, Stuart Yoder

=============================================================

-CPUs and threads are numbered through a unified
 number-space  that should match as closely as possible
 the interrupt controller's numbering of CPUs/threads

-What goes under a CPU node?

  -generally, a CPU node represents a hardware
   execution block that is sufficiently independent that
   it is capable of running an operating system without
   interfering with other CPUs possibly running other
   operating systems.

  -threads that share an MMU would generally be
   represented under 1 CPU node

  -If other more complex CPU topographies
   are designed the CPU binding must describe the
   topology (e.g. threads that don't share an MMU).

-If a CPU/thread can be the target of an external
 interrupt the "reg" property value must contain a unique
 CPU/thread id that is addressable by the interrupt
 controller

-If a CPU/thread cannot be the target of an external
 interrupt, then "reg" must be unique and out of bounds
 of the range addressed by the interrupt controller

-If a CPU supports more than one thread (i.e. multiple
 streams of execution) the "reg" property is an array with
 1  element per thread

-The #address-cells property on the "cpus" container
 specifies  how many cells each element of the "reg"
 property array takes

-Software can determine the number of threads by dividing
 the size of "reg" by "#address-cells"

-If a CPU/thread's PIR is modifiable it should be modified
 to match the "reg" property value.  If PIR cannot be
 modified and the PIR value is distinct from the interrupt
 controller numberspace, the CPUs binding may define a
 binding-specific representation of PIR values if desired.

-Regarding a standardized way to start secondary threads--

   -The current spin table mechanism is not thread safe and
    is insufficient for starting threads.

   -Given the variety of implementations in IBM and
    Freescale roadmaps, it is highly questionable whether
    a common thread release mechanism can be defined for
    all threaded CPUs.  Specific release methods for
    different processors most likely needs to be created
    and defined in processor bindings.

Re: [Power.org:parch] cpu nodes and Power ISA categories

[David Gibson]

On Tue, Aug 24, 2010 at 06:25:21PM -0700, Dan Hettena wrote:
> As requested, here is a proposal for how cpu nodes could specify what Power ISA categories are implemented. Jimi doesn't like string lists and Ben doesn't like large numbers of Boolean properties, so I'm doing whatever happens to be most convenient for me right this second (always a great way to make difficult decisions).
> 
> In section 3.7.1 (General Properties of CPU nodes), add the following properties to the table in ePAPR 1.1:
> 
> (1)
> 
> Property Name: power-isa-impl-version
> Usage: O
> Value Type: <string>
> Definition: Specifies the architecture version number (e.g., "2.06") with which this cpu is compliant.
> 
> Note: The cpu must be fully compliant with all requirements of the
> indicated architecture version, except for any errata indicated by
> the power-isa-impl-errata property. A later architecture version may
> not be reported merely on account of the cpu having implemented
> additional features that were later incorporated into the
> architecture.

Rather than a separate property, I'd suggest here we define a
convention for strings which can go into the compatible property of
the cpu nodes.  So for example:
	cpu@0 {
		compatible = "ibm,POWER6", "powerpc,ISA-2.06";
		....
	};

(I'm not sure if that example is actually correct, but you get the idea).

> (2)
> 
> Property Name: power-isa-impl-categories
> Usage: O
> Value Type: <stringlist>
> Definition: Specifies all the implemented architectural categories,
> in their abbreviated form, as specified in Book I of the selected
> ISA version (e.g., "B","E","ECL","E.HV").

This sounds ok, but I'd suggest some minor changes to the naming.  It
seems to me to make sense to treat "powerpc" as a quasi-vendor, and I
think we want to avoid abbreviations in property name.  So perhaps:
	powerpc,implementation-categories = "E.HV";

I'm not really familiar with the Book I defined categories, so I can't
comment on the content encoding at this point.

> (3) For my amusement...
> 
> Property Name: power-isa-impl-errata
> Usage: O
> Value Type: <stringlist>
> Definition: Specifies a list of exceptions to the cpu's compliance
> with the architecture. Each exception consists of a pair of strings
> where the first string in the pair is the most general architectural
> category impacted by the exception and the second string is a name
> for the erratum. For example, a cpu node for an e500v1 core might
> include "ECL","fsl-icbtls-isync-itlb-miss" to indicate that
> undefined behavior can result if the execution of an icbtls
> instruction is not separated from a subsequent ITLB miss by an isync
> instruction. An OS may choose to avoid making use of a category if
> it does not have workarounds for all listed errata.

Interesting idea, not sure if it's practical or not.  I think we'd
need to pin down a bit more where the errata defintions would reside /
who'd be responsible for the official list of defined errata and so
forth.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

RE: [Power.org:parch] cpu nodes and Power ISA categories

[Dan Hettena]

> > In section 3.7.1 (General Properties of CPU nodes), add the following
> properties to the table in ePAPR 1.1:
> >
> > (1)
> >
> > Property Name: power-isa-impl-version
> > Usage: O
> > Value Type: <string>
> > Definition: Specifies the architecture version number (e.g., "2.06")
> with which this cpu is compliant.
> ...
> Rather than a separate property, I'd suggest here we define a
> convention for strings which can go into the compatible property of
> the cpu nodes.  So for example:
> 	cpu@0 {
> 		compatible = "ibm,POWER6", "powerpc,ISA-2.06";
> 		....
> 	};
> 
> (I'm not sure if that example is actually correct, but you get the idea).

That would be fine if powerpc were changed to power.

> > (2)
> >
> > Property Name: power-isa-impl-categories
> > Usage: O
> > Value Type: <stringlist>
> > Definition: Specifies all the implemented architectural categories,
> > in their abbreviated form, as specified in Book I of the selected
> > ISA version (e.g., "B","E","ECL","E.HV").
> 
> This sounds ok, but I'd suggest some minor changes to the naming.  It
> seems to me to make sense to treat "powerpc" as a quasi-vendor, and I
> think we want to avoid abbreviations in property name.  So perhaps:
> 	powerpc,implementation-categories = "E.HV";
> 
> I'm not really familiar with the Book I defined categories, so I can't
> comment on the content encoding at this point.

Again, change powerpc to power and I'm happy.

> > (3) For my amusement...
> >
> > Property Name: power-isa-impl-errata
> > Usage: O
> > Value Type: <stringlist>
> > Definition: Specifies a list of exceptions to the cpu's compliance
> > with the architecture. Each exception consists of a pair of strings
> > where the first string in the pair is the most general architectural
> > category impacted by the exception and the second string is a name
> > for the erratum. For example, a cpu node for an e500v1 core might
> > include "ECL","fsl-icbtls-isync-itlb-miss" to indicate that
> > undefined behavior can result if the execution of an icbtls
> > instruction is not separated from a subsequent ITLB miss by an isync
> > instruction. An OS may choose to avoid making use of a category if
> > it does not have workarounds for all listed errata.
> 
> Interesting idea, not sure if it's practical or not.  I think we'd
> need to pin down a bit more where the errata defintions would reside /
> who'd be responsible for the official list of defined errata and so
> forth.

I doubt it's practical. The biggest problem I see is that device trees ship (or at least always ought to ship) with hardware, and hardware tends to ship before all errata are known.

There is a general problem here though: Devices often fail to comply fully with the specifications implied by their "compatible" properties, so a generic driver for a device might not work unless there's some way to tell it what functionality of the device to avoid.

As to your question, the provider of the device (and thus the device tree node) would have to be responsible for specifying the errata, and I imagine there would be various pressures not to do that. Not everyone even makes their errata public.

I suspect the only real solution to this problem is the one we're used to: If the generic driver can't work with the device because the device is broken, then use a driver more specific to the device that knows how to get it to behave.

Dan

Re: [Power.org:parch] cpu nodes and Power ISA categories

[David Gibson]

On Sun, Sep 19, 2010 at 11:35:40PM -0700, Dan Hettena wrote:
> > > In section 3.7.1 (General Properties of CPU nodes), add the following
> > properties to the table in ePAPR 1.1:
> > >
> > > (1)
> > >
> > > Property Name: power-isa-impl-version
> > > Usage: O
> > > Value Type: <string>
> > > Definition: Specifies the architecture version number (e.g., "2.06")
> > with which this cpu is compliant.
> > ...
> > Rather than a separate property, I'd suggest here we define a
> > convention for strings which can go into the compatible property of
> > the cpu nodes.  So for example:
> > 	cpu@0 {
> > 		compatible = "ibm,POWER6", "powerpc,ISA-2.06";
> > 		....
> > 	};
> > 
> > (I'm not sure if that example is actually correct, but you get the idea).
> 
> That would be fine if powerpc were changed to power.

Um.. why?  I thought the name of the actual architecture was "PowerPC"
and that "power" was just an IBM brand name (where POWER3 and later in
the line are PowerPC architecture compliant).

> > > (2)
> > >
> > > Property Name: power-isa-impl-categories
> > > Usage: O
> > > Value Type: <stringlist>
> > > Definition: Specifies all the implemented architectural categories,
> > > in their abbreviated form, as specified in Book I of the selected
> > > ISA version (e.g., "B","E","ECL","E.HV").
> > 
> > This sounds ok, but I'd suggest some minor changes to the naming.  It
> > seems to me to make sense to treat "powerpc" as a quasi-vendor, and I
> > think we want to avoid abbreviations in property name.  So perhaps:
> > 	powerpc,implementation-categories = "E.HV";
> > 
> > I'm not really familiar with the Book I defined categories, so I can't
> > comment on the content encoding at this point.
> 
> Again, change powerpc to power and I'm happy.
> 
> > > (3) For my amusement...
> > >
> > > Property Name: power-isa-impl-errata
> > > Usage: O
> > > Value Type: <stringlist>
> > > Definition: Specifies a list of exceptions to the cpu's compliance
> > > with the architecture. Each exception consists of a pair of strings
> > > where the first string in the pair is the most general architectural
> > > category impacted by the exception and the second string is a name
> > > for the erratum. For example, a cpu node for an e500v1 core might
> > > include "ECL","fsl-icbtls-isync-itlb-miss" to indicate that
> > > undefined behavior can result if the execution of an icbtls
> > > instruction is not separated from a subsequent ITLB miss by an isync
> > > instruction. An OS may choose to avoid making use of a category if
> > > it does not have workarounds for all listed errata.
> > 
> > Interesting idea, not sure if it's practical or not.  I think we'd
> > need to pin down a bit more where the errata defintions would reside /
> > who'd be responsible for the official list of defined errata and so
> > forth.
> 
> I doubt it's practical. The biggest problem I see is that device
> trees ship (or at least always ought to ship) with hardware, and
> hardware tends to ship before all errata are known.

Yeah.  Well.. almost never would the *board* errata all be known.  The
cpu errata might, if plenty of other boards have been made with it
first.  But still.

> There is a general problem here though: Devices often fail to comply
> fully with the specifications implied by their "compatible"
> properties, so a generic driver for a device might not work unless
> there's some way to tell it what functionality of the device to
> avoid.
> 
> As to your question, the provider of the device (and thus the device
> tree node) would have to be responsible for specifying the errata,
> and I imagine there would be various pressures not to do that. Not
> everyone even makes their errata public.

Uh, sorry.  I didn't mean who decides what errata a particular device
advertises.  I mean who manages the database of what each errata
string means.

> I suspect the only real solution to this problem is the one we're
> used to: If the generic driver can't work with the device because
> the device is broken, then use a driver more specific to the device
> that knows how to get it to behave.

Indeed.  Or the theoretically similar approach of having a general
driver occasionally check for more specific compatible strings in
order to switch workarounds on or off.

It's ugly, but I think inevitable, and at least pretty flexible.  I
think we can only reasonably aim for "best effort" in providing
complete and correct information in the device tree.  Code workarounds
for the odd mis-specified thing is unavoidable, alas, we just want to
make them as infrequent as we can within the bounds of reasonable
effort.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [Power.org:parch] cpu nodes and Power ISA categories

[Jimi Xenidis]

On Sep 20, 2010, at 9:57 PM, David Gibson wrote:

> On Sun, Sep 19, 2010 at 11:35:40PM -0700, Dan Hettena wrote:
>>>> In section 3.7.1 (General Properties of CPU nodes), add the following
>>> properties to the table in ePAPR 1.1:
>>>> 
>>>> (1)
>>>> 
>>>> Property Name: power-isa-impl-version
>>>> Usage: O
>>>> Value Type: <string>
>>>> Definition: Specifies the architecture version number (e.g., "2.06")
>>> with which this cpu is compliant.
>>> ...
>>> Rather than a separate property, I'd suggest here we define a
>>> convention for strings which can go into the compatible property of
>>> the cpu nodes.  So for example:
>>> 	cpu@0 {
>>> 		compatible = "ibm,POWER6", "powerpc,ISA-2.06";
>>> 		....
>>> 	};
>>> 
>>> (I'm not sure if that example is actually correct, but you get the idea).
>> 
>> That would be fine if powerpc were changed to power.
> 
> Um.. why?  I thought the name of the actual architecture was "PowerPC"
> and that "power" was just an IBM brand name (where POWER3 and later in
> the line are PowerPC architecture compliant).

That was like 5 years ago.
The formation of power.org and the activity of converging the multiple companies on one arch definition called "The Power ISA".
Its time we fixed this in the device tree.
NOTE: I'm not suggesting that Linux make this change.

-JX

> 
>>>> (2)
>>>> 
>>>> Property Name: power-isa-impl-categories
>>>> Usage: O
>>>> Value Type: <stringlist>
>>>> Definition: Specifies all the implemented architectural categories,
>>>> in their abbreviated form, as specified in Book I of the selected
>>>> ISA version (e.g., "B","E","ECL","E.HV").
>>> 
>>> This sounds ok, but I'd suggest some minor changes to the naming.  It
>>> seems to me to make sense to treat "powerpc" as a quasi-vendor, and I
>>> think we want to avoid abbreviations in property name.  So perhaps:
>>> 	powerpc,implementation-categories = "E.HV";
>>> 
>>> I'm not really familiar with the Book I defined categories, so I can't
>>> comment on the content encoding at this point.
>> 
>> Again, change powerpc to power and I'm happy.
>> 
>>>> (3) For my amusement...
>>>> 
>>>> Property Name: power-isa-impl-errata
>>>> Usage: O
>>>> Value Type: <stringlist>
>>>> Definition: Specifies a list of exceptions to the cpu's compliance
>>>> with the architecture. Each exception consists of a pair of strings
>>>> where the first string in the pair is the most general architectural
>>>> category impacted by the exception and the second string is a name
>>>> for the erratum. For example, a cpu node for an e500v1 core might
>>>> include "ECL","fsl-icbtls-isync-itlb-miss" to indicate that
>>>> undefined behavior can result if the execution of an icbtls
>>>> instruction is not separated from a subsequent ITLB miss by an isync
>>>> instruction. An OS may choose to avoid making use of a category if
>>>> it does not have workarounds for all listed errata.
>>> 
>>> Interesting idea, not sure if it's practical or not.  I think we'd
>>> need to pin down a bit more where the errata defintions would reside /
>>> who'd be responsible for the official list of defined errata and so
>>> forth.
>> 
>> I doubt it's practical. The biggest problem I see is that device
>> trees ship (or at least always ought to ship) with hardware, and
>> hardware tends to ship before all errata are known.
> 
> Yeah.  Well.. almost never would the *board* errata all be known.  The
> cpu errata might, if plenty of other boards have been made with it
> first.  But still.
> 
>> There is a general problem here though: Devices often fail to comply
>> fully with the specifications implied by their "compatible"
>> properties, so a generic driver for a device might not work unless
>> there's some way to tell it what functionality of the device to
>> avoid.
>> 
>> As to your question, the provider of the device (and thus the device
>> tree node) would have to be responsible for specifying the errata,
>> and I imagine there would be various pressures not to do that. Not
>> everyone even makes their errata public.
> 
> Uh, sorry.  I didn't mean who decides what errata a particular device
> advertises.  I mean who manages the database of what each errata
> string means.
> 
>> I suspect the only real solution to this problem is the one we're
>> used to: If the generic driver can't work with the device because
>> the device is broken, then use a driver more specific to the device
>> that knows how to get it to behave.
> 
> Indeed.  Or the theoretically similar approach of having a general
> driver occasionally check for more specific compatible strings in
> order to switch workarounds on or off.
> 
> It's ugly, but I think inevitable, and at least pretty flexible.  I
> think we can only reasonably aim for "best effort" in providing
> complete and correct information in the device tree.  Code workarounds
> for the odd mis-specified thing is unavoidable, alas, we just want to
> make them as infrequent as we can within the bounds of reasonable
> effort.
> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson
> 

RE: [Power.org:parch] cpu nodes and Power ISA categories

[Dan Hettena]

> > That would be fine if powerpc were changed to power.
> 
> Um.. why?  I thought the name of the actual architecture was "PowerPC"
> and that "power" was just an IBM brand name (where POWER3 and later in
> the line are PowerPC architecture compliant).

"Power" has been the name of the architecture ever since 2.03.

"PowerPC" was the name of the Server architecture only through 2.02 (and also of the original AIM architecture). AFAIK, all new implementations of the Power Architecture outside IBM's 4xx line avoid using the "PowerPC" name.

Dan

Re: [Power.org:parch] cpu nodes and Power ISA categories

[David Gibson]

On Tue, Sep 21, 2010 at 07:34:03AM -0700, Dan Hettena wrote:
> > > That would be fine if powerpc were changed to power.
> > 
> > Um.. why?  I thought the name of the actual architecture was "PowerPC"
> > and that "power" was just an IBM brand name (where POWER3 and later in
> > the line are PowerPC architecture compliant).
> 
> "Power" has been the name of the architecture ever since 2.03.
> 
> "PowerPC" was the name of the Server architecture only through 2.02
> (and also of the original AIM architecture). AFAIK, all new
> implementations of the Power Architecture outside IBM's 4xx line
> avoid using the "PowerPC" name.

Huh.  Shows how up to date I am.  Ok, "power" it is.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [Power.org:parch] cpu nodes and Power ISA categories

[Steve Thurber]

[-- Attachment #1.1: Type: text/plain, Size: 1701 bytes --]

Specifically, I believe it is "Power" (upper case "P", lower case "ower"). 
 Prior to sometime in history, it was POWER (all upper case).
Steve Thurber 
  Senior Technical Staff Member
    IBM Power Systems Platform Architecture
 



From:   David Gibson <dwg-8fk3Idey6ehBDgjK7y7TUQ@public.gmane.org>
To:     Dan Hettena <danh-5j0BnJlmnLM@public.gmane.org>
Cc:     devicetree-discuss <devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org>, 
"parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org" <parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org>
Date:   09/21/2010 11:18 PM
Subject:        Re: [Power.org:parch] cpu nodes and Power ISA categories
Sent by:        <parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org>



On Tue, Sep 21, 2010 at 07:34:03AM -0700, Dan Hettena wrote:
> > > That would be fine if powerpc were changed to power.
> > 
> > Um.. why?  I thought the name of the actual architecture was "PowerPC"
> > and that "power" was just an IBM brand name (where POWER3 and later in
> > the line are PowerPC architecture compliant).
> 
> "Power" has been the name of the architecture ever since 2.03.
> 
> "PowerPC" was the name of the Server architecture only through 2.02
> (and also of the original AIM architecture). AFAIK, all new
> implementations of the Power Architecture outside IBM's 4xx line
> avoid using the "PowerPC" name.

Huh.  Shows how up to date I am.  Ok, "power" it is.

-- 
David Gibson                                             | I'll have my 
music baroque, and my code
david AT gibson.dropbear.id.au           | minimalist, thank you.  NOT 
_the_ _other_
                                                                 | _way_ 
_around_!
http://www.ozlabs.org/~dgibson


[-- Attachment #1.2: Type: text/html, Size: 3366 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

Re: [Power.org:parch] cpu nodes and Power ISA categories

[David Gibson]

On Wed, Sep 22, 2010 at 08:47:11AM -0500, Steve Thurber wrote:
> Specifically, I believe it is "Power" (upper case "P", lower case "ower"). 
>  Prior to sometime in history, it was POWER (all upper case).

Right, however in the device tree context I think we should use
"power", not "Power" to match normal practice.  In particular
uppercase vendor prefixes are supposed to be stock tickers, not other
names.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson