[RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Ganapatrao Kulkarni]

From: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>

This patch adds property "nid" to memory node to provide the memory range to
numa node id mapping.

Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>

---
 Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/numa.txt

diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
new file mode 100644
index 0000000..c4a94f2
--- /dev/null
+++ b/Documentation/devicetree/bindings/numa.txt
@@ -0,0 +1,58 @@
+======================================================
+numa id binding description
+======================================================
+
+======================================================
+1 - Introduction
+======================================================
+The device node  property "nid(numa node id)" can be added to memory
+device node to map the range of memory addresses as defined in property "reg".
+The property "nid" maps the memory range to the numa node id, which is used to
+find the local and remory pages on numa aware systems.
+
+======================================================
+2 - nid property
+======================================================
+Numa node id, "nid" is required property of memory device node for 
+numa enabled platforms.
+
+|------------------------------------------------------|
+|Property Type  | Usage | Value Type | Definition      |
+|------------------------------------------------------|
+|  nid          |  R    |    <u32>   | Numa Node id    |
+|               |       |            | for this memory |
+|------------------------------------------------------|
+
+========================================================
+4 - Example memory nodes with numa node id mapping
+========================================================
+
+Example 1 (2 memory nodes, each mapped to a numa node.):
+
+	memory@00000000 {
+		device_type = "memory";
+		reg = <0x0 0x00000000 0x0 0x80000000>;
+		nid = <0x0>;
+	};
+
+	memory@10000000000 {
+		device_type = "memory";
+		reg = <0x100 0x00000000 0x0 0x80000000>;
+		nid = <0x1>;
+	};
+
+Example 2 (multiple memory ranges in each memory node and mapped to numa node):
+
+	memory@00000000 {
+		device_type = "memory";
+		reg = <0x0 0x00000000 0x0 0x80000000>,
+		      <0x0 0x80000000 0x0 0x80000000>;
+		nid = <0x0>;
+	};
+
+	memory@10000000000 {
+		device_type = "memory";
+		reg = <0x100 0x00000000 0x0 0x80000000>,
+		      <0x100 0x80000000 0x0 0x80000000>;
+		nid = <0x1>;
+	};
-- 
1.8.1.4

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Kumar Gala]

On Sep 17, 2014, at 1:56 AM, Ganapatrao Kulkarni <ganapatrao.kulkarni@caviumnetworks.com> wrote:

> From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> 
> This patch adds property "nid" to memory node to provide the memory range to
> numa node id mapping.
> 
> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> 
> —

Adding the PPC guys as they’ve been doing NUMA on IBM Power Servers for years with OF/DT.  So we should really try and follow what they’ve done.

> Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
> 1 file changed, 58 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/numa.txt
> 
> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
> new file mode 100644
> index 0000000..c4a94f2
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/numa.txt
> @@ -0,0 +1,58 @@
> +======================================================
> +numa id binding description
> +======================================================
> +
> +======================================================
> +1 - Introduction
> +======================================================
> +The device node  property "nid(numa node id)" can be added to memory
> +device node to map the range of memory addresses as defined in property "reg".
> +The property "nid" maps the memory range to the numa node id, which is used to
> +find the local and remory pages on numa aware systems.
> +
> +======================================================
> +2 - nid property
> +======================================================
> +Numa node id, "nid" is required property of memory device node for 
> +numa enabled platforms.
> +
> +|------------------------------------------------------|
> +|Property Type  | Usage | Value Type | Definition      |
> +|------------------------------------------------------|
> +|  nid          |  R    |    <u32>   | Numa Node id    |
> +|               |       |            | for this memory |
> +|------------------------------------------------------|
> +
> +========================================================
> +4 - Example memory nodes with numa node id mapping
> +========================================================
> +
> +Example 1 (2 memory nodes, each mapped to a numa node.):
> +
> +	memory@00000000 {
> +		device_type = "memory";
> +		reg = <0x0 0x00000000 0x0 0x80000000>;
> +		nid = <0x0>;
> +	};
> +
> +	memory@10000000000 {
> +		device_type = "memory";
> +		reg = <0x100 0x00000000 0x0 0x80000000>;
> +		nid = <0x1>;
> +	};
> +
> +Example 2 (multiple memory ranges in each memory node and mapped to numa node):
> +
> +	memory@00000000 {
> +		device_type = "memory";
> +		reg = <0x0 0x00000000 0x0 0x80000000>,
> +		      <0x0 0x80000000 0x0 0x80000000>;
> +		nid = <0x0>;
> +	};
> +
> +	memory@10000000000 {
> +		device_type = "memory";
> +		reg = <0x100 0x00000000 0x0 0x80000000>,
> +		      <0x100 0x80000000 0x0 0x80000000>;
> +		nid = <0x1>;
> +	};
> -- 
> 1.8.1.4
> 

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Mark Rutland]

On Wed, Sep 17, 2014 at 04:37:30PM +0100, Kumar Gala wrote:
> 
> On Sep 17, 2014, at 1:56 AM, Ganapatrao Kulkarni <ganapatrao.kulkarni@caviumnetworks.com> wrote:
> 
> > From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> > 
> > This patch adds property "nid" to memory node to provide the memory range to
> > numa node id mapping.
> > 
> > Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> > 
> > —
> 
> Adding the PPC guys as they’ve been doing NUMA on IBM Power Servers
> for years with OF/DT.  So we should really try and follow what they’ve
> done.

Agreed.

> > Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
> > 1 file changed, 58 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/numa.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
> > new file mode 100644
> > index 0000000..c4a94f2
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/numa.txt
> > @@ -0,0 +1,58 @@
> > +======================================================
> > +numa id binding description
> > +======================================================
> > +
> > +======================================================
> > +1 - Introduction
> > +======================================================
> > +The device node  property "nid(numa node id)" can be added to memory

Why the quotes?

> > +device node to map the range of memory addresses as defined in property "reg".
> > +The property "nid" maps the memory range to the numa node id, which is used to
> > +find the local and remory pages on numa aware systems.

What is a "numa node id", exactly, and how is the OS intended to use it?

I don't see how this can be of any use as-is.

Mark.

> > +
> > +======================================================
> > +2 - nid property
> > +======================================================
> > +Numa node id, "nid" is required property of memory device node for 
> > +numa enabled platforms.
> > +
> > +|------------------------------------------------------|
> > +|Property Type  | Usage | Value Type | Definition      |
> > +|------------------------------------------------------|
> > +|  nid          |  R    |    <u32>   | Numa Node id    |
> > +|               |       |            | for this memory |
> > +|------------------------------------------------------|
> > +
> > +========================================================
> > +4 - Example memory nodes with numa node id mapping
> > +========================================================
> > +
> > +Example 1 (2 memory nodes, each mapped to a numa node.):
> > +
> > +	memory@00000000 {
> > +		device_type = "memory";
> > +		reg = <0x0 0x00000000 0x0 0x80000000>;
> > +		nid = <0x0>;
> > +	};
> > +
> > +	memory@10000000000 {
> > +		device_type = "memory";
> > +		reg = <0x100 0x00000000 0x0 0x80000000>;
> > +		nid = <0x1>;
> > +	};
> > +
> > +Example 2 (multiple memory ranges in each memory node and mapped to numa node):
> > +
> > +	memory@00000000 {
> > +		device_type = "memory";
> > +		reg = <0x0 0x00000000 0x0 0x80000000>,
> > +		      <0x0 0x80000000 0x0 0x80000000>;
> > +		nid = <0x0>;
> > +	};
> > +
> > +	memory@10000000000 {
> > +		device_type = "memory";
> > +		reg = <0x100 0x00000000 0x0 0x80000000>,
> > +		      <0x100 0x80000000 0x0 0x80000000>;
> > +		nid = <0x1>;
> > +	};
> > -- 
> > 1.8.1.4
> > 
> 
> -- 
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
> 
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Hanjun Guo]

On 2014-9-18 3:34, Mark Rutland wrote:
> On Wed, Sep 17, 2014 at 04:37:30PM +0100, Kumar Gala wrote:
>>
>> On Sep 17, 2014, at 1:56 AM, Ganapatrao Kulkarni <ganapatrao.kulkarni@caviumnetworks.com> wrote:
>>
>>> From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>>
>>> This patch adds property "nid" to memory node to provide the memory range to
>>> numa node id mapping.
>>>
>>> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>>
>>> —
>>
>> Adding the PPC guys as they’ve been doing NUMA on IBM Power Servers
>> for years with OF/DT.  So we should really try and follow what they’ve
>> done.
> 
> Agreed.
> 
>>> Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>>> 1 file changed, 58 insertions(+)
>>> create mode 100644 Documentation/devicetree/bindings/numa.txt
>>>
>>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
>>> new file mode 100644
>>> index 0000000..c4a94f2
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/numa.txt
>>> @@ -0,0 +1,58 @@
>>> +======================================================
>>> +numa id binding description
>>> +======================================================
>>> +
>>> +======================================================
>>> +1 - Introduction
>>> +======================================================
>>> +The device node  property "nid(numa node id)" can be added to memory
> 
> Why the quotes?
> 
>>> +device node to map the range of memory addresses as defined in property "reg".
>>> +The property "nid" maps the memory range to the numa node id, which is used to
>>> +find the local and remory pages on numa aware systems.
> 
> What is a "numa node id", exactly, and how is the OS intended to use it?

I think "Proximity Domain" would be more suitably, processors and memory or IOs
in the same domain will have better performance than crossing other domains.

Thanks
Hanjun
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Nathan Lynch]

On 09/17/2014 03:56 AM, Ganapatrao Kulkarni wrote:
> From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> 
> This patch adds property "nid" to memory node to provide the memory range to
> numa node id mapping.
> 
> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> 
> ---
>  Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>  1 file changed, 58 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/numa.txt
> 
> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
> new file mode 100644
> index 0000000..c4a94f2
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/numa.txt
> @@ -0,0 +1,58 @@
> +======================================================
> +numa id binding description
> +======================================================
> +
> +======================================================
> +1 - Introduction
> +======================================================
> +The device node  property "nid(numa node id)" can be added to memory
> +device node to map the range of memory addresses as defined in property "reg".
> +The property "nid" maps the memory range to the numa node id, which is used to
> +find the local and remory pages on numa aware systems.

"Local" and "remote" memory are notions that relate to some other
resource -- typically a CPU, but also I/O resources on some systems.  It
seems to me that a useful NUMA binding would at least specify a "nid"
property, or something like it, for both cpu and memory nodes.  But this
document speaks only of memory nodes.

As Kumar said, the device tree on powerpc server systems already has
properties that express NUMA information.  If you can get hold of a copy
of the PAPR (not ePAPR) from power.org, refer to the description of
"ibm,associativity" and related properties.  I recall that it's a bit
more complex than this proposal, though.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Zi Shen Lim]

On Wed, Sep 17, 2014 at 2:48 PM, Nathan Lynch <Nathan_Lynch-nmGgyN9QBj3QT0dZR+AlfA@public.gmane.org> wrote:
> On 09/17/2014 03:56 AM, Ganapatrao Kulkarni wrote:
>> From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>
>> This patch adds property "nid" to memory node to provide the memory range to
>> numa node id mapping.
>>
>> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>
>> ---
>>  Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>>  1 file changed, 58 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/numa.txt
>>
>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
>> new file mode 100644
>> index 0000000..c4a94f2
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/numa.txt
>> @@ -0,0 +1,58 @@
>> +======================================================
>> +numa id binding description
>> +======================================================
>> +
>> +======================================================
>> +1 - Introduction
>> +======================================================
>> +The device node  property "nid(numa node id)" can be added to memory
>> +device node to map the range of memory addresses as defined in property "reg".
>> +The property "nid" maps the memory range to the numa node id, which is used to
>> +find the local and remory pages on numa aware systems.
>
> "Local" and "remote" memory are notions that relate to some other
> resource -- typically a CPU, but also I/O resources on some systems.  It
> seems to me that a useful NUMA binding would at least specify a "nid"
> property, or something like it, for both cpu and memory nodes.  But this
> document speaks only of memory nodes.
>
> As Kumar said, the device tree on powerpc server systems already has
> properties that express NUMA information.  If you can get hold of a copy
> of the PAPR (not ePAPR) from power.org, refer to the description of
> "ibm,associativity" and related properties.  I recall that it's a bit
> more complex than this proposal, though.

I'm not able to find a link to the actual PAPR (not ePAPR)
specification. Anyone has a linky?

I did find some code in tree, but couldn't find the bindings in
Documentation/devicetree/.

Seems like we'd care about form1?
-----8<-----
commit 41eab6f88f24124df89e38067b3766b7bef06ddb
Author: Anton Blanchard <anton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
Date:   Sun May 16 20:22:31 2010 +0000

    powerpc/numa: Use form 1 affinity to setup node distance

    Form 1 affinity allows multiple entries in
ibm,associativity-reference-points
    which represent affinity domains in decreasing order of importance. The
    Linux concept of a node is always the first entry, but using the other
    values as an input to node_distance() allows the memory allocator to make
    better decisions on which node to go first when local memory has been
    exhausted.

    We keep things simple and create an array indexed by NUMA node, capped at
    4 entries. Each time we lookup an associativity property we initialise
    the array which is overkill, but since we should only hit this path during
    boot it didn't seem worth adding a per node valid bit.

    Signed-off-by: Anton Blanchard <anton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
    Signed-off-by: Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
----->8-----


>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Ganapatrao Kulkarni]

On Thu, Sep 18, 2014 at 4:42 AM, Zi Shen Lim <zlim.lnx-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Wed, Sep 17, 2014 at 2:48 PM, Nathan Lynch <Nathan_Lynch-nmGgyN9QBj3QT0dZR+AlfA@public.gmane.org> wrote:
>> On 09/17/2014 03:56 AM, Ganapatrao Kulkarni wrote:
>>> From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>>
>>> This patch adds property "nid" to memory node to provide the memory range to
>>> numa node id mapping.
>>>
>>> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>>
>>> ---
>>>  Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>>>  1 file changed, 58 insertions(+)
>>>  create mode 100644 Documentation/devicetree/bindings/numa.txt
>>>
>>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
>>> new file mode 100644
>>> index 0000000..c4a94f2
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/numa.txt
>>> @@ -0,0 +1,58 @@
>>> +======================================================
>>> +numa id binding description
>>> +======================================================
>>> +
>>> +======================================================
>>> +1 - Introduction
>>> +======================================================
>>> +The device node  property "nid(numa node id)" can be added to memory
>>> +device node to map the range of memory addresses as defined in property "reg".
>>> +The property "nid" maps the memory range to the numa node id, which is used to
>>> +find the local and remory pages on numa aware systems.
>>
>> "Local" and "remote" memory are notions that relate to some other
>> resource -- typically a CPU, but also I/O resources on some systems.  It
>> seems to me that a useful NUMA binding would at least specify a "nid"
>> property, or something like it, for both cpu and memory nodes.  But this
>> document speaks only of memory nodes.
IMO, nid can be extended to cpu node also if the arch dont have any
other way to do mapping, in fact this can be scaled to any node.
for ARM arch, cpu mapping can be fetched from topology parsing and
more over cpu mapping is required much later when secondary cpus comes
online.
i have added for memory, since this is parsed very early in the boot
sequence and there is no support for dt based map at this moment(at
least i am not aware off).
numa distance matrix can also be defined, if the architecture has
variable node distance other than just local/remote.
>>
>> As Kumar said, the device tree on powerpc server systems already has
>> properties that express NUMA information.  If you can get hold of a copy
>> of the PAPR (not ePAPR) from power.org, refer to the description of
>> "ibm,associativity" and related properties.  I recall that it's a bit
>> more complex than this proposal, though.
>
> I'm not able to find a link to the actual PAPR (not ePAPR)
> specification. Anyone has a linky?
>
> I did find some code in tree, but couldn't find the bindings in
> Documentation/devicetree/.
yes, there is nothing in Documentation, if PPC are using since several
years, then i humbly request to publish the documentation for numa
bindings.
No point in reinventing the wheel. Also i did not find any spec named
PAPR, is it open spec, available to all like ePAPR?

>
> Seems like we'd care about form1?
> -----8<-----
> commit 41eab6f88f24124df89e38067b3766b7bef06ddb
> Author: Anton Blanchard <anton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
> Date:   Sun May 16 20:22:31 2010 +0000
>
>     powerpc/numa: Use form 1 affinity to setup node distance
>
>     Form 1 affinity allows multiple entries in
> ibm,associativity-reference-points
>     which represent affinity domains in decreasing order of importance. The
>     Linux concept of a node is always the first entry, but using the other
>     values as an input to node_distance() allows the memory allocator to make
>     better decisions on which node to go first when local memory has been
>     exhausted.
>
>     We keep things simple and create an array indexed by NUMA node, capped at
>     4 entries. Each time we lookup an associativity property we initialise
>     the array which is overkill, but since we should only hit this path during
>     boot it didn't seem worth adding a per node valid bit.
>
>     Signed-off-by: Anton Blanchard <anton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
>     Signed-off-by: Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
> ----->8-----
>
>
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Ganapatrao Kulkarni]

looks like previous email not had the email ids added by kumar gala.
adding the missing ids and sending again.

On Thu, Sep 18, 2014 at 9:32 AM, Ganapatrao Kulkarni
<gpkulkarni-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Thu, Sep 18, 2014 at 4:42 AM, Zi Shen Lim <zlim.lnx-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> On Wed, Sep 17, 2014 at 2:48 PM, Nathan Lynch <Nathan_Lynch-nmGgyN9QBj3QT0dZR+AlfA@public.gmane.org> wrote:
>>> On 09/17/2014 03:56 AM, Ganapatrao Kulkarni wrote:
>>>> From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>>>
>>>> This patch adds property "nid" to memory node to provide the memory range to
>>>> numa node id mapping.
>>>>
>>>> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>>>
>>>> ---
>>>>  Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>>>>  1 file changed, 58 insertions(+)
>>>>  create mode 100644 Documentation/devicetree/bindings/numa.txt
>>>>
>>>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
>>>> new file mode 100644
>>>> index 0000000..c4a94f2
>>>> --- /dev/null
>>>> +++ b/Documentation/devicetree/bindings/numa.txt
>>>> @@ -0,0 +1,58 @@
>>>> +======================================================
>>>> +numa id binding description
>>>> +======================================================
>>>> +
>>>> +======================================================
>>>> +1 - Introduction
>>>> +======================================================
>>>> +The device node  property "nid(numa node id)" can be added to memory
>>>> +device node to map the range of memory addresses as defined in property "reg".
>>>> +The property "nid" maps the memory range to the numa node id, which is used to
>>>> +find the local and remory pages on numa aware systems.
>>>
>>> "Local" and "remote" memory are notions that relate to some other
>>> resource -- typically a CPU, but also I/O resources on some systems.  It
>>> seems to me that a useful NUMA binding would at least specify a "nid"
>>> property, or something like it, for both cpu and memory nodes.  But this
>>> document speaks only of memory nodes.
> IMO, nid can be extended to cpu node also if the arch dont have any
> other way to do mapping, in fact this can be scaled to any node.
> for ARM arch, cpu mapping can be fetched from topology parsing and
> more over cpu mapping is required much later when secondary cpus comes
> online.
> i have added for memory, since this is parsed very early in the boot
> sequence and there is no support for dt based map at this moment(at
> least i am not aware off).
> numa distance matrix can also be defined, if the architecture has
> variable node distance other than just local/remote.
>>>
>>> As Kumar said, the device tree on powerpc server systems already has
>>> properties that express NUMA information.  If you can get hold of a copy
>>> of the PAPR (not ePAPR) from power.org, refer to the description of
>>> "ibm,associativity" and related properties.  I recall that it's a bit
>>> more complex than this proposal, though.
>>
>> I'm not able to find a link to the actual PAPR (not ePAPR)
>> specification. Anyone has a linky?
>>
>> I did find some code in tree, but couldn't find the bindings in
>> Documentation/devicetree/.
> yes, there is nothing in Documentation, if PPC are using since several
> years, then i humbly request to publish the documentation for numa
> bindings.
> No point in reinventing the wheel. Also i did not find any spec named
> PAPR, is it open spec, available to all like ePAPR?
>
>>
>> Seems like we'd care about form1?
>> -----8<-----
>> commit 41eab6f88f24124df89e38067b3766b7bef06ddb
>> Author: Anton Blanchard <anton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
>> Date:   Sun May 16 20:22:31 2010 +0000
>>
>>     powerpc/numa: Use form 1 affinity to setup node distance
>>
>>     Form 1 affinity allows multiple entries in
>> ibm,associativity-reference-points
>>     which represent affinity domains in decreasing order of importance. The
>>     Linux concept of a node is always the first entry, but using the other
>>     values as an input to node_distance() allows the memory allocator to make
>>     better decisions on which node to go first when local memory has been
>>     exhausted.
>>
>>     We keep things simple and create an array indexed by NUMA node, capped at
>>     4 entries. Each time we lookup an associativity property we initialise
>>     the array which is overkill, but since we should only hit this path during
>>     boot it didn't seem worth adding a per node valid bit.
>>
>>     Signed-off-by: Anton Blanchard <anton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
>>     Signed-off-by: Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
>> ----->8-----
>>
>>
>>>
>>> _______________________________________________
>>> linux-arm-kernel mailing list
>>> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] dt:numa: adding numa node mapping for memory nodes.

[Hanjun Guo]

On 2014-9-18 5:48, Nathan Lynch wrote:
> On 09/17/2014 03:56 AM, Ganapatrao Kulkarni wrote:
>> From: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>
>> This patch adds property "nid" to memory node to provide the memory range to
>> numa node id mapping.
>>
>> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>>
>> ---
>>  Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++
>>  1 file changed, 58 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/numa.txt
>>
>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt
>> new file mode 100644
>> index 0000000..c4a94f2
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/numa.txt
>> @@ -0,0 +1,58 @@
>> +======================================================
>> +numa id binding description
>> +======================================================
>> +
>> +======================================================
>> +1 - Introduction
>> +======================================================
>> +The device node  property "nid(numa node id)" can be added to memory
>> +device node to map the range of memory addresses as defined in property "reg".
>> +The property "nid" maps the memory range to the numa node id, which is used to
>> +find the local and remory pages on numa aware systems.
> 
> "Local" and "remote" memory are notions that relate to some other
> resource -- typically a CPU, but also I/O resources on some systems.  It
> seems to me that a useful NUMA binding would at least specify a "nid"
> property, or something like it, for both cpu and memory nodes.  But this
> document speaks only of memory nodes.

Agreed. and more, I think I/O resources also need such property, it will
have performance influence for the proximity domain of I/O devices too.

Thanks
Hanjun

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html