devicetree: Musings on reserved regions

devicetree: Musings on reserved regions

[Grant Likely]

As part of the process of bringing dt support up on non-powerpc
platforms, I've been thinking about the usage mode of the reserved
regions section in the flattened device tree structure.  First, for
reference, here is the description of reserved regions from ePAPR
1.0:

> 7.3 Memory Reservation Block
> 7.3.1 Purpose
> The memory reservation block provides the client program with
> a list of areas in physical memory which are reserved; that
> is, which shall not be used for general memory allocations. It
> is used to protect vital data structures from being
> overwritten by the client program. For example, on some
> systems with an IOMMU, the TCE (translation control entry)
> tables initialized by an ePAPR boot program would need to be
> protected in this manner. Likewise, any boot program code or
> data used during the client program’s runtime would need to be
> reserved (e.g., RTAS on Open Firmware platforms). The ePAPR
> does not require the boot program to provide any such runtime
> components, but it does not prohibit implementations from
> doing so as an extension.
>
> More specifically, a client program shall not access memory in
> a reserved region unless other information provided by the
> boot program explicitly indicates that it shall do so. The
> client program may then access the indicated section of the
> reserved memory in the indicated manner. Methods by which the
> boot program can indicate to the client program specific uses
> for reserved memory may appear in this document, in optional
> extensions to it, or in platform-specific documentation.
>
> The reserved regions supplied by a boot program may, but are
> not required to, encompass the device tree blob itself. The
> client program shall ensure that it does not overwrite this
> data structure before it is used, whether or not it is in the
> reserved areas.

The problem that I'm having, is that the lifecycle for reserved
regions is not defined and the client program has no mechanism to
determine what each reserved region is intended for.  This issue has
been raised before in the context of kexec from one Linux instance to
another[1].  When going from one kernel to the next, kexec needs to
populate a valid set of reserved regions, but it isn't easy to figure
out what the new set should be because there isn't a reliable way to
figure out which regions from the previous boot need to be preserved
(ie. runtime firmware or framebuffers) vs regions that are obsolete
(original initrd and dtb regions).

[1] http://www.mail-archive.com/linuxppc-dev-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org/msg44999.html

Before committing to implementing reserved regions on non-powerpc
architectures, I would like to nail down a stricter definition of how
client programs are to interpret reserved regions.  I'm not sure if it
is reasonable to make this part of ePAPR 1.1, but here are my thoughts
at the moment:

In addition to the reserved regions block in the header, define a set
of properties in the memory node that specify the reserved regions
with the name reflecting the usage.
For example:

	memory@0 {
		device_type = "memory";
		reg = <0 0x40000000>;
		reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
		reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
		reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
	};

Each reserved property would start with "reserved-" followed by a
name.  reserved-ramdisk and reserved-dtb would be reserved for ramdisk
and dtb images respectively.  Other names could also be defined; for
example, reserved-openfirmware.

Firmware would be responsible to ensure the reserved regions are
accurately reflected in the memory nodes.  Client programs may
continue to use the header values for the purpose of determining
reserved regions, but after the client program is initialized, the
data in reserved- properties should be preferred over the header
reserved regions.  On new platforms, I'm tempted to say that the
reserved-* properties should be considered authoritative over the
header values.  If the client program is able to parse the memory
nodes early enough, then it shouldn't be necessary to look at the
header fields at all (for instance, this is currently the case on
ARM).

After the client has initialized itself (for whatever definition of
initialized it chooses) the client must always respect the reserved
regions specified in the memory nodes *unless* it recognizes the
region and knows that it is safe to drop it.  For instance, it is safe
to reclaim the dtb and ramdisk regions.  It *might* be safe to reclaim
a framebuffer region too, but that depends on the framebuffer driver.

For the kexec use case, this would mean that kexec would drop the old
dtb and kexec regions while adding new ones before jumping into the
new kernel.  It would also mean that the kernel can reclaim regions
that it know are no longer in use; for instance, if the client no
longer needs OpenFirmware to be resident, or if a new framebuffer has
been allocated and the old one is no longer in use.

Thoughts?
g.




-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Benjamin Herrenschmidt]

> In addition to the reserved regions block in the header, define a set
> of properties in the memory node that specify the reserved regions
> with the name reflecting the usage.
> For example:
> 
> 	memory@0 {
> 		device_type = "memory";
> 		reg = <0 0x40000000>;
> 		reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
> 		reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
> 		reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
> 	};
> 
> Each reserved property would start with "reserved-" followed by a
> name.  reserved-ramdisk and reserved-dtb would be reserved for ramdisk
> and dtb images respectively.  Other names could also be defined; for
> example, reserved-openfirmware.

I very much dislike the list of properties with magic names. I'd rather
have a pair of properties containing lists (reserved-names and
reserved-ranges).

Cheers,
Ben.

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Grant Likely]

On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
<benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
>
>> In addition to the reserved regions block in the header, define a set
>> of properties in the memory node that specify the reserved regions
>> with the name reflecting the usage.
>> For example:
>>
>>       memory@0 {
>>               device_type = "memory";
>>               reg = <0 0x40000000>;
>>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
>>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
>>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
>>       };
>>
>> Each reserved property would start with "reserved-" followed by a
>> name.  reserved-ramdisk and reserved-dtb would be reserved for ramdisk
>> and dtb images respectively.  Other names could also be defined; for
>> example, reserved-openfirmware.
>
> I very much dislike the list of properties with magic names. I'd rather
> have a pair of properties containing lists (reserved-names and
> reserved-ranges).

Heh, I have very much the opposite opinion.  I'd rather have property
names that match the usage than have the names and values split into
two properties.  To me, keeping them together is more tasteful.  Queue
the debate...  :-)

g.

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Scott Wood]

On Tue, 8 Feb 2011 08:39:26 +1100
Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:

> 
> > In addition to the reserved regions block in the header, define a set
> > of properties in the memory node that specify the reserved regions
> > with the name reflecting the usage.
> > For example:
> > 
> > 	memory@0 {
> > 		device_type = "memory";
> > 		reg = <0 0x40000000>;
> > 		reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
> > 		reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
> > 		reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
> > 	};
> > 
> > Each reserved property would start with "reserved-" followed by a
> > name.  reserved-ramdisk and reserved-dtb would be reserved for ramdisk
> > and dtb images respectively.  Other names could also be defined; for
> > example, reserved-openfirmware.
> 
> I very much dislike the list of properties with magic names. I'd rather
> have a pair of properties containing lists (reserved-names and
> reserved-ranges).

How about a node:

memory@0 {
	device_type = "memory";
	reg = <0 0x40000000>;

	reserved {
		ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
		dtb = <0xbf0000 0x1000>; /* devicetree */
		fb0 = <0x1000000 0x400000>; /* framebuffer */
		something-fragmented = <0x800000 0x100000 0xa00000 0x100000>;
	};
};

Or is the objection to any use of the property name to indicate reservation type,
rather than the magic "reserved-" prefix?

-Scott

RE: [Power.org:parch] devicetree: Musings on reserved regions

[Yoder Stuart-B08248]

> -----Original Message-----
> From: glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org [mailto:glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org] On Behalf Of Grant
> Likely
> Sent: Monday, February 07, 2011 3:45 PM
> To: Benjamin Herrenschmidt
> Cc: parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org; devicetree-discuss; David Gibson; Yoder Stuart-B08248;
> McClintock Matthew-B29882; Wood Scott-B07421
> Subject: Re: [Power.org:parch] devicetree: Musings on reserved regions
> 
> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
> >
> >> In addition to the reserved regions block in the header, define a set
> >> of properties in the memory node that specify the reserved regions
> >> with the name reflecting the usage.
> >> For example:
> >>
> >>       memory@0 {
> >>               device_type = "memory";
> >>               reg = <0 0x40000000>;
> >>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk
> >> */
> >>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
> >>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
> >>       };
> >>
> >> Each reserved property would start with "reserved-" followed by a
> >> name.  reserved-ramdisk and reserved-dtb would be reserved for
> >> ramdisk and dtb images respectively.  Other names could also be
> >> defined; for example, reserved-openfirmware.
> >
> > I very much dislike the list of properties with magic names. I'd
> > rather have a pair of properties containing lists (reserved-names and
> > reserved-ranges).
> 
> Heh, I have very much the opposite opinion.  I'd rather have property names
> that match the usage than have the names and values split into two
> properties.  To me, keeping them together is more tasteful.  Queue the
> debate...  :-)

Could we not do both? Use an enum to identify the region type:

    reserved = <0x1 0xc00000 0x200000>; /* 2MB ramdisk
    reserved = <0x2 0xbf0000 0x1000>; /* devicetree */
    reserved = <0x3 0x1000000 0x400000>; /* framebuffer */

Stuart

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Grant Likely]

On Mon, Feb 7, 2011 at 2:53 PM, Yoder Stuart-B08248
<B08248-KZfg59tc24xl57MIdRCFDg@public.gmane.org> wrote:
>
>
>> -----Original Message-----
>> From: glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org [mailto:glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org] On Behalf Of Grant
>> Likely
>> Sent: Monday, February 07, 2011 3:45 PM
>> To: Benjamin Herrenschmidt
>> Cc: parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org; devicetree-discuss; David Gibson; Yoder Stuart-B08248;
>> McClintock Matthew-B29882; Wood Scott-B07421
>> Subject: Re: [Power.org:parch] devicetree: Musings on reserved regions
>>
>> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
>> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
>> >
>> >> In addition to the reserved regions block in the header, define a set
>> >> of properties in the memory node that specify the reserved regions
>> >> with the name reflecting the usage.
>> >> For example:
>> >>
>> >>       memory@0 {
>> >>               device_type = "memory";
>> >>               reg = <0 0x40000000>;
>> >>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk
>> >> */
>> >>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
>> >>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
>> >>       };
>> >>
>> >> Each reserved property would start with "reserved-" followed by a
>> >> name.  reserved-ramdisk and reserved-dtb would be reserved for
>> >> ramdisk and dtb images respectively.  Other names could also be
>> >> defined; for example, reserved-openfirmware.
>> >
>> > I very much dislike the list of properties with magic names. I'd
>> > rather have a pair of properties containing lists (reserved-names and
>> > reserved-ranges).
>>
>> Heh, I have very much the opposite opinion.  I'd rather have property names
>> that match the usage than have the names and values split into two
>> properties.  To me, keeping them together is more tasteful.  Queue the
>> debate...  :-)
>
> Could we not do both? Use an enum to identify the region type:
>
>    reserved = <0x1 0xc00000 0x200000>; /* 2MB ramdisk
>    reserved = <0x2 0xbf0000 0x1000>; /* devicetree */
>    reserved = <0x3 0x1000000 0x400000>; /* framebuffer */

I really dislike enums for this sort of thing.  I'd rather have a
string encoded somewhere, even if it is in a different property.

g.

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Scott Wood]

On Mon, 7 Feb 2011 15:53:09 -0600
Yoder Stuart-B08248 <B08248-KZfg59tc24xl57MIdRCFDg@public.gmane.org> wrote:

> Could we not do both? Use an enum to identify the region type:
> 
>     reserved = <0x1 0xc00000 0x200000>; /* 2MB ramdisk
>     reserved = <0x2 0xbf0000 0x1000>; /* devicetree */
>     reserved = <0x3 0x1000000 0x400000>; /* framebuffer */

Enums are bad, you're asking for conflicts (this is worse than the mmu
type enum, since it's tied to the needs of software which changes faster
than hardware).

And you can't define the same property multiple times, it'd have to be one
large array.

-Scott

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Jimi Xenidis]

On Feb 7, 2011, at 3:45 PM, Grant Likely wrote:
> 
> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
>> 
>>> In addition to the reserved regions block in the header, define a set
>>> of properties in the memory node that specify the reserved regions
>>> with the name reflecting the usage.
>>> For example:
>>> 
>>>       memory@0 {
>>>               device_type = "memory";
>>>               reg = <0 0x40000000>;
>>>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
>>>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
>>>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
>>>       };
>>> 
>>> Each reserved property would start with "reserved-" followed by a
>>> name.  reserved-ramdisk and reserved-dtb would be reserved for ramdisk
>>> and dtb images respectively.  Other names could also be defined; for
>>> example, reserved-openfirmware.
>> 
>> I very much dislike the list of properties with magic names. I'd rather
>> have a pair of properties containing lists (reserved-names and
>> reserved-ranges).
> 
> Heh, I have very much the opposite opinion.  I'd rather have property
> names that match the usage than have the names and values split into
> two properties.  To me, keeping them together is more tasteful.  Queue
> the debate...  :-)

ok.. here goes nothing...
When I see these "reservations", it makes me think that they should be nodes anyway. (off root?)
Once we have nodes for these interesting things, perhaps the next step is to just have a "reserved" property for these nodes?
so:
	ramdisk@<addr> {
		reg = < addr, size >;
		format = < "gzip", "cpio" >;  // optional, order of wrapper?
		reserved;
	}

No, I'm not ready to propose the ramdisk node, but it is something to consider.

-JX


> 
> g.
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Jimi Xenidis]

On Feb 7, 2011, at 3:56 PM, Grant Likely wrote:
> 
> On Mon, Feb 7, 2011 at 2:53 PM, Yoder Stuart-B08248
> <B08248-KZfg59tc24xl57MIdRCFDg@public.gmane.org> wrote:
>> 
>> 
>>> -----Original Message-----
>>> From: glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org [mailto:glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org] On Behalf Of Grant
>>> Likely
>>> Sent: Monday, February 07, 2011 3:45 PM
>>> To: Benjamin Herrenschmidt
>>> Cc: parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org; devicetree-discuss; David Gibson; Yoder Stuart-B08248;
>>> McClintock Matthew-B29882; Wood Scott-B07421
>>> Subject: Re: [Power.org:parch] devicetree: Musings on reserved regions
>>> 
>>> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
>>> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
>>>> 
>>>>> In addition to the reserved regions block in the header, define a set
>>>>> of properties in the memory node that specify the reserved regions
>>>>> with the name reflecting the usage.
>>>>> For example:
>>>>> 
>>>>>       memory@0 {
>>>>>               device_type = "memory";
>>>>>               reg = <0 0x40000000>;
>>>>>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk
>>>>> */
>>>>>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
>>>>>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
>>>>>       };
>>>>> 
>>>>> Each reserved property would start with "reserved-" followed by a
>>>>> name.  reserved-ramdisk and reserved-dtb would be reserved for
>>>>> ramdisk and dtb images respectively.  Other names could also be
>>>>> defined; for example, reserved-openfirmware.
>>>> 
>>>> I very much dislike the list of properties with magic names. I'd
>>>> rather have a pair of properties containing lists (reserved-names and
>>>> reserved-ranges).
>>> 
>>> Heh, I have very much the opposite opinion.  I'd rather have property names
>>> that match the usage than have the names and values split into two
>>> properties.  To me, keeping them together is more tasteful.  Queue the
>>> debate...  :-)
>> 
>> Could we not do both? Use an enum to identify the region type:
>> 
>>    reserved = <0x1 0xc00000 0x200000>; /* 2MB ramdisk
>>    reserved = <0x2 0xbf0000 0x1000>; /* devicetree */
>>    reserved = <0x3 0x1000000 0x400000>; /* framebuffer */
> 
> I really dislike enums for this sort of thing.  I'd rather have a
> string encoded somewhere, even if it is in a different property.

ack
-JX

> 
> g.

RE: [Power.org:parch] devicetree: Musings on reserved regions

[Dan Hettena]

I've seen three types of reserved regions so far:

(1) Reserved regions that are permanently reserved. For example,
    RAM used for configuration of a device not communicated in the
    device tree. Nothing further needs to be said about this class.
(2) Reserved regions that are in use by something in the device tree
    (for example, a spin table, or a frame buffer for a video device).
    The client program can free these if it recognizes the node that
    is using them and reprograms it (e.g., boot all processors, or
    reprogram the video device).
(3) Reserved regions that contain data the client program might want to
    use but are not actually being used by the hardware for anything.
    For example, a RAM root file system.

I think this points strongly at having type 2 and type 3 reservations specified along with the nodes that make use of them, so that the client program can free the memory as soon as it has caused the reserved memory to no longer be in use for the purpose for which it was reserved (provided that we make it clear that reservations declared as such are not permitted to overlap).

Primarily because of type 1 reservations, I see no conflict between the conventional reservations list and the idea of explicitly identifying single-purpose reservations throughout the device tree where they are used.

However, in the spirit of distributing memory nodes, I think I would rather see the conventional reservations list moved to the corresponding memory nodes (with an added rule that a reservation may not cross a reg entry boundary, since that's just a big pain). If we're going to work towards keeping crap out of the header, then I don't see why the reservations block should be treated any differently.

Dan

> -----Original Message-----
> From: parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org [mailto:parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org] On Behalf Of Jimi Xenidis
> Sent: Monday, February 07, 2011 2:01 PM
> To: Grant Likely
> Cc: Yoder Stuart-B08248; Benjamin Herrenschmidt; parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org;
> devicetree-discuss; David Gibson; McClintock Matthew-B29882; Wood Scott-
> B07421
> Subject: Re: [Power.org:parch] devicetree: Musings on reserved regions
> 
> On Feb 7, 2011, at 3:56 PM, Grant Likely wrote:
> >
> > On Mon, Feb 7, 2011 at 2:53 PM, Yoder Stuart-B08248
> > <B08248-KZfg59tc24xl57MIdRCFDg@public.gmane.org> wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org [mailto:glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org] On Behalf Of
> Grant
> >>> Likely
> >>> Sent: Monday, February 07, 2011 3:45 PM
> >>> To: Benjamin Herrenschmidt
> >>> Cc: parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org; devicetree-discuss; David Gibson; Yoder Stuart-
> B08248;
> >>> McClintock Matthew-B29882; Wood Scott-B07421
> >>> Subject: Re: [Power.org:parch] devicetree: Musings on reserved regions
> >>>
> >>> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
> >>> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
> >>>>
> >>>>> In addition to the reserved regions block in the header, define a
> set
> >>>>> of properties in the memory node that specify the reserved regions
> >>>>> with the name reflecting the usage.
> >>>>> For example:
> >>>>>
> >>>>>       memory@0 {
> >>>>>               device_type = "memory";
> >>>>>               reg = <0 0x40000000>;
> >>>>>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk
> >>>>> */
> >>>>>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
> >>>>>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
> >>>>>       };
> >>>>>
> >>>>> Each reserved property would start with "reserved-" followed by a
> >>>>> name.  reserved-ramdisk and reserved-dtb would be reserved for
> >>>>> ramdisk and dtb images respectively.  Other names could also be
> >>>>> defined; for example, reserved-openfirmware.
> >>>>
> >>>> I very much dislike the list of properties with magic names. I'd
> >>>> rather have a pair of properties containing lists (reserved-names and
> >>>> reserved-ranges).
> >>>
> >>> Heh, I have very much the opposite opinion.  I'd rather have property
> names
> >>> that match the usage than have the names and values split into two
> >>> properties.  To me, keeping them together is more tasteful.  Queue the
> >>> debate...  :-)
> >>
> >> Could we not do both? Use an enum to identify the region type:
> >>
> >>    reserved = <0x1 0xc00000 0x200000>; /* 2MB ramdisk
> >>    reserved = <0x2 0xbf0000 0x1000>; /* devicetree */
> >>    reserved = <0x3 0x1000000 0x400000>; /* framebuffer */
> >
> > I really dislike enums for this sort of thing.  I'd rather have a
> > string encoded somewhere, even if it is in a different property.
> 
> ack
> -JX
> 
> >
> > g.
> 

Re: [Power.org:parch] devicetree: Musings on reserved regions

[David VomLehn]

On Mon, Feb 07, 2011 at 04:01:28PM -0600, Jimi Xenidis wrote:
> On Feb 7, 2011, at 3:56 PM, Grant Likely wrote:
> > 
> > On Mon, Feb 7, 2011 at 2:53 PM, Yoder Stuart-B08248
> > <B08248-KZfg59tc24xl57MIdRCFDg@public.gmane.org> wrote:
> >> 
> >> 
> >>> -----Original Message-----
> >>> From: glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org [mailto:glikely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org] On Behalf Of Grant
> >>> Likely
> >>> Sent: Monday, February 07, 2011 3:45 PM
> >>> To: Benjamin Herrenschmidt
> >>> Cc: parch-QRwYI7m9GJLYtjvyW6yDsg@public.gmane.org; devicetree-discuss; David Gibson; Yoder Stuart-B08248;
> >>> McClintock Matthew-B29882; Wood Scott-B07421
> >>> Subject: Re: [Power.org:parch] devicetree: Musings on reserved regions
> >>> 
> >>> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
> >>> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
> >>>> 
> >>>>> In addition to the reserved regions block in the header, define a set
> >>>>> of properties in the memory node that specify the reserved regions
> >>>>> with the name reflecting the usage.
> >>>>> For example:
> >>>>> 
> >>>>>       memory@0 {
> >>>>>               device_type = "memory";
> >>>>>               reg = <0 0x40000000>;
> >>>>>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk
> >>>>> */
> >>>>>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
> >>>>>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
> >>>>>       };

This is covering pretty much the same territory that I went through, mostly
on this mailing list, a few months ago. The "reserved-" prefix is a Bad Thing
because there is no way to search for a property with a given prefix, nor
do I think there should be such a thing because it defeats the whole purpose
of having properties common to all devices of a given type.

Also, most reserved things (and I'm dealing with quite a few of them) are
associated with a particular device. One common exception is the device
tree itself.  Another is the bootloader, if it sticks around for a while
like a BIOS.

Given that you've got memory reserved for a device, it should really have
a name. If your framebuffer can move, you better be able to find it! So, I
came up with (in crude BNF):

cisco,static-bufs = "\"" <name> "\"" "," "<" <start> <size> ">"
	( [ , ] "\"" <name> "\"" "," "<" <start> <size> ">" )* ;

For example:

cisco,static-bufs = "buf1", <0x12345678 111>,
	"buf2", <0x87654321 222>;

This property can appear under any device. As currently implemented, all
buffer names must be unique device tree-wide, but that allows scoping per
device later, if desired.

This does end up with the start and size values generally being unaligned,
but as I understand it, if you mix items like this, it's just the way it
works. The code to parse this is really simple and doesn't involve adding
an API to look for properties starting with some string.

As far as the device tree and bootloader memory are concerned, well, we could
make them pseudo-devices.

(I'm *not* proposing cisco,static-bufs be the name of this property,
just tossing out another direction).
-- 
David VL

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Benjamin Herrenschmidt]

On Mon, 2011-02-07 at 14:45 -0700, Grant Likely wrote:
> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
> >
> >> In addition to the reserved regions block in the header, define a set
> >> of properties in the memory node that specify the reserved regions
> >> with the name reflecting the usage.
> >> For example:
> >>
> >>       memory@0 {
> >>               device_type = "memory";
> >>               reg = <0 0x40000000>;
> >>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
> >>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
> >>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
> >>       };
> >>
> >> Each reserved property would start with "reserved-" followed by a
> >> name.  reserved-ramdisk and reserved-dtb would be reserved for ramdisk
> >> and dtb images respectively.  Other names could also be defined; for
> >> example, reserved-openfirmware.
> >
> > I very much dislike the list of properties with magic names. I'd rather
> > have a pair of properties containing lists (reserved-names and
> > reserved-ranges).
> 
> Heh, I have very much the opposite opinion.  I'd rather have property
> names that match the usage than have the names and values split into
> two properties.  To me, keeping them together is more tasteful.  Queue
> the debate...  :-)

Ideally you want a single property but it's nasty to mix strings and
numbers in a single property like that.

We also have no good APIs to iterate properties, we'd have to add one.

And then there's the property name limits in the spec, charset limits
etc... property names shall not per-se contain data I believe.

Cheers,
Ben.

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Grant Likely]

On Mon, Feb 7, 2011 at 3:00 PM, Jimi Xenidis <jimix-e+AXbWqSrlAAvxtiuMwx3w@public.gmane.org> wrote:
> On Feb 7, 2011, at 3:45 PM, Grant Likely wrote:
>>
>> On Mon, Feb 7, 2011 at 2:39 PM, Benjamin Herrenschmidt
>> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
>>>
>>>> In addition to the reserved regions block in the header, define a set
>>>> of properties in the memory node that specify the reserved regions
>>>> with the name reflecting the usage.
>>>> For example:
>>>>
>>>>       memory@0 {
>>>>               device_type = "memory";
>>>>               reg = <0 0x40000000>;
>>>>               reserved-ramdisk = <0xc00000 0x200000>; /* 2MB ramdisk */
>>>>               reserved-dtb = <0xbf0000 0x1000>; /* devicetree */
>>>>               reserved-fb0 = 0x1000000 0x400000>; /* framebuffer */
>>>>       };
>>>>
>>>> Each reserved property would start with "reserved-" followed by a
>>>> name.  reserved-ramdisk and reserved-dtb would be reserved for ramdisk
>>>> and dtb images respectively.  Other names could also be defined; for
>>>> example, reserved-openfirmware.
>>>
>>> I very much dislike the list of properties with magic names. I'd rather
>>> have a pair of properties containing lists (reserved-names and
>>> reserved-ranges).
>>
>> Heh, I have very much the opposite opinion.  I'd rather have property
>> names that match the usage than have the names and values split into
>> two properties.  To me, keeping them together is more tasteful.  Queue
>> the debate...  :-)
>
> ok.. here goes nothing...
> When I see these "reservations", it makes me think that they should be nodes anyway. (off root?)
> Once we have nodes for these interesting things, perhaps the next step is to just have a "reserved" property for these nodes?
> so:
>        ramdisk@<addr> {
>                reg = < addr, size >;
>                format = < "gzip", "cpio" >;  // optional, order of wrapper?
>                reserved;
>        }
>
> No, I'm not ready to propose the ramdisk node, but it is something to consider.

Hmmm, of all the ideas, I think I like this one the best.  It is a
little more verbose than I was thinking, but making each reserved
region (or set of reserved regions) have separate node has some very
real advantages.  For one, the address decoding *just works* with our
existing helpers.  Also it means that device nodes can directly
indicate the reserved regions associated with it by way of a phandle.

Naming I think could simply be "reserved@<addr>" since the usage of
each node can be indicated by phandles or driver-specific properties
within the node, plus it kind of follows the lead of the 'generic
names' recommended practise.  Something like 'format' I suppose could
be specified, but I don't think it needs to be part of the reserved
binding.  Should these nodes have a compatible property that describes
it's usage?  ie; something like:

for a data block: compatible = "reserved-data", "reserved-memory";
	(like for an initrd or the dtb)
for a framebuffer: compatible = "reserved-framebuffer", "reserved-memory":
for shared memory: compatible = "reserved-shared", "reserved-memory";

On Mon, Feb 7, 2011 at 3:50 PM, Dan Hettena <danh-5j0BnJlmnLM@public.gmane.org> wrote:
> I've seen three types of reserved regions so far:
>
> (1) Reserved regions that are permanently reserved. For example,
>    RAM used for configuration of a device not communicated in the
>    device tree. Nothing further needs to be said about this class.
> (2) Reserved regions that are in use by something in the device tree
>    (for example, a spin table, or a frame buffer for a video device).
>    The client program can free these if it recognizes the node that
>    is using them and reprograms it (e.g., boot all processors, or
>    reprogram the video device).
> (3) Reserved regions that contain data the client program might want to
>    use but are not actually being used by the hardware for anything.
>    For example, a RAM root file system.
>
> I think this points strongly at having type 2 and type 3
> reservations specified along with the nodes that make use of them,
> so that the client program can free the memory as soon as it has
> caused the reserved memory to no longer be in use for the purpose
> for which it was reserved (provided that we make it clear that
> reservations declared as such are not permitted to overlap).

Yes.  I think phandles would do the trick nicely for this.  It would
also be possible to allow an initrd-region = <&initrd_reserved>;
property in the chosen node, but there probably isn't much benefit in
doing so, and it would just create yet another new binding that client
programs need to support in addition to the old binding.

>
> Primarily because of type 1 reservations, I see no conflict between
> the conventional reservations list and the idea of explicitly
> identifying single-purpose reservations throughout the device tree
> where they are used.

IMO, type 1 reservations still need to be specified explicitly in the
memory nodes (instead of, or in addition to, in the header).  It just
makes life so much easier when all the data about the machine is fully
specified in the device tree proper.  Otherwise extra hoops need to be
jumped through to get at it.

However, if nothing in the tree actually references a reserved
region node, then the client must treat it as an 'anonymous' region
and it does not have the option of ignoring or freeing it.

> However, in the spirit of distributing memory nodes, I think I would
> rather see the conventional reservations list moved to the
> corresponding memory nodes (with an added rule that a reservation
> may not cross a reg entry boundary, since that's just a big pain).
> If we're going to work towards keeping crap out of the header, then
> I don't see why the reservations block should be treated any
> differently.

I 100% agree.  The reserved regions nodes should be children of the
associated memory node, and reservations should not cross boundaries.
Collecting all the reservations in one place also give the client a
fighting chance of locating them easily when it parses the memory
node.

g.

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Benjamin Herrenschmidt]

On Mon, 2011-02-07 at 22:05 -0700, Grant Likely wrote:
> 
> Hmmm, of all the ideas, I think I like this one the best.  It is a
> little more verbose than I was thinking, but making each reserved
> region (or set of reserved regions) have separate node has some very
> real advantages.  For one, the address decoding *just works* with our
> existing helpers.  Also it means that device nodes can directly
> indicate the reserved regions associated with it by way of a phandle.

But that means that anything that needs to setup a simple memory
allocator (and thus needs to know what to avoid) will need to walk the
entire tree for nodes with a "reserved" property, be able to parse
addresses down to that node (not always obvious accross some bus types),
etc...

Or do we propose to keep a "consolidated" reserve map in addition to
having nodes self-describe which part of this reserve map they are
responsible for, which thus gives the OS the option to:

 - Reserve it all
 - Then, based on specific knowledge of some nodes, maybe release
or juts "tag" (for diag purposes) portions of reserved memory

Ben.

Re: [Power.org:parch] devicetree: Musings on reserved regions

[Jimi Xenidis]

On Feb 8, 2011, at 12:45 AM, Benjamin Herrenschmidt wrote:
> 
> On Mon, 2011-02-07 at 22:05 -0700, Grant Likely wrote:
>> 
>> Hmmm, of all the ideas, I think I like this one the best.  It is a
>> little more verbose than I was thinking, but making each reserved
>> region (or set of reserved regions) have separate node has some very
>> real advantages.  For one, the address decoding *just works* with our
>> existing helpers.  Also it means that device nodes can directly
>> indicate the reserved regions associated with it by way of a phandle.
> 
> But that means that anything that needs to setup a simple memory
> allocator (and thus needs to know what to avoid) will need to walk the
> entire tree for nodes with a "reserved" property, be able to parse
> addresses down to that node (not always obvious accross some bus types),
> etc...
> 
> Or do we propose to keep a "consolidated" reserve map in addition to
> having nodes self-describe which part of this reserve map they are
> responsible for, which thus gives the OS the option to:
> 
> - Reserve it all
> - Then, based on specific knowledge of some nodes, maybe release
> or juts "tag" (for diag purposes) portions of reserved memory

Excellent point.
I always thought we were keeping the reservation list and it starts "anonymous".
Only later to we want to "name" it.
We could define a standard property (for any node) called "reserved" that is the range within the list.
I suppose if it is empty, then it is all of "reg"
-JX


> 
> Ben.

Re: devicetree: Musings on reserved regions

[David Gibson]

On Mon, Feb 07, 2011 at 01:59:53PM -0700, Grant Likely wrote:
> As part of the process of bringing dt support up on non-powerpc
> platforms, I've been thinking about the usage mode of the reserved
> regions section in the flattened device tree structure.  First, for
> reference, here is the description of reserved regions from ePAPR
> 1.0:
> 
> > 7.3 Memory Reservation Block
> > 7.3.1 Purpose
> > The memory reservation block provides the client program with
> > a list of areas in physical memory which are reserved; that
> > is, which shall not be used for general memory allocations. It
> > is used to protect vital data structures from being
> > overwritten by the client program. For example, on some
> > systems with an IOMMU, the TCE (translation control entry)
> > tables initialized by an ePAPR boot program would need to be
> > protected in this manner. Likewise, any boot program code or
> > data used during the client program’s runtime would need to be
> > reserved (e.g., RTAS on Open Firmware platforms). The ePAPR
> > does not require the boot program to provide any such runtime
> > components, but it does not prohibit implementations from
> > doing so as an extension.
> >
> > More specifically, a client program shall not access memory in
> > a reserved region unless other information provided by the
> > boot program explicitly indicates that it shall do so. The
> > client program may then access the indicated section of the
> > reserved memory in the indicated manner. Methods by which the
> > boot program can indicate to the client program specific uses
> > for reserved memory may appear in this document, in optional
> > extensions to it, or in platform-specific documentation.
> >
> > The reserved regions supplied by a boot program may, but are
> > not required to, encompass the device tree blob itself. The
> > client program shall ensure that it does not overwrite this
> > data structure before it is used, whether or not it is in the
> > reserved areas.
> 
> The problem that I'm having, is that the lifecycle for reserved
> regions is not defined and the client program has no mechanism to
> determine what each reserved region is intended for.  This issue has
> been raised before in the context of kexec from one Linux instance to
> another[1].  When going from one kernel to the next, kexec needs to
> populate a valid set of reserved regions, but it isn't easy to figure
> out what the new set should be because there isn't a reliable way to
> figure out which regions from the previous boot need to be preserved
> (ie. runtime firmware or framebuffers) vs regions that are obsolete
> (original initrd and dtb regions).

Hrm, I agree that the fact that the reserved sections meaning and
lifetime may not be known is awkward, but I'm not yet convinced that
we need something like this.

Here's how I envisaged this working under the current spec, which the
text is meant to convey (but maybe needs work):

So, you start with the reserved regions from the special block.  It
wants to be here, rather than built into the tree, because you need to
parse this before doing *any* memory allocation, and doing tree
parsing and tracking the regions that early, without memory allocation
is painful.

Absent other information, those regions must remain untouched
*forever*.  The client program must not read or right them, and it
must pass those regions onto subsequent things to run - that could be
kexec(), or the real OS if this client is a second stage loader.

However other parts of the spec may desribe things giving a specific
use of part of the reserved area, for example, the initrd-start/end
properties and the spin-table entries.

The binding that describes some use of reserved memory may also bound
that reservation's lifetime.  So, once you've processed the initrd
properties, you may excise the initrd area from the reserved sections.
There is no guarantee the initrd reservation maps to exactly one entry
in the reserve map, so yes, this may mean some fairly involved extent
intersection type code to do.

Likewise the spin-table means that once you have brought a secondary
cpu out of spin with that method, you may reclaim the 16 (?) bytes of
its spin table entry (and no more).

Now, that's pretty ad-hoc, but the point is that you *must* understand
the use of a section of reserved memory before de-reserving it anyway,
and there is no guarantee that a reserved area can *ever* be freed.

Furthermore, I don't think we should require the boot program to bind
each reserve entry to exactly one purpose.  It seems to be a obvious
scheme for a boot program would be to lay out all the things it needs
to sequentially in RAM, and put in just one reserve entry for the
whole block.  For one thing, that means it will never need to resize /
memmove() the dt blob even if what things it needs to reserve vary.

Remember that all else being equal client program complexity is
preferable to boot program complexity.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
_______________________________________________
devicetree-discuss mailing list
devicetree-discuss@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/devicetree-discuss