[Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Peter Maydell]

The documentation of how overlapping memory regions behave and how
the priority system works was rather brief, and confusion about
priorities seems to be quite common for developers trying to understand
how the memory region system works, so expand and clarify it.
This includes a worked example with overlaps, documentation of the
behaviour when an overlapped container has "holes", and mention
that it's valid for a region to have both MMIO callbacks and
subregions (and how this interacts with priorities when it does).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
Changes v1->v2: various minor improvements as per review comments
 from MST and subsequent discussion.

 docs/memory.txt | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 62 insertions(+), 2 deletions(-)

diff --git a/docs/memory.txt b/docs/memory.txt
index feb9fe9..4035a20 100644
--- a/docs/memory.txt
+++ b/docs/memory.txt
@@ -52,6 +52,15 @@ MemoryRegion):
   hole".  Aliases may point to any type of region, including other aliases,
   but an alias may not point back to itself, directly or indirectly.
 
+It is valid to add subregions to a region which is not a pure container
+(that is, to an MMIO, RAM or ROM region). This means that the region
+will act like a container, except that any addresses within the container's
+region which are not claimed by any subregion are handled by the
+container itself (ie by its MMIO callbacks or RAM backing). However
+it is generally possible to achieve the same effect with a pure container
+one of whose subregions is a low priority "background" region covering
+the whole address range; this is often clearer and is preferred.
+Subregions cannot be added to an alias region.
 
 Region names
 ------------
@@ -81,6 +90,49 @@ allows the region to overlap any other region in the same container, and
 specifies a priority that allows the core to decide which of two regions at
 the same address are visible (highest wins).
 
+If the higher priority region in an overlap is a container or alias, then
+the lower priority region will appear in any "holes" that the higher priority
+region has left by not mapping subregions to that area of its address range.
+(This applies recursively -- if the subregions are themselves containers or
+aliases that leave holes then the lower priority region will appear in these
+holes too.)
+
+For example, suppose we have a container A of size 0x8000 with two subregions
+B and C. B is a container mapped at 0x2000, size 0x4000, priority 1; C is
+an MMIO region mapped at 0x0, size 0x6000, priority 2. B currently has two
+of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at
+offset 0x2000. As a diagram:
+
+        0      1000   2000   3000   4000   5000   6000   7000    8000
+        |------|------|------|------|------|------|------|-------|
+  A:    [                                                       ]
+  C:    [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC]
+  B:                  [                          ]
+  D:                  [DDDDD]
+  E:                                [EEEEE]
+
+The regions that will be seen within this address range then are:
+        [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC]
+
+Since B has higher priority than C, its subregions appear in the flat map
+even where they overlap with C. In ranges where B has not mapped anything
+C's region appears.
+
+If B had provided its own MMIO operations (ie it was not a pure container)
+then these would be used for any addresses in its range not handled by
+D or E, and the result would be:
+        [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB]
+
+Priority values are local to a container, because the priorities of two
+regions are only compared when they are both children of the same container.
+This means that the device in charge of the container (typically modelling
+a bus or a memory controller) can use them to manage the interaction of
+its child regions without any side effects on other parts of the system.
+In the example above, the priorities of D and E are unimportant because
+they do not overlap each other. It is the relative priority of B and C
+that causes D and E to appear on top of C: D and E's priorities are never
+compared against the priority of C.
+
 Visibility
 ----------
 The memory core uses the following rules to select a memory region when the
@@ -90,11 +142,19 @@ guest accesses an address:
   descending priority order
   - if the address lies outside the region offset/size, the subregion is
     discarded
-  - if the subregion is a leaf (RAM or MMIO), the search terminates
+  - if the subregion is a leaf (RAM or MMIO), the search terminates, returning
+    this leaf region
   - if the subregion is a container, the same algorithm is used within the
     subregion (after the address is adjusted by the subregion offset)
-  - if the subregion is an alias, the search is continues at the alias target
+  - if the subregion is an alias, the search is continued at the alias target
     (after the address is adjusted by the subregion offset and alias offset)
+  - if a recursive search within a container or alias subregion does not
+    find a match (because of a "hole" in the container's coverage of its
+    address range), then if this is a container with its own MMIO or RAM
+    backing the search terminates, returning the container itself. Otherwise
+    we continue with the next subregion in priority order
+- if none of the subregions match the address then the search terminates
+  with no match found
 
 Example memory map
 ------------------
-- 
1.7.11.4

Re: [Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Michael S. Tsirkin]

On Tue, Oct 15, 2013 at 03:42:34PM +0100, Peter Maydell wrote:
> The documentation of how overlapping memory regions behave and how
> the priority system works was rather brief, and confusion about
> priorities seems to be quite common for developers trying to understand
> how the memory region system works, so expand and clarify it.
> This includes a worked example with overlaps, documentation of the
> behaviour when an overlapped container has "holes", and mention
> that it's valid for a region to have both MMIO callbacks and
> subregions (and how this interacts with priorities when it does).
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Thanks a lot for writing this!

Reviewed-by: Michael S. Tsirkin <mst@redhat.com>

> ---
> Changes v1->v2: various minor improvements as per review comments
>  from MST and subsequent discussion.
> 
>  docs/memory.txt | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 62 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/memory.txt b/docs/memory.txt
> index feb9fe9..4035a20 100644
> --- a/docs/memory.txt
> +++ b/docs/memory.txt
> @@ -52,6 +52,15 @@ MemoryRegion):
>    hole".  Aliases may point to any type of region, including other aliases,
>    but an alias may not point back to itself, directly or indirectly.
>  
> +It is valid to add subregions to a region which is not a pure container
> +(that is, to an MMIO, RAM or ROM region). This means that the region
> +will act like a container, except that any addresses within the container's
> +region which are not claimed by any subregion are handled by the
> +container itself (ie by its MMIO callbacks or RAM backing). However
> +it is generally possible to achieve the same effect with a pure container
> +one of whose subregions is a low priority "background" region covering
> +the whole address range; this is often clearer and is preferred.
> +Subregions cannot be added to an alias region.
>  
>  Region names
>  ------------
> @@ -81,6 +90,49 @@ allows the region to overlap any other region in the same container, and
>  specifies a priority that allows the core to decide which of two regions at
>  the same address are visible (highest wins).
>  
> +If the higher priority region in an overlap is a container or alias, then
> +the lower priority region will appear in any "holes" that the higher priority
> +region has left by not mapping subregions to that area of its address range.
> +(This applies recursively -- if the subregions are themselves containers or
> +aliases that leave holes then the lower priority region will appear in these
> +holes too.)
> +
> +For example, suppose we have a container A of size 0x8000 with two subregions
> +B and C. B is a container mapped at 0x2000, size 0x4000, priority 1; C is
> +an MMIO region mapped at 0x0, size 0x6000, priority 2. B currently has two
> +of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at
> +offset 0x2000. As a diagram:
> +
> +        0      1000   2000   3000   4000   5000   6000   7000    8000
> +        |------|------|------|------|------|------|------|-------|
> +  A:    [                                                       ]
> +  C:    [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC]
> +  B:                  [                          ]
> +  D:                  [DDDDD]
> +  E:                                [EEEEE]
> +
> +The regions that will be seen within this address range then are:
> +        [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC]
> +
> +Since B has higher priority than C, its subregions appear in the flat map
> +even where they overlap with C. In ranges where B has not mapped anything
> +C's region appears.
> +
> +If B had provided its own MMIO operations (ie it was not a pure container)
> +then these would be used for any addresses in its range not handled by
> +D or E, and the result would be:
> +        [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB]
> +
> +Priority values are local to a container, because the priorities of two
> +regions are only compared when they are both children of the same container.
> +This means that the device in charge of the container (typically modelling
> +a bus or a memory controller) can use them to manage the interaction of
> +its child regions without any side effects on other parts of the system.
> +In the example above, the priorities of D and E are unimportant because
> +they do not overlap each other. It is the relative priority of B and C
> +that causes D and E to appear on top of C: D and E's priorities are never
> +compared against the priority of C.
> +
>  Visibility
>  ----------
>  The memory core uses the following rules to select a memory region when the
> @@ -90,11 +142,19 @@ guest accesses an address:
>    descending priority order
>    - if the address lies outside the region offset/size, the subregion is
>      discarded
> -  - if the subregion is a leaf (RAM or MMIO), the search terminates
> +  - if the subregion is a leaf (RAM or MMIO), the search terminates, returning
> +    this leaf region
>    - if the subregion is a container, the same algorithm is used within the
>      subregion (after the address is adjusted by the subregion offset)
> -  - if the subregion is an alias, the search is continues at the alias target
> +  - if the subregion is an alias, the search is continued at the alias target
>      (after the address is adjusted by the subregion offset and alias offset)
> +  - if a recursive search within a container or alias subregion does not
> +    find a match (because of a "hole" in the container's coverage of its
> +    address range), then if this is a container with its own MMIO or RAM
> +    backing the search terminates, returning the container itself. Otherwise
> +    we continue with the next subregion in priority order
> +- if none of the subregions match the address then the search terminates
> +  with no match found
>  
>  Example memory map
>  ------------------
> -- 
> 1.7.11.4

Re: [Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1588 bytes --]

On 10/15/2013 08:42 AM, Peter Maydell wrote:
> The documentation of how overlapping memory regions behave and how
> the priority system works was rather brief, and confusion about
> priorities seems to be quite common for developers trying to understand
> how the memory region system works, so expand and clarify it.
> This includes a worked example with overlaps, documentation of the
> behaviour when an overlapped container has "holes", and mention
> that it's valid for a region to have both MMIO callbacks and
> subregions (and how this interacts with priorities when it does).
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> Changes v1->v2: various minor improvements as per review comments
>  from MST and subsequent discussion.
> 

>  
> +It is valid to add subregions to a region which is not a pure container
> +(that is, to an MMIO, RAM or ROM region). This means that the region
> +will act like a container, except that any addresses within the container's
> +region which are not claimed by any subregion are handled by the
> +container itself (ie by its MMIO callbacks or RAM backing). However

s/ie/ie./

> +it is generally possible to achieve the same effect with a pure container

s/container/container,/

> +one of whose subregions is a low priority "background" region covering
> +the whole address range; this is often clearer and is preferred.
> +Subregions cannot be added to an alias region.
>  


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]

On 10/15/2013 08:52 AM, Eric Blake wrote:
> On 10/15/2013 08:42 AM, Peter Maydell wrote:
>> The documentation of how overlapping memory regions behave and how
>> the priority system works was rather brief, and confusion about
>> priorities seems to be quite common for developers trying to understand
>> how the memory region system works, so expand and clarify it.
>> This includes a worked example with overlaps, documentation of the
>> behaviour when an overlapped container has "holes", and mention
>> that it's valid for a region to have both MMIO callbacks and
>> subregions (and how this interacts with priorities when it does).
>>
>> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>> ---
>> Changes v1->v2: various minor improvements as per review comments
>>  from MST and subsequent discussion.
>>
> 
>>  
>> +It is valid to add subregions to a region which is not a pure container
>> +(that is, to an MMIO, RAM or ROM region). This means that the region
>> +will act like a container, except that any addresses within the container's
>> +region which are not claimed by any subregion are handled by the
>> +container itself (ie by its MMIO callbacks or RAM backing). However
> 
> s/ie/ie./
> 
>> +it is generally possible to achieve the same effect with a pure container
> 
> s/container/container,/

Oh, I forgot to add: I just reviewed for grammar and spelling; I can't
state whether the contents are technically sound, hence I didn't feel
comfortable adding a reviewed-by.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Peter Maydell]

On 15 October 2013 15:52, Eric Blake <eblake@redhat.com> wrote:
> On 10/15/2013 08:42 AM, Peter Maydell wrote:
>> +It is valid to add subregions to a region which is not a pure container
>> +(that is, to an MMIO, RAM or ROM region). This means that the region
>> +will act like a container, except that any addresses within the container's
>> +region which are not claimed by any subregion are handled by the
>> +container itself (ie by its MMIO callbacks or RAM backing). However
>
> s/ie/ie./

I'll take 'ie' or 'i.e.' but not 'ie.' (and 'i.e.' is pretty
unwieldy).

>> +it is generally possible to achieve the same effect with a pure container
>
> s/container/container,/
>
>> +one of whose subregions is a low priority "background" region covering
>> +the whole address range; this is often clearer and is preferred.
>> +Subregions cannot be added to an alias region.

No, the comma would be wrong: this is a restrictive relative
clause, not a non-restrictive one.

-- PMM

Re: [Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Peter Maydell]

On 15 October 2013 15:42, Peter Maydell <peter.maydell@linaro.org> wrote:
> The documentation of how overlapping memory regions behave and how
> the priority system works was rather brief, and confusion about
> priorities seems to be quite common for developers trying to understand
> how the memory region system works, so expand and clarify it.
> This includes a worked example with overlaps, documentation of the
> behaviour when an overlapped container has "holes", and mention
> that it's valid for a region to have both MMIO callbacks and
> subregions (and how this interacts with priorities when it does).
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Ping!

thanks
-- PMM

Re: [Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Michael S. Tsirkin]

On Tue, Nov 05, 2013 at 05:11:09PM +0000, Peter Maydell wrote:
> On 15 October 2013 15:42, Peter Maydell <peter.maydell@linaro.org> wrote:
> > The documentation of how overlapping memory regions behave and how
> > the priority system works was rather brief, and confusion about
> > priorities seems to be quite common for developers trying to understand
> > how the memory region system works, so expand and clarify it.
> > This includes a worked example with overlaps, documentation of the
> > behaviour when an overlapped container has "holes", and mention
> > that it's valid for a region to have both MMIO callbacks and
> > subregions (and how this interacts with priorities when it does).
> >
> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> 
> Ping!
> 
> thanks
> -- PMM

Want to fix minor typos that Eric found?

Re: [Qemu-devel] [PATCH v2] docs/memory.txt: Clarify and expand priority/overlap documentation

[Peter Maydell]

On 5 November 2013 17:16, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Tue, Nov 05, 2013 at 05:11:09PM +0000, Peter Maydell wrote:
>> On 15 October 2013 15:42, Peter Maydell <peter.maydell@linaro.org> wrote:
>> > The documentation of how overlapping memory regions behave and how
>> > the priority system works was rather brief, and confusion about
>> > priorities seems to be quite common for developers trying to understand
>> > how the memory region system works, so expand and clarify it.
>> > This includes a worked example with overlaps, documentation of the
>> > behaviour when an overlapped container has "holes", and mention
>> > that it's valid for a region to have both MMIO callbacks and
>> > subregions (and how this interacts with priorities when it does).
>> >
>> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>>
>> Ping!

> Want to fix minor typos that Eric found?

I disagreed with them, see my reply to him.

-- PMM