Re: [PATCH] Document Linux Memory Policy - V2

Christoph Lameter
2007-07-27T18:55:35Z

On Fri, 27 Jul 2007, Lee Schermerhorn wrote:

> + Shared Policy: This policy applies to "memory objects" mapped shared into
> + one or more tasks' distinct address spaces. Shared policies are applied
> + directly to the shared object. Thus, all tasks that attach to the object
> + share the policy, and all pages allocated for the shared object, by any
> + task, will obey the shared policy.

This applies to shmem only not to shared memory. Shared memory can also
come about by mmapping a file etc. Its better to describe shmem
as an exceptional situation later and warn of the surprises coming with
the use of memory policies on shmem in a separate section.

> + MPOL_BIND: This mode specifies that memory must come from the
> + set of nodes specified by the policy. The kernel builds a custom
> + zonelist pointed to by the zonelist member of struct mempolicy,
> + containing just the nodes specified by the Bind policy. If the kernel
> + is unable to allocate a page from the first node in the custom zonelist,
> + it moves on to the next, and so forth. If it is unable to allocate a
> + page from any of the nodes in this list, the allocation will fail.

The implementation details may not be useful to explain here and may
change soon. Maybe just describe the effect?

> + The memory policy APIs do not specify an order in which the nodes
> + will be searched. However, unlike the per node zonelists mentioned
> + above, the custom zonelist for the Bind policy do not consider the
> + distance between the nodes. Rather, the lists are built in order
> + of numeric node id.

Yea another reson to get the nodemask as a parameter for alloc_pages().

> +2) when tasks in two cpusets share access to a memory region, such as shared
> + memory segments created by shmget() of mmap() with the MAP_ANONYMOUS and
> + MAP_SHARED flags, only nodes whose memories are allowed in both cpusets
> + may be used in the policies. Again, obtaining this information requires
> + "stepping outside" the memory policy APIs to use the cpuset information.
> + Furthermore, if the cpusets' "allowed memory" sets are disjoint, "local"
> + allocation is the only valid policy.

In general this works fine with a shared mapping via mmap (which is much
more common). The problem exists if one uses shmem with the strange shared
semantics.