[RFC 4/4] mm: Add documentation for coherent memory

[Balbir Singh <bsingharora@gmail.com>]

Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
 Documentation/vm/00-INDEX            |  2 ++
 Documentation/vm/coherent-memory.txt | 59 ++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100644 Documentation/vm/coherent-memory.txt

diff --git a/Documentation/vm/00-INDEX b/Documentation/vm/00-INDEX
index 11d3d8d..99175e9 100644
--- a/Documentation/vm/00-INDEX
+++ b/Documentation/vm/00-INDEX
@@ -6,6 +6,8 @@ balance
 	- various information on memory balancing.
 cleancache.txt
 	- Intro to cleancache and page-granularity victim cache.
+coherent-memory.txt
+	- Introduction to coherent memory handling (N_COHERENT_MEMORY)
 frontswap.txt
 	- Outline frontswap, part of the transcendent memory frontend.
 highmem.txt
diff --git a/Documentation/vm/coherent-memory.txt b/Documentation/vm/coherent-memory.txt
new file mode 100644
index 0000000..bd60e5b
--- /dev/null
+++ b/Documentation/vm/coherent-memory.txt
@@ -0,0 +1,59 @@
+Introduction
+
+This document describes a new type of node called N_COHERENT_MEMORY.
+This memory is cache coherent with system memory and we would like
+this to show up as a NUMA node, however there are certain algorithms
+that might not be currently suitable for N_COHERENT_MEMORY
+
+1. AutoNUMA balancing
+2. kswapd reclaim
+
+The reason for exposing this device memory as NUMA is to simplify
+the programming model, where memory allocation via malloc() or
+mmap() for example would seamlessly work across both kinds of
+memory. Since we expect the size of device memory to be smaller
+than system RAM, we would like to control the allocation of such
+memory. The proposed mechanism reuses nodemasks and explicit
+specification of the coherent node in the nodemask for allocation
+from device memory. This implementation also allows for kernel
+level allocation via __GFP_THISNODE and existing techniques
+such as page migration to work.
+
+Assumptions:
+
+1. Nodes with N_COHERENT_MEMORY don't have CPUs on them, so
+effectively they are CPUless memory nodes
+2. Nodes with N_COHERENT_MEMORY are marked as movable_nodes.
+Slub allocations from these nodes will fail otherwise.
+
+Implementation Details
+
+A new node state N_COHERENT_MEMORY is created. Each architecture
+can then mark devices as being N_COHERENT_MEMORY and the implementation
+makes sure this node set is disjoint from the N_MEMORY node state
+nodes. A typical node zonelist (FALLBACK) with N_COHERENT_MEMORY would
+be:
+
+Assuming we have 2 nodes and 1 coherent memory node
+
+Node1:	Node 1 --> Node 2
+
+Node2:	Node 2 --> Node 1
+
+Node3:	Node 3 --> Node 2 --> Node 1
+
+This effectively means that allocations that have Node 1 and Node 2
+in the nodemask will not allocate from Node 3. Allocations with __GFP_THISNODE
+use the NOFALLBACK list and should allocate from Node 3, if it
+is specified.  Since Node 3 has no CPUs, we don't expect any default
+allocations occurring from it.
+
+However to support allocation from the coherent node, changes have been
+made to mempolicy, specifically policy_nodemask() and policy_zonelist()
+such that
+
+1. MPOL_BIND with the coherent node (Node 3 in the above example) will
+not filter out N_COHERENT_MEMORY if any of the nodes in the nodemask
+is in N_COHERENT_MEMORY
+2. MPOL_PREFERRED will use the FALLBACK list of the coherent node (Node 3)
+if a policy that specifies a preference to it is used.
-- 
2.9.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>