From: Anshuman Khandual <khandual@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org, linux-mm@kvack.org
Cc: mhocko@suse.com, js1304@gmail.com, vbabka@suse.cz,
mgorman@suse.de, minchan@kernel.org, akpm@linux-foundation.org,
aneesh.kumar@linux.vnet.ibm.com, bsingharora@gmail.com
Subject: [DEBUG 00/10] Test and debug patches for coherent device memory
Date: Mon, 24 Oct 2016 10:12:19 +0530 [thread overview]
Message-ID: <1477284149-2976-1-git-send-email-khandual@linux.vnet.ibm.com> (raw)
In-Reply-To: <1477283517-2504-1-git-send-email-khandual@linux.vnet.ibm.com>
Coherent device memory support has been experimented around on
POWER platform with simulations and QEMU changes. This series contains
patches which can be classified into three categories.
(1) Memory less node hot plug support
(2) Identifying coherent device nodes during NUMA init
(3) Debug patches to observe zonelists information
(4) Test drivers and scripts
Patch (2) could have been part of the RFC series but because of the
dependency on patch (1), it goes here. Now lets look at the how all these
components work.
Before Hotplug
==============
NUMACTL Information:
--------------------
available: 5 nodes (0-4)
node 0 cpus: 0 1 2 5 6 20 21 23 27 28 31 32 37 38 39 43 44 48 49 50 51 52
53 54 55 56 57 58 59 60 61 62
node 0 size: 4059 MB
node 0 free: 2956 MB
node 1 cpus: 3 4 7 8 9 10 11 12 13 14 15 16 17 18 19 22 24 25 26 29 30 33
34 35 36 40 41 42 45 46 47 63
node 1 size: 4091 MB
node 1 free: 3920 MB
node 2 cpus:
node 2 size: 0 MB
node 2 free: 0 MB
node 3 cpus:
node 3 size: 0 MB
node 3 free: 0 MB
node 4 cpus:
node 4 size: 0 MB
node 4 free: 0 MB
node distances:
node 0 1 2 3 4
0: 10 40 40 40 40
1: 40 10 40 40 40
2: 40 40 10 40 40
3: 40 40 40 10 40
4: 40 40 40 40 10
ZONELIST Information
--------------------
[NODE (0)]
ZONELIST_FALLBACK (0xc00000000140da00)
(0) (node 0) (DMA 0xc00000000140c000)
(1) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc000000001411a10)
(0) (node 0) (DMA 0xc00000000140c000)
[NODE (1)]
ZONELIST_FALLBACK (0xc000000100001a00)
(0) (node 1) (DMA 0xc000000100000000)
(1) (node 0) (DMA 0xc00000000140c000)
ZONELIST_NOFALLBACK (0xc000000100005a10)
(0) (node 1) (DMA 0xc000000100000000)
[NODE (2)]
ZONELIST_FALLBACK (0xc000000001427700)
(0) (node 0) (DMA 0xc00000000140c000)
(1) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc00000000142b710)
[NODE (3)]
ZONELIST_FALLBACK (0xc000000001431400)
(0) (node 0) (DMA 0xc00000000140c000)
(1) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc000000001435410)
[NODE (4)]
ZONELIST_FALLBACK (0xc00000000143b100)
(0) (node 0) (DMA 0xc00000000140c000)
(1) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc00000000143f110)
After Hotplug
=============
NUMACTL Information:
--------------------
available: 5 nodes (0-4)
node 0 cpus: 0 1 2 5 6 20 21 23 27 28 31 32 37 38 39 43 44 48 49 50 51 52
53 54 55 56 57 58 59 60 61 62
node 0 size: 4059 MB
node 0 free: 2804 MB
node 1 cpus: 3 4 7 8 9 10 11 12 13 14 15 16 17 18 19 22 24 25 26 29 30 33
34 35 36 40 41 42 45 46 47 63
node 1 size: 4091 MB
node 1 free: 3860 MB
node 2 cpus:
node 2 size: 4096 MB
node 2 free: 4095 MB
node 3 cpus:
node 3 size: 4096 MB
node 3 free: 4095 MB
node 4 cpus:
node 4 size: 4096 MB
node 4 free: 4095 MB
node distances:
node 0 1 2 3 4
0: 10 40 40 40 40
1: 40 10 40 40 40
2: 40 40 10 40 40
3: 40 40 40 10 40
4: 40 40 40 40 10
ZONELIST Information:
---------------------
[NODE (0)]
ZONELIST_FALLBACK (0xc00000000140da00)
(0) (node 0) (DMA 0xc00000000140c000)
(1) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc000000001411a10)
(0) (node 0) (DMA 0xc00000000140c000)
[NODE (1)]
ZONELIST_FALLBACK (0xc000000100001a00)
(0) (node 1) (DMA 0xc000000100000000)
(1) (node 0) (DMA 0xc00000000140c000)
ZONELIST_NOFALLBACK (0xc000000100005a10)
(0) (node 1) (DMA 0xc000000100000000)
[NODE (2)]
ZONELIST_FALLBACK (0xc000000001427700)
(0) (node 2) (Movable 0xc000000001427080)
(1) (node 0) (DMA 0xc00000000140c000)
(2) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc00000000142b710)
(0) (node 2) (Movable 0xc000000001427080)
[NODE (3)]
ZONELIST_FALLBACK (0xc000000001431400)
(0) (node 3) (Movable 0xc000000001430d80)
(1) (node 0) (DMA 0xc00000000140c000)
(2) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc000000001435410)
(0) (node 3) (Movable 0xc000000001430d80)
[NODE (4)]
ZONELIST_FALLBACK (0xc00000000143b100)
(0) (node 4) (Movable 0xc00000000143aa80)
(1) (node 0) (DMA 0xc00000000140c000)
(2) (node 1) (DMA 0xc000000100000000)
ZONELIST_NOFALLBACK (0xc00000000143f110)
(0) (node 4) (Movable 0xc00000000143aa80)
After the coherent device memory nodes have been hot plugged into
the kernel, did some simple VMA migration tests to verify it's stability.
cdm_migration.sh does the actual test of moving VMAs of ebizzy workload
which results in the following stats and traces.
Results:
-------
passed 13
failed 0
queuef 0
empty 3
missing 0
Traces:
-------
migrate_virtual_range: 55094 10000000 10010000 0: migration_passed
migrate_virtual_range: 55094 10010000 10020000 0: migration_passed
migrate_virtual_range: 55094 10020000 10030000 3: migration_passed
migrate_virtual_range: 55094 3fff3b6a0000 3fff8b3c0000 0: list_empty
migrate_virtual_range: 55094 3fff8b3c0000 3fff8b580000 1: migration_passed
migrate_virtual_range: 55094 3fff8b580000 3fff8b590000 2: migration_passed
migrate_virtual_range: 55094 3fff8b590000 3fff8b5a0000 0: migration_passed
migrate_virtual_range: 55094 3fff8b5a0000 3fff8b5c0000 2: migration_passed
migrate_virtual_range: 55094 3fff8b5c0000 3fff8b5d0000 2: migration_passed
migrate_virtual_range: 55094 3fff8b5d0000 3fff8b5e0000 0: migration_passed
migrate_virtual_range: 55094 3fff8b5e0000 3fff8b5f0000 0: list_empty
migrate_virtual_range: 55094 3fff8b5f0000 3fff8b610000 3: list_empty
migrate_virtual_range: 55094 3fff8b610000 3fff8b640000 3: migration_passed
migrate_virtual_range: 55094 3fff8b640000 3fff8b650000 2: migration_passed
migrate_virtual_range: 55094 3fff8b650000 3fff8b660000 1: migration_passed
migrate_virtual_range: 55094 3ffff25e0000 3ffff2610000 1: migration_passed
Anshuman Khandual (6):
powerpc/mm: Identify isolation seeking coherent memory nodes during boot
mm: Export definition of 'zone_names' array through mmzone.h
mm: Add debugfs interface to dump each node's zonelist information
powerpc: Enable CONFIG_MOVABLE_NODE for PPC64 platform
drivers: Add two drivers for coherent device memory tests
test: Add a script to perform random VMA migrations across nodes
Reza Arbab (4):
dt-bindings: Add doc for ibm,hotplug-aperture
powerpc/mm: Create numa nodes for hotplug memory
powerpc/mm: Allow memory hotplug into a memory less node
mm: Enable CONFIG_MOVABLE_NODE on powerpc
.../bindings/powerpc/opal/hotplug-aperture.txt | 26 ++
Documentation/kernel-parameters.txt | 2 +-
arch/powerpc/Kconfig | 4 +
arch/powerpc/mm/numa.c | 43 ++-
drivers/char/Kconfig | 23 ++
drivers/char/Makefile | 2 +
drivers/char/coherent_hotplug_demo.c | 133 ++++++++
drivers/char/coherent_memory_demo.c | 337 +++++++++++++++++++++
drivers/char/memory_online_sysfs.h | 148 +++++++++
include/linux/mmzone.h | 1 +
mm/Kconfig | 2 +-
mm/memory.c | 63 ++++
mm/migrate.c | 10 +
mm/page_alloc.c | 2 +-
tools/testing/selftests/vm/cdm_migration.sh | 76 +++++
15 files changed, 855 insertions(+), 17 deletions(-)
create mode 100644 Documentation/devicetree/bindings/powerpc/opal/hotplug-aperture.txt
create mode 100644 drivers/char/coherent_hotplug_demo.c
create mode 100644 drivers/char/coherent_memory_demo.c
create mode 100644 drivers/char/memory_online_sysfs.h
create mode 100755 tools/testing/selftests/vm/cdm_migration.sh
--
2.1.0
next prev parent reply other threads:[~2016-10-24 4:42 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-24 4:31 [RFC 0/8] Define coherent device memory node Anshuman Khandual
2016-10-24 4:31 ` [RFC 1/8] mm: " Anshuman Khandual
2016-10-24 17:09 ` Dave Hansen
2016-10-25 1:22 ` Anshuman Khandual
2016-10-25 15:47 ` Dave Hansen
2016-10-24 4:31 ` [RFC 2/8] mm: Add specialized fallback zonelist for coherent device memory nodes Anshuman Khandual
2016-10-24 17:10 ` Dave Hansen
2016-10-25 1:27 ` Anshuman Khandual
2016-11-17 7:40 ` Anshuman Khandual
2016-11-17 7:59 ` [DRAFT 1/2] mm/cpuset: Exclude CDM nodes from each task's mems_allowed node mask Anshuman Khandual
2016-11-17 7:59 ` [DRAFT 2/2] mm/hugetlb: Restrict HugeTLB allocations only to the system RAM nodes Anshuman Khandual
2016-11-17 8:28 ` [DRAFT 1/2] mm/cpuset: Exclude CDM nodes from each task's mems_allowed node mask kbuild test robot
2016-10-24 4:31 ` [RFC 3/8] mm: Isolate coherent device memory nodes from HugeTLB allocation paths Anshuman Khandual
2016-10-24 17:16 ` Dave Hansen
2016-10-25 4:15 ` Aneesh Kumar K.V
2016-10-25 7:17 ` Balbir Singh
2016-10-25 7:25 ` Balbir Singh
2016-10-24 4:31 ` [RFC 4/8] mm: Accommodate coherent device memory nodes in MPOL_BIND implementation Anshuman Khandual
2016-10-24 4:31 ` [RFC 5/8] mm: Add new flag VM_CDM for coherent device memory Anshuman Khandual
2016-10-24 17:38 ` Dave Hansen
2016-10-24 18:00 ` Dave Hansen
2016-10-25 12:36 ` Balbir Singh
2016-10-25 19:20 ` Aneesh Kumar K.V
2016-10-25 20:01 ` Dave Hansen
2016-10-24 4:31 ` [RFC 6/8] mm: Make VM_CDM marked VMAs non migratable Anshuman Khandual
2016-10-24 4:31 ` [RFC 7/8] mm: Add a new migration function migrate_virtual_range() Anshuman Khandual
2016-10-24 4:31 ` [RFC 8/8] mm: Add N_COHERENT_DEVICE node type into node_states[] Anshuman Khandual
2016-10-25 7:22 ` Balbir Singh
2016-10-26 4:52 ` Anshuman Khandual
2016-10-24 4:42 ` Anshuman Khandual [this message]
2016-10-24 4:42 ` [DEBUG 01/10] dt-bindings: Add doc for ibm,hotplug-aperture Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 02/10] powerpc/mm: Create numa nodes for hotplug memory Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 03/10] powerpc/mm: Allow memory hotplug into a memory less node Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 04/10] mm: Enable CONFIG_MOVABLE_NODE on powerpc Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 05/10] powerpc/mm: Identify isolation seeking coherent memory nodes during boot Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 06/10] mm: Export definition of 'zone_names' array through mmzone.h Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 07/10] mm: Add debugfs interface to dump each node's zonelist information Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 08/10] powerpc: Enable CONFIG_MOVABLE_NODE for PPC64 platform Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 09/10] drivers: Add two drivers for coherent device memory tests Anshuman Khandual
2016-10-24 4:42 ` [DEBUG 10/10] test: Add a script to perform random VMA migrations across nodes Anshuman Khandual
2016-10-24 17:09 ` [RFC 0/8] Define coherent device memory node Jerome Glisse
2016-10-25 4:26 ` Aneesh Kumar K.V
2016-10-25 15:16 ` Jerome Glisse
2016-10-26 11:09 ` Aneesh Kumar K.V
2016-10-26 16:07 ` Jerome Glisse
2016-10-28 5:29 ` Aneesh Kumar K.V
2016-10-28 16:16 ` Jerome Glisse
2016-11-05 5:21 ` Anshuman Khandual
2016-11-05 18:02 ` Jerome Glisse
2016-10-25 4:59 ` Aneesh Kumar K.V
2016-10-25 15:32 ` Jerome Glisse
2016-10-25 17:31 ` Aneesh Kumar K.V
2016-10-25 18:52 ` Jerome Glisse
2016-10-26 11:13 ` Anshuman Khandual
2016-10-26 16:02 ` Jerome Glisse
2016-10-27 4:38 ` Anshuman Khandual
2016-10-27 7:03 ` Anshuman Khandual
2016-10-27 15:05 ` Jerome Glisse
2016-10-28 5:47 ` Anshuman Khandual
2016-10-28 16:08 ` Jerome Glisse
2016-10-26 12:56 ` Anshuman Khandual
2016-10-26 16:28 ` Jerome Glisse
2016-10-27 10:23 ` Balbir Singh
2016-10-25 12:07 ` Balbir Singh
2016-10-25 15:21 ` Jerome Glisse
2016-10-24 18:04 ` Dave Hansen
2016-10-24 18:32 ` David Nellans
2016-10-24 19:36 ` Dave Hansen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1477284149-2976-1-git-send-email-khandual@linux.vnet.ibm.com \
--to=khandual@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=bsingharora@gmail.com \
--cc=js1304@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@suse.com \
--cc=minchan@kernel.org \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).