From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752323AbcKQH7U (ORCPT ); Thu, 17 Nov 2016 02:59:20 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:37288 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750998AbcKQH7T (ORCPT ); Thu, 17 Nov 2016 02:59:19 -0500 From: Anshuman Khandual To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: mhocko@suse.com, js1304@gmail.com, vbabka@suse.cz, mgorman@suse.de, minchan@kernel.org, akpm@linux-foundation.org, aneesh.kumar@linux.vnet.ibm.com, bsingharora@gmail.com Subject: [DRAFT 1/2] mm/cpuset: Exclude CDM nodes from each task's mems_allowed node mask Date: Thu, 17 Nov 2016 13:29:08 +0530 X-Mailer: git-send-email 2.1.0 In-Reply-To: <582D5F02.6010705@linux.vnet.ibm.com> References: <582D5F02.6010705@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16111707-1617-0000-0000-0000017E3E02 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16111707-1618-0000-0000-00004728D6E4 Message-Id: <1479369549-13309-1-git-send-email-khandual@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-11-17_04:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1611170147 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org task->mems_allowed decides the final node mask of nodes from which memory can be allocated irrespective of process or VMA based memory policy. CDM nodes should not be used for any user space memory allocation, hence they should not be part of any mems_allowed mask in user space to begin with. This adds a function system_ram() which computes system RAM only nodes and excludes all the CDM nodes on the platform. This resultant system RAM nodemask is used instead of N_MEMORY mask during cpuset and mems_allowed initialization. This achieves isolation of the coherent device memory from userspace allocations. Signed-off-by: Anshuman Khandual --- This completely isolates CDM nodes from user space allocations. Hence explicit allocation to the CDM nodes would not be possible any more. To again enable explicit allocation capability from user space, cpuset needs to be changed to accommodate CDM nodes into task's mems_allowed. include/linux/mm.h | 9 +++++++++ kernel/cpuset.c | 12 +++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a92c8d7..f338492 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -446,6 +446,15 @@ static inline int put_page_testzero(struct page *page) return page_ref_dec_and_test(page); } +static inline nodemask_t system_ram(void) +{ + nodemask_t ram_nodes; + + nodes_clear(ram_nodes); + nodes_andnot(ram_nodes, node_states[N_MEMORY], node_states[N_COHERENT_DEVICE]); + return ram_nodes; +} + /* * Try to grab a ref unless the page has a refcount of zero, return false if * that is the case. diff --git a/kernel/cpuset.c b/kernel/cpuset.c index 29f815d..78c6fa3 100644 --- a/kernel/cpuset.c +++ b/kernel/cpuset.c @@ -364,9 +364,11 @@ static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) */ static void guarantee_online_mems(struct cpuset *cs, nodemask_t *pmask) { - while (!nodes_intersects(cs->effective_mems, node_states[N_MEMORY])) + nodemask_t nodes = system_ram(); + + while (!nodes_intersects(cs->effective_mems, nodes)) cs = parent_cs(cs); - nodes_and(*pmask, cs->effective_mems, node_states[N_MEMORY]); + nodes_and(*pmask, cs->effective_mems, nodes); } /* @@ -2301,7 +2303,7 @@ static void cpuset_hotplug_workfn(struct work_struct *work) /* fetch the available cpus/mems and find out which changed how */ cpumask_copy(&new_cpus, cpu_active_mask); - new_mems = node_states[N_MEMORY]; + new_mems = system_ram(); cpus_updated = !cpumask_equal(top_cpuset.effective_cpus, &new_cpus); mems_updated = !nodes_equal(top_cpuset.effective_mems, new_mems); @@ -2393,11 +2395,11 @@ static int cpuset_track_online_nodes(struct notifier_block *self, void __init cpuset_init_smp(void) { cpumask_copy(top_cpuset.cpus_allowed, cpu_active_mask); - top_cpuset.mems_allowed = node_states[N_MEMORY]; + top_cpuset.mems_allowed = system_ram(); top_cpuset.old_mems_allowed = top_cpuset.mems_allowed; cpumask_copy(top_cpuset.effective_cpus, cpu_active_mask); - top_cpuset.effective_mems = node_states[N_MEMORY]; + top_cpuset.effective_mems = system_ram(); register_hotmemory_notifier(&cpuset_track_online_nodes_nb); -- 1.8.3.1