From: Anshuman Khandual <khandual@linux.vnet.ibm.com>
To: Dave Hansen <dave.hansen@intel.com>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org
Cc: mhocko@suse.com, vbabka@suse.cz, mgorman@suse.de,
minchan@kernel.org, aneesh.kumar@linux.vnet.ibm.com,
bsingharora@gmail.com, srikar@linux.vnet.ibm.com,
haren@linux.vnet.ibm.com, jglisse@redhat.com
Subject: Re: [RFC 4/4] mm: Ignore cpuset enforcement when allocation flag has __GFP_THISNODE
Date: Wed, 30 Nov 2016 16:47:01 +0530 [thread overview]
Message-ID: <583EB52D.3080307@linux.vnet.ibm.com> (raw)
In-Reply-To: <9a2e3fd7-1955-b347-2447-4b66402c1ce8@intel.com>
On 11/29/2016 10:22 PM, Dave Hansen wrote:
> On 11/28/2016 10:51 PM, Anshuman Khandual wrote:
>> On 11/29/2016 02:42 AM, Dave Hansen wrote:
>>>> On 11/22/2016 06:19 AM, Anshuman Khandual wrote:
>>>>>> --- a/mm/page_alloc.c
>>>>>> +++ b/mm/page_alloc.c
>>>>>> @@ -3715,7 +3715,7 @@ struct page *
>>>>>> .migratetype = gfpflags_to_migratetype(gfp_mask),
>>>>>> };
>>>>>>
>>>>>> - if (cpusets_enabled()) {
>>>>>> + if (cpusets_enabled() && !(alloc_mask & __GFP_THISNODE)) {
>>>>>> alloc_mask |= __GFP_HARDWALL;
>>>>>> alloc_flags |= ALLOC_CPUSET;
>>>>>> if (!ac.nodemask)
>>>>
>>>> This means now that any __GFP_THISNODE allocation can "escape" the
>>>> cpuset. That seems like a pretty major change to how cpusets works. Do
>>>> we know that *ALL* __GFP_THISNODE allocations are truly lacking in a
>>>> cpuset context that can be enforced?
>> Right, I know its a very blunt change. With the cpuset based isolation
>> of coherent device node for the user space tasks leads to a side effect
>> that a driver or even kernel cannot allocate memory from the coherent
> ...
>
> Well, we have __GFP_HARDWALL:
>
> * __GFP_HARDWALL enforces the cpuset memory allocation policy.
>
> which you can clear in the places where you want to do an allocation but
> want to ignore cpusets. But, __cpuset_node_allowed() looks like it gets
> a little funky if you do that since it would probably be falling back to
> the root cpuset that also would not have the new node in mems_allowed.
Right but what is the rationale behind this ? This what is in the in-code
documentation for this function __cpuset_node_allowed().
* GFP_KERNEL - any node in enclosing hardwalled cpuset ok
If the allocation has requested GFP_KERNEL, should not it look for the
entire system for memory ? Does cpuset still has to be enforced ?
>
> What exactly are the kernel-internal places that need to allocate from
> the coherent device node? When would this be done out of the context of
> an application *asking* for memory in the new node?
The primary user right now is a driver who wants to move around mapped
pages of an application from system RAM to CDM nodes and back. If the
application has requested for it though an ioctl(), during migration
the destination pages will be allocated on the CDM *in* the task context.
The driver could also have scheduled migration chunks in the work queue
which can execute later on. IIUC those execution and corresponding
allocation into CDM node will be *out* of context of the task.
Ideally looking for both the scenarios to work which dont right now.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-11-30 11:17 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-11-22 14:19 [RFC 0/4] Define coherent device memory node Anshuman Khandual
2016-11-22 14:19 ` [RFC 1/4] mm: " Anshuman Khandual
2016-11-29 17:57 ` Dave Hansen
2016-11-30 11:46 ` Anshuman Khandual
2016-11-22 14:19 ` [RFC 2/4] mm/cpuset: Exclude coherent device memory nodes from mems_allowed Anshuman Khandual
2016-11-22 14:19 ` [RFC 3/4] mm/hugetlb: Restrict HugeTLB page allocations only to system ram nodemask Anshuman Khandual
2016-11-22 14:19 ` [RFC 4/4] mm: Ignore cpuset enforcement when allocation flag has __GFP_THISNODE Anshuman Khandual
2016-11-28 21:12 ` Dave Hansen
2016-11-29 6:51 ` Anshuman Khandual
2016-11-29 16:52 ` Dave Hansen
2016-11-30 11:17 ` Anshuman Khandual [this message]
2016-11-30 19:43 ` Dave Hansen
2016-11-22 14:19 ` [DEBUG 05/12] powerpc/mm: Identify coherent device memory nodes during platform init Anshuman Khandual
2016-11-22 14:19 ` [DEBUG 06/12] powerpc/mm: Create numa nodes for hotplug memory Anshuman Khandual
2016-11-22 14:19 ` [DEBUG 07/12] powerpc/mm: Allow memory hotplug into a memory less node Anshuman Khandual
2016-11-22 14:19 ` [DEBUG 08/12] mm: Enable CONFIG_MOVABLE_NODE on powerpc Anshuman Khandual
2016-11-22 14:19 ` [DEBUG 09/12] powerpc: Enable CONFIG_MOVABLE_NODE for PPC64 platform Anshuman Khandual
2016-11-22 14:19 ` [DEBUG 10/12] mm: Add a new migration function migrate_virtual_range() Anshuman Khandual
2016-11-22 14:19 ` [DEBUG 11/12] drivers: Add two drivers for coherent device memory tests Anshuman Khandual
2016-11-22 14:19 ` [DEBUG 12/12] test: Add a script to perform random VMA migrations across nodes Anshuman Khandual
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=583EB52D.3080307@linux.vnet.ibm.com \
--to=khandual@linux.vnet.ibm.com \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=bsingharora@gmail.com \
--cc=dave.hansen@intel.com \
--cc=haren@linux.vnet.ibm.com \
--cc=jglisse@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@suse.com \
--cc=minchan@kernel.org \
--cc=srikar@linux.vnet.ibm.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).