* Re: [PATCH 1/6] powerpc/mm: Wire up hpte_removebolted for powernv
From: Balbir Singh @ 2017-05-23 9:27 UTC (permalink / raw)
To: Oliver O'Halloran
Cc: open list:LINUX FOR POWERPC (32-BIT AND 64-BIT), linux-mm,
Anton Blanchard
In-Reply-To: <20170523040524.13717-1-oohall@gmail.com>
On Tue, May 23, 2017 at 2:05 PM, Oliver O'Halloran <oohall@gmail.com> wrote:
> From: Anton Blanchard <anton@samba.org>
>
> Adds support for removing bolted (i.e kernel linear mapping) mappings on
> powernv. This is needed to support memory hot unplug operations which
> are required for the teardown of DAX/PMEM devices.
>
> Reviewed-by: Rashmica Gupta <rashmica.g@gmail.com>
> Signed-off-by: Anton Blanchard <anton@samba.org>
> Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
> ---
> v1 -> v2: Fixed the commit author
> Added VM_WARN_ON() if we attempt to remove an unbolted hpte
> ---
> arch/powerpc/mm/hash_native_64.c | 33 +++++++++++++++++++++++++++++++++
> 1 file changed, 33 insertions(+)
>
> diff --git a/arch/powerpc/mm/hash_native_64.c b/arch/powerpc/mm/hash_native_64.c
> index 65bb8f33b399..b534d041cfe8 100644
> --- a/arch/powerpc/mm/hash_native_64.c
> +++ b/arch/powerpc/mm/hash_native_64.c
> @@ -407,6 +407,38 @@ static void native_hpte_updateboltedpp(unsigned long newpp, unsigned long ea,
> tlbie(vpn, psize, psize, ssize, 0);
> }
>
> +/*
> + * Remove a bolted kernel entry. Memory hotplug uses this.
> + *
> + * No need to lock here because we should be the only user.
> + */
> +static int native_hpte_removebolted(unsigned long ea, int psize, int ssize)
> +{
> + unsigned long vpn;
> + unsigned long vsid;
> + long slot;
> + struct hash_pte *hptep;
> +
> + vsid = get_kernel_vsid(ea, ssize);
> + vpn = hpt_vpn(ea, vsid, ssize);
> +
> + slot = native_hpte_find(vpn, psize, ssize);
> + if (slot == -1)
> + return -ENOENT;
> +
> + hptep = htab_address + slot;
> +
> + VM_WARN_ON(!(be64_to_cpu(hptep->v) & HPTE_V_BOLTED));
> +
> + /* Invalidate the hpte */
> + hptep->v = 0;
> +
> + /* Invalidate the TLB */
> + tlbie(vpn, psize, psize, ssize, 0);
> + return 0;
> +}
> +
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 3/6] powerpc/vmemmap: Add altmap support
From: Balbir Singh @ 2017-05-23 9:25 UTC (permalink / raw)
To: Oliver O'Halloran
Cc: open list:LINUX FOR POWERPC (32-BIT AND 64-BIT), linux-mm
In-Reply-To: <20170523040524.13717-3-oohall@gmail.com>
On Tue, May 23, 2017 at 2:05 PM, Oliver O'Halloran <oohall@gmail.com> wrote:
> Adds support to powerpc for the altmap feature of ZONE_DEVICE memory. An
> altmap is a driver provided region that is used to provide the backing
> storage for the struct pages of ZONE_DEVICE memory. In situations where
> large amount of ZONE_DEVICE memory is being added to the system the
> altmap reduces pressure on main system memory by allowing the mm/
> metadata to be stored on the device itself rather in main memory.
>
> Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
> ---
> arch/powerpc/mm/init_64.c | 15 +++++++++++++--
> arch/powerpc/mm/mem.c | 16 +++++++++++++---
> 2 files changed, 26 insertions(+), 5 deletions(-)
>
> diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
> index 8851e4f5dbab..225fbb8034e6 100644
> --- a/arch/powerpc/mm/init_64.c
> +++ b/arch/powerpc/mm/init_64.c
> @@ -44,6 +44,7 @@
> #include <linux/slab.h>
> #include <linux/of_fdt.h>
> #include <linux/libfdt.h>
> +#include <linux/memremap.h>
>
> #include <asm/pgalloc.h>
> #include <asm/page.h>
> @@ -171,13 +172,17 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node)
> pr_debug("vmemmap_populate %lx..%lx, node %d\n", start, end, node);
>
> for (; start < end; start += page_size) {
> + struct vmem_altmap *altmap;
> void *p;
> int rc;
>
> if (vmemmap_populated(start, page_size))
> continue;
>
> - p = vmemmap_alloc_block(page_size, node);
> + /* altmap lookups only work at section boundaries */
> + altmap = to_vmem_altmap(SECTION_ALIGN_DOWN(start));
> +
> + p = __vmemmap_alloc_block_buf(page_size, node, altmap);
> if (!p)
> return -ENOMEM;
>
> @@ -242,6 +247,8 @@ void __ref vmemmap_free(unsigned long start, unsigned long end)
>
> for (; start < end; start += page_size) {
> unsigned long nr_pages, addr;
> + struct vmem_altmap *altmap;
> + struct page *section_base;
> struct page *page;
>
> /*
> @@ -257,9 +264,13 @@ void __ref vmemmap_free(unsigned long start, unsigned long end)
> continue;
>
> page = pfn_to_page(addr >> PAGE_SHIFT);
> + section_base = pfn_to_page(vmemmap_section_start(start));
> nr_pages = 1 << page_order;
>
> - if (PageReserved(page)) {
> + altmap = to_vmem_altmap((unsigned long) section_base);
> + if (altmap) {
> + vmem_altmap_free(altmap, nr_pages);
> + } else if (PageReserved(page)) {
> /* allocated from bootmem */
> if (page_size < PAGE_SIZE) {
> /*
> diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
> index 9ee536ec0739..2c0c16f11eee 100644
> --- a/arch/powerpc/mm/mem.c
> +++ b/arch/powerpc/mm/mem.c
> @@ -36,6 +36,7 @@
> #include <linux/hugetlb.h>
> #include <linux/slab.h>
> #include <linux/vmalloc.h>
> +#include <linux/memremap.h>
>
> #include <asm/pgalloc.h>
> #include <asm/prom.h>
> @@ -159,11 +160,20 @@ int arch_remove_memory(u64 start, u64 size)
> {
> unsigned long start_pfn = start >> PAGE_SHIFT;
> unsigned long nr_pages = size >> PAGE_SHIFT;
> - struct zone *zone;
> + struct vmem_altmap *altmap;
> + struct page *page;
> int ret;
>
> - zone = page_zone(pfn_to_page(start_pfn));
> - ret = __remove_pages(zone, start_pfn, nr_pages);
> + /*
> + * If we have an altmap then we need to skip over any reserved PFNs
> + * when querying the zone.
> + */
> + page = pfn_to_page(start_pfn);
> + altmap = to_vmem_altmap((unsigned long) page);
> + if (altmap)
> + page += vmem_altmap_offset(altmap);
> +
> + ret = __remove_pages(page_zone(page), start_pfn, nr_pages);
> if (ret)
> return ret;
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
Balbir
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: mm, something wring in page_lock_anon_vma_read()?
From: zhong jiang @ 2017-05-23 9:21 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Hugh Dickins, Xishi Qiu, Andrew Morton, Tejun Heo, Michal Hocko,
Johannes Weiner, Mel Gorman, Michal Hocko, Minchan Kim,
David Rientjes, Joonsoo Kim, aarcange, sumeet.keswani,
Rik van Riel, Linux MM, LKML
In-Reply-To: <a94c202d-7d9f-0a62-3049-9f825a1db50d@suse.cz>
On 2017/5/23 0:51, Vlastimil Babka wrote:
> On 05/20/2017 05:01 AM, zhong jiang wrote:
>> On 2017/5/20 10:40, Hugh Dickins wrote:
>>> On Sat, 20 May 2017, Xishi Qiu wrote:
>>>> Here is a bug report form redhat: https://bugzilla.redhat.com/show_bug.cgi?id=1305620
>>>> And I meet the bug too. However it is hard to reproduce, and
>>>> 624483f3ea82598("mm: rmap: fix use-after-free in __put_anon_vma") is not help.
>>>>
>>>> From the vmcore, it seems that the page is still mapped(_mapcount=0 and _count=2),
>>>> and the value of mapping is a valid address(mapping = 0xffff8801b3e2a101),
>>>> but anon_vma has been corrupted.
>>>>
>>>> Any ideas?
>>> Sorry, no. I assume that _mapcount has been misaccounted, for example
>>> a pte mapped in on top of another pte; but cannot begin tell you where
>>> in Red Hat's kernel-3.10.0-229.4.2.el7 that might happen.
>>>
>>> Hugh
>>>
>>> .
>>>
>> Hi, Hugh
>>
>> I find the following message from the dmesg.
>>
>> [26068.316592] BUG: Bad rss-counter state mm:ffff8800a7de2d80 idx:1 val:1
>>
>> I can prove that the __mapcount is misaccount. when task is exited. the rmap
>> still exist.
> Check if the kernel in question contains this commit: ad33bb04b2a6 ("mm:
> thp: fix SMP race condition between THP page fault and MADV_DONTNEED")
HI, Vlastimil
I miss the patch. when I read the patch. I find the following issue. but I am sure it is right.
if (unlikely(pmd_trans_unstable(pmd)))
return 0;
/*
* A regular pmd is established and it can't morph into a huge pmd
* from under us anymore at this point because we hold the mmap_sem
* read mode and khugepaged takes it in write mode. So now it's
* safe to run pte_offset_map().
*/
pte = pte_offset_map(pmd, address);
after pmd_trans_unstable call, without any protect method. by the comments,
it think the pte_offset_map is safe. before pte_offset_map call, it still may be
unstable. it is possible?
Thanks
zhongjiang
>> Thanks
>> zhongjiang
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org. For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>
>
> .
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 6/6] powerpc/mm: Enable ZONE_DEVICE on powerpc
From: Balbir Singh @ 2017-05-23 9:21 UTC (permalink / raw)
To: Oliver O'Halloran
Cc: open list:LINUX FOR POWERPC (32-BIT AND 64-BIT), linux-mm
In-Reply-To: <20170523040524.13717-6-oohall@gmail.com>
On Tue, May 23, 2017 at 2:05 PM, Oliver O'Halloran <oohall@gmail.com> wrote:
> Flip the switch. Running around and screaming "IT'S ALIVE" is optional,
> but recommended.
>
> Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
> ---
> arch/powerpc/Kconfig | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
> index f7c8f9972f61..bf3365c34244 100644
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -138,6 +138,7 @@ config PPC
> select ARCH_HAS_SG_CHAIN
> select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
> select ARCH_HAS_UBSAN_SANITIZE_ALL
> + select ARCH_HAS_ZONE_DEVICE if PPC64
Does this work for Book E as well?
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 5/6] mm, x86: Add ARCH_HAS_ZONE_DEVICE
From: Balbir Singh @ 2017-05-23 9:20 UTC (permalink / raw)
To: Oliver O'Halloran
Cc: open list:LINUX FOR POWERPC (32-BIT AND 64-BIT), linux-mm,
maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)
In-Reply-To: <20170523040524.13717-5-oohall@gmail.com>
On Tue, May 23, 2017 at 2:05 PM, Oliver O'Halloran <oohall@gmail.com> wrote:
> Currently ZONE_DEVICE depends on X86_64. This is fine for now, but it
> will get unwieldly as new platforms get ZONE_DEVICE support. Moving it
> to an arch selected Kconfig option to save us some trouble in the
> future.
>
> Cc: x86@kernel.org
> Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
> ---
> arch/x86/Kconfig | 1 +
> mm/Kconfig | 5 ++++-
> 2 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index cd18994a9555..acbb15234562 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -59,6 +59,7 @@ config X86
> select ARCH_HAS_STRICT_KERNEL_RWX
> select ARCH_HAS_STRICT_MODULE_RWX
> select ARCH_HAS_UBSAN_SANITIZE_ALL
> + select ARCH_HAS_ZONE_DEVICE if X86_64
> select ARCH_HAVE_NMI_SAFE_CMPXCHG
> select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI
> select ARCH_MIGHT_HAVE_PC_PARPORT
> diff --git a/mm/Kconfig b/mm/Kconfig
> index beb7a455915d..2d38a4abe957 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -683,12 +683,15 @@ config IDLE_PAGE_TRACKING
>
> See Documentation/vm/idle_page_tracking.txt for more details.
>
> +config ARCH_HAS_ZONE_DEVICE
> + def_bool n
> +
> config ZONE_DEVICE
> bool "Device memory (pmem, etc...) hotplug support"
> depends on MEMORY_HOTPLUG
> depends on MEMORY_HOTREMOVE
> depends on SPARSEMEM_VMEMMAP
> - depends on X86_64 #arch_add_memory() comprehends device memory
> + depends on ARCH_HAS_ZONE_DEVICE
>
> help
> Device memory hotplug support allows for establishing pmem,
Acked-by: Balbir Singh <bsingharora@gmail.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: Define KB, MB, GB, TB in core VM
From: Christoph Hellwig @ 2017-05-23 8:40 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Christoph Hellwig, Andrew Morton, Anshuman Khandual, linux-mm,
linux-kernel
In-Reply-To: <09a6bafa-5743-425e-8def-bd9219cd756c@suse.cz>
On Tue, May 23, 2017 at 10:38:17AM +0200, Vlastimil Babka wrote:
> Those defined in the patch are binary, not decimal. Do we even need
> decimal ones?
Oh, good point. In which case the names should change to avoid the
confusion.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: Define KB, MB, GB, TB in core VM
From: Vlastimil Babka @ 2017-05-23 8:38 UTC (permalink / raw)
To: Christoph Hellwig, Andrew Morton
Cc: Anshuman Khandual, linux-mm, linux-kernel
In-Reply-To: <20170523070227.GA27864@infradead.org>
On 05/23/2017 09:02 AM, Christoph Hellwig wrote:
> On Mon, May 22, 2017 at 02:11:49PM -0700, Andrew Morton wrote:
>> On Mon, 22 May 2017 16:47:42 +0530 Anshuman Khandual <khandual@linux.vnet.ibm.com> wrote:
>>
>>> There are many places where we define size either left shifting integers
>>> or multiplying 1024s without any generic definition to fall back on. But
>>> there are couples of (powerpc and lz4) attempts to define these standard
>>> memory sizes. Lets move these definitions to core VM to make sure that
>>> all new usage come from these definitions eventually standardizing it
>>> across all places.
>>
>> Grep further - there are many more definitions and some may now
>> generate warnings.
>>
>> Newly including mm.h for these things seems a bit heavyweight. I can't
>> immediately think of a more appropriate place. Maybe printk.h or
>> kernel.h.
>
> IFF we do these kernel.h is the right place. And please also add the
> MiB & co variants for the binary versions right next to the decimal
> ones.
Those defined in the patch are binary, not decimal. Do we even need
decimal ones?
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [HMM 07/15] mm/ZONE_DEVICE: new type of ZONE_DEVICE for unaddressable memory v2
From: kbuild test robot @ 2017-05-23 8:36 UTC (permalink / raw)
To: Jérôme Glisse
Cc: kbuild-all, akpm, linux-kernel, linux-mm, John Hubbard,
David Nellans, Dan Williams, Ross Zwisler
In-Reply-To: <20170522165206.6284-8-jglisse@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 2100 bytes --]
Hi Jerome,
[auto build test ERROR on mmotm/master]
[also build test ERROR on next-20170523]
[cannot apply to v4.12-rc2]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/J-r-me-Glisse/HMM-Heterogeneous-Memory-Management-v22/20170523-153623
base: git://git.cmpxchg.org/linux-mmotm.git master
config: x86_64-acpi-redef (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64
All errors (new ones prefixed by >>):
In file included from include/linux/suspend.h:8:0,
from arch/x86/kernel/asm-offsets.c:12:
include/linux/mm.h: In function 'is_device_private_page':
>> include/linux/mm.h:796:15: error: dereferencing pointer to incomplete type 'struct dev_pagemap'
(page->pgmap->type == MEMORY_DEVICE_PRIVATE));
^~
>> include/linux/mm.h:796:25: error: 'MEMORY_DEVICE_PRIVATE' undeclared (first use in this function)
(page->pgmap->type == MEMORY_DEVICE_PRIVATE));
^~~~~~~~~~~~~~~~~~~~~
include/linux/mm.h:796:25: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [arch/x86/kernel/asm-offsets.s] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [sub-make] Error 2
vim +796 include/linux/mm.h
790 }
791
792 static inline bool is_device_private_page(const struct page *page)
793 {
794 /* See MEMORY_DEVICE_PRIVATE in include/linux/memory_hotplug.h */
795 return ((page_zonenum(page) == ZONE_DEVICE) &&
> 796 (page->pgmap->type == MEMORY_DEVICE_PRIVATE));
797 }
798 #else
799 static inline bool is_zone_device_page(const struct page *page)
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31755 bytes --]
^ permalink raw reply
* Re: [RFC PATCH 0/4 v2] mm: give __GFP_REPEAT a better semantic
From: Vlastimil Babka @ 2017-05-23 8:12 UTC (permalink / raw)
To: Michal Hocko, linux-mm
Cc: Johannes Weiner, Mel Gorman, Andrew Morton, LKML, Darrick J. Wong,
Heiko Carstens, NeilBrown, Jonathan Corbet, Paolo Bonzini,
Eric W. Biederman
In-Reply-To: <20170516091022.GD2481@dhcp22.suse.cz>
On 05/16/2017 11:10 AM, Michal Hocko wrote:
> So, is there some interest in this? I am not going to push this if there
> is a general consensus that we do not need to do anything about the
> current situation or need a different approach.
After the recent LWN article [1] I think that we should really support
marking allocations as failable, without making them too easily failable
via __GFP_NORETRY. The __GFP_RETRY_MAY_FAIL flag sounds like a good way
to do that without introducing a new __GFP_MAYFAIL. We could also
introduce a wrapper such as GFP_KERNEL_MAYFAIL.
[1] https://lwn.net/Articles/723317/
> On Tue 07-03-17 16:48:39, Michal Hocko wrote:
>> Hi,
>> this is a follow up for __GFP_REPEAT clean up merged in 4.7. The previous
>> version of this patch series was posted as an RFC
>> http://lkml.keprnel.org/r/1465212736-14637-1-git-send-email-mhocko@kernel.org
>> Since then I have reconsidered the semantic and made it a counterpart
>> to the __GFP_NORETRY and made it the other extreme end of the retry
>> logic. Both are not invoking the OOM killer so they are suitable
>> for allocation paths with a fallback. Also a new potential user has
>> emerged (kvmalloc - see patch 4). I have also renamed the flag from
>> __GFP_RETRY_HARD to __GFP_RETRY_MAY_FAIL as this should be more clear.
>>
>> I have kept the RFC status because of the semantic change. The patch 1
>> is an exception because it should be merge regardless of the rest.
>>
>> The main motivation for the change is that the current implementation of
>> __GFP_REPEAT is not very much useful.
>>
>> The documentation says:
>> * __GFP_REPEAT: Try hard to allocate the memory, but the allocation attempt
>> * _might_ fail. This depends upon the particular VM implementation.
>>
>> It just fails to mention that this is true only for large (costly) high
>> order which has been the case since the flag was introduced. A similar
>> semantic would be really helpful for smal orders as well, though,
>> because we have places where a failure with a specific fallback error
>> handling is preferred to a potential endless loop inside the page
>> allocator.
>>
>> The earlier cleanup dropped __GFP_REPEAT usage for low (!costly) order
>> users so only those which might use larger orders have stayed. One user
>> which slipped through cracks is addressed in patch 1.
>>
>> Let's rename the flag to something more verbose and use it for existing
>> users. Semantic for those will not change. Then implement low (!costly)
>> orders failure path which is hit after the page allocator is about to
>> invoke the oom killer. Now we have a good counterpart for __GFP_NORETRY
>> and finally can tell try as hard as possible without the OOM killer.
>>
>> Xfs code already has an existing annotation for allocations which are
>> allowed to fail and we can trivially map them to the new gfp flag
>> because it will provide the semantic KM_MAYFAIL wants.
>>
>> kvmalloc will allow also !costly high order allocations to retry hard
>> before falling back to the vmalloc.
>>
>> The patchset is based on the current linux-next.
>>
>> Shortlog
>> Michal Hocko (4):
>> s390: get rid of superfluous __GFP_REPEAT
>> mm, tree wide: replace __GFP_REPEAT by __GFP_RETRY_MAYFAIL with more useful semantic
>> xfs: map KM_MAYFAIL to __GFP_RETRY_MAYFAIL
>> mm: kvmalloc support __GFP_RETRY_MAYFAIL for all sizes
>>
>> Diffstat
>> Documentation/DMA-ISA-LPC.txt | 2 +-
>> arch/powerpc/include/asm/book3s/64/pgalloc.h | 2 +-
>> arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +-
>> arch/s390/mm/pgalloc.c | 2 +-
>> drivers/mmc/host/wbsd.c | 2 +-
>> drivers/s390/char/vmcp.c | 2 +-
>> drivers/target/target_core_transport.c | 2 +-
>> drivers/vhost/net.c | 2 +-
>> drivers/vhost/scsi.c | 2 +-
>> drivers/vhost/vsock.c | 2 +-
>> fs/btrfs/check-integrity.c | 2 +-
>> fs/btrfs/raid56.c | 2 +-
>> fs/xfs/kmem.h | 10 +++++++++
>> include/linux/gfp.h | 32 +++++++++++++++++++---------
>> include/linux/slab.h | 3 ++-
>> include/trace/events/mmflags.h | 2 +-
>> mm/hugetlb.c | 4 ++--
>> mm/internal.h | 2 +-
>> mm/page_alloc.c | 14 +++++++++---
>> mm/sparse-vmemmap.c | 4 ++--
>> mm/util.c | 14 ++++--------
>> mm/vmalloc.c | 2 +-
>> mm/vmscan.c | 8 +++----
>> net/core/dev.c | 6 +++---
>> net/core/skbuff.c | 2 +-
>> net/sched/sch_fq.c | 2 +-
>> tools/perf/builtin-kmem.c | 2 +-
>> 27 files changed, 78 insertions(+), 53 deletions(-)
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org. For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm/oom_kill: count global and memory cgroup oom kills
From: David Rientjes @ 2017-05-23 7:49 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: Roman Guschin, linux-mm, Andrew Morton, Tejun Heo, cgroups,
linux-kernel, Vlastimil Babka, Michal Hocko, hannes
In-Reply-To: <ecd4a7ea-06c0-f549-a1bf-6d2d3c0af719@yandex-team.ru>
On Mon, 22 May 2017, Konstantin Khlebnikov wrote:
> Nope, they are different. I think we should rephase documentation somehow
>
> low - count of reclaims below low level
> high - count of post-allocation reclaims above high level
> max - count of direct reclaims
> oom - count of failed direct reclaims
> oom_kill - count of oom killer invocations and killed processes
>
In our kernel, we've maintained counts of oom kills per memcg for years as
part of memory.oom_control for memcg v1, but we've also found it helpful
to complement that with another count that specifies the number of
processes oom killed that were attached to that exact memcg.
In your patch, oom_kill in memory.oom_control specifies that number of oom
events that resulted in an oom kill of a process from that hierarchy, but
not the number of processes killed from a specific memcg (the difference
between oc->memcg and mem_cgroup_from_task(victim)). Not sure if you
would also find it helpful.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] ib/core: not to set page dirty bit if it's already set.
From: Christoph Hellwig @ 2017-05-23 7:42 UTC (permalink / raw)
To: Qing Huang
Cc: Christoph Hellwig, linux-rdma, linux-kernel, dledford, sean.hefty,
artemyko, linux-mm
In-Reply-To: <9f4a4f90-a7b1-b1dc-6e7a-042f26254681@oracle.com>
On Mon, May 22, 2017 at 04:43:57PM -0700, Qing Huang wrote:
>
> On 5/19/2017 6:05 AM, Christoph Hellwig wrote:
> > On Thu, May 18, 2017 at 04:33:53PM -0700, Qing Huang wrote:
> > > This change will optimize kernel memory deregistration operations.
> > > __ib_umem_release() used to call set_page_dirty_lock() against every
> > > writable page in its memory region. Its purpose is to keep data
> > > synced between CPU and DMA device when swapping happens after mem
> > > deregistration ops. Now we choose not to set page dirty bit if it's
> > > already set by kernel prior to calling __ib_umem_release(). This
> > > reduces memory deregistration time by half or even more when we ran
> > > application simulation test program.
> > As far as I can tell this code doesn't even need set_page_dirty_lock
> > and could just use set_page_dirty
>
> It seems that set_page_dirty_lock has been used here for more than 10 years.
> Don't know the original purpose. Maybe it was used to prevent races between
> setting dirty bits and swapping out pages?
I suspect copy & paste. Or maybe I don't actually understand the
explanation of set_page_dirty vs set_page_dirty_lock enough. But
I'd rather not hack around the problem.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm/oom_kill: count global and memory cgroup oom kills
From: Michal Hocko @ 2017-05-23 7:27 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: linux-mm, Andrew Morton, Tejun Heo, cgroups, linux-kernel,
Vlastimil Babka
In-Reply-To: <149520375057.74196.2843113275800730971.stgit@buzz>
On Fri 19-05-17 17:22:30, Konstantin Khlebnikov wrote:
> Show count of global oom killer invocations in /proc/vmstat and
> count of oom kills inside memory cgroup in knob "memory.events"
> (in memory.oom_control for v1 cgroup).
>
> Also describe difference between "oom" and "oom_kill" in memory
> cgroup documentation. Currently oom in memory cgroup kills tasks
> iff shortage has happened inside page fault.
>
> These counters helps in monitoring oom kills - for now
> the only way is grepping for magic words in kernel log.
Have you considered adding memcg's oom alternative for the global case
as well. It would be useful to see how many times we hit the OOM
condition without killing anything. That could help debugging issues
when the OOM killer cannot be invoked (e.g. GFP_NO{FS,IO} contextx)
and the system cannot get out of the oom situation.
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> Documentation/cgroup-v2.txt | 12 +++++++++++-
> include/linux/memcontrol.h | 1 +
> include/linux/vm_event_item.h | 1 +
> mm/memcontrol.c | 2 ++
> mm/oom_kill.c | 6 ++++++
> mm/vmstat.c | 1 +
> 6 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/cgroup-v2.txt b/Documentation/cgroup-v2.txt
> index dc5e2dcdbef4..a742008d76aa 100644
> --- a/Documentation/cgroup-v2.txt
> +++ b/Documentation/cgroup-v2.txt
> @@ -830,9 +830,19 @@ PAGE_SIZE multiple when read back.
>
> oom
>
> + The number of time the cgroup's memory usage was
> + reached the limit and allocation was about to fail.
> + Result could be oom kill, -ENOMEM from any syscall or
> + completely ignored in cases like disk readahead.
> + For now oom in memory cgroup kills tasks iff shortage
> + has happened inside page fault.
> +
> + oom_kill
> +
> The number of times the OOM killer has been invoked in
> the cgroup. This may not exactly match the number of
> - processes killed but should generally be close.
> + processes killed but should generally be close: each
> + invocation could kill several processes at once.
>
> memory.stat
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 899949bbb2f9..2cdcebb78b58 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -55,6 +55,7 @@ enum memcg_event_item {
> MEMCG_HIGH,
> MEMCG_MAX,
> MEMCG_OOM,
> + MEMCG_OOM_KILL,
> MEMCG_NR_EVENTS,
> };
>
> diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
> index d84ae90ccd5c..1707e0a7d943 100644
> --- a/include/linux/vm_event_item.h
> +++ b/include/linux/vm_event_item.h
> @@ -41,6 +41,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
> KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
> PAGEOUTRUN, PGROTATED,
> DROP_PAGECACHE, DROP_SLAB,
> + OOM_KILL,
> #ifdef CONFIG_NUMA_BALANCING
> NUMA_PTE_UPDATES,
> NUMA_HUGE_PTE_UPDATES,
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 94172089f52f..416024837b81 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -3574,6 +3574,7 @@ static int mem_cgroup_oom_control_read(struct seq_file *sf, void *v)
>
> seq_printf(sf, "oom_kill_disable %d\n", memcg->oom_kill_disable);
> seq_printf(sf, "under_oom %d\n", (bool)memcg->under_oom);
> + seq_printf(sf, "oom_kill %lu\n", memcg_sum_events(memcg, MEMCG_OOM_KILL));
> return 0;
> }
>
> @@ -5165,6 +5166,7 @@ static int memory_events_show(struct seq_file *m, void *v)
> seq_printf(m, "high %lu\n", memcg_sum_events(memcg, MEMCG_HIGH));
> seq_printf(m, "max %lu\n", memcg_sum_events(memcg, MEMCG_MAX));
> seq_printf(m, "oom %lu\n", memcg_sum_events(memcg, MEMCG_OOM));
> + seq_printf(m, "oom_kill %lu\n", memcg_sum_events(memcg, MEMCG_OOM_KILL));
>
> return 0;
> }
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 04c9143a8625..c50bff3c3409 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -873,6 +873,12 @@ static void oom_kill_process(struct oom_control *oc, const char *message)
> victim = p;
> }
>
> + /* Raise event before sending signal: reaper must see this */
> + if (!is_memcg_oom(oc))
> + count_vm_event(OOM_KILL);
> + else
> + mem_cgroup_event(oc->memcg, MEMCG_OOM_KILL);
> +
> /* Get a reference to safely compare mm after task_unlock(victim) */
> mm = victim->mm;
> mmgrab(mm);
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 76f73670200a..fe80b81a86e0 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -1018,6 +1018,7 @@ const char * const vmstat_text[] = {
>
> "drop_pagecache",
> "drop_slab",
> + "oom_kill",
>
> #ifdef CONFIG_NUMA_BALANCING
> "numa_pte_updates",
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: Define KB, MB, GB, TB in core VM
From: kbuild test robot @ 2017-05-23 7:24 UTC (permalink / raw)
To: Anshuman Khandual; +Cc: kbuild-all, linux-mm, linux-kernel, akpm
In-Reply-To: <20170522111742.29433-1-khandual@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 4206 bytes --]
Hi Anshuman,
[auto build test WARNING on linus/master]
[also build test WARNING on v4.12-rc2 next-20170523]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-Define-KB-MB-GB-TB-in-core-VM/20170523-141359
config: x86_64-nfsroot+CONFIG_DEBUG_INFO_REDUCED (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64
All warnings (new ones prefixed by >>):
In file included from drivers/char/agp/backend.c:41:0:
>> drivers/char/agp/agp.h:158:0: warning: "KB" redefined
#define KB(x) ((x) * 1024)
In file included from include/linux/scatterlist.h:7:0,
from include/linux/dmapool.h:14,
from include/linux/pci.h:1281,
from drivers/char/agp/backend.c:31:
include/linux/mm.h:2552:0: note: this is the location of the previous definition
#define KB (1UL << 10)
In file included from drivers/char/agp/backend.c:41:0:
>> drivers/char/agp/agp.h:159:0: warning: "MB" redefined
#define MB(x) (KB (KB (x)))
In file included from include/linux/scatterlist.h:7:0,
from include/linux/dmapool.h:14,
from include/linux/pci.h:1281,
from drivers/char/agp/backend.c:31:
include/linux/mm.h:2553:0: note: this is the location of the previous definition
#define MB (1UL << 20)
In file included from drivers/char/agp/backend.c:41:0:
>> drivers/char/agp/agp.h:160:0: warning: "GB" redefined
#define GB(x) (MB (KB (x)))
In file included from include/linux/scatterlist.h:7:0,
from include/linux/dmapool.h:14,
from include/linux/pci.h:1281,
from drivers/char/agp/backend.c:31:
include/linux/mm.h:2554:0: note: this is the location of the previous definition
#define GB (1UL << 30)
--
>> drivers/gpu/drm/i915/i915_gem_stolen.c:33:0: warning: "KB" redefined
#define KB(x) ((x) * 1024)
In file included from include/linux/scatterlist.h:7:0,
from include/linux/dma-mapping.h:10,
from include/drm/drmP.h:37,
from drivers/gpu/drm/i915/i915_gem_stolen.c:29:
include/linux/mm.h:2552:0: note: this is the location of the previous definition
#define KB (1UL << 10)
>> drivers/gpu/drm/i915/i915_gem_stolen.c:34:0: warning: "MB" redefined
#define MB(x) (KB(x) * 1024)
In file included from include/linux/scatterlist.h:7:0,
from include/linux/dma-mapping.h:10,
from include/drm/drmP.h:37,
from drivers/gpu/drm/i915/i915_gem_stolen.c:29:
include/linux/mm.h:2553:0: note: this is the location of the previous definition
#define MB (1UL << 20)
vim +/KB +158 drivers/char/agp/agp.h
b0825488 Matthew Garrett 2005-07-29 152 u32 apbase_config;
a8c84df9 Keith Packard 2008-07-31 153 /* list of agp_memory mapped to the aperture */
a8c84df9 Keith Packard 2008-07-31 154 struct list_head mapped_list;
a8c84df9 Keith Packard 2008-07-31 155 spinlock_t mapped_lock;
^1da177e Linus Torvalds 2005-04-16 156 };
^1da177e Linus Torvalds 2005-04-16 157
^1da177e Linus Torvalds 2005-04-16 @158 #define KB(x) ((x) * 1024)
^1da177e Linus Torvalds 2005-04-16 @159 #define MB(x) (KB (KB (x)))
^1da177e Linus Torvalds 2005-04-16 @160 #define GB(x) (MB (KB (x)))
^1da177e Linus Torvalds 2005-04-16 161
^1da177e Linus Torvalds 2005-04-16 162 #define A_SIZE_8(x) ((struct aper_size_info_8 *) x)
^1da177e Linus Torvalds 2005-04-16 163 #define A_SIZE_16(x) ((struct aper_size_info_16 *) x)
:::::: The code at line 158 was first introduced by commit
:::::: 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 Linux-2.6.12-rc2
:::::: TO: Linus Torvalds <torvalds@ppc970.osdl.org>
:::::: CC: Linus Torvalds <torvalds@ppc970.osdl.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 28961 bytes --]
^ permalink raw reply
* Re: [PATCH v2 6/6] mm, mempolicy: don't check cpuset seqlock where it doesn't matter
From: Michal Hocko @ 2017-05-23 7:16 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, linux-mm, linux-api, linux-kernel, cgroups,
Li Zefan, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov
In-Reply-To: <20170517081140.30654-7-vbabka@suse.cz>
On Wed 17-05-17 10:11:40, Vlastimil Babka wrote:
> Two wrappers of __alloc_pages_nodemask() are checking task->mems_allowed_seq
> themselves to retry allocation that has raced with a cpuset update. This has
> been shown to be ineffective in preventing premature OOM's which can happen in
> __alloc_pages_slowpath() long before it returns back to the wrappers to detect
> the race at that level. Previous patches have made __alloc_pages_slowpath()
> more robust, so we can now simply remove the seqlock checking in the wrappers
> to prevent further wrong impression that it can actually help.
>
> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> mm/mempolicy.c | 16 ----------------
> 1 file changed, 16 deletions(-)
>
> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index 047181452040..7d8e56214ac0 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -1898,12 +1898,9 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
> struct mempolicy *pol;
> struct page *page;
> int preferred_nid;
> - unsigned int cpuset_mems_cookie;
> nodemask_t *nmask;
>
> -retry_cpuset:
> pol = get_vma_policy(vma, addr);
> - cpuset_mems_cookie = read_mems_allowed_begin();
>
> if (pol->mode == MPOL_INTERLEAVE) {
> unsigned nid;
> @@ -1945,8 +1942,6 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
> page = __alloc_pages_nodemask(gfp, order, preferred_nid, nmask);
> mpol_cond_put(pol);
> out:
> - if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
> - goto retry_cpuset;
> return page;
> }
>
> @@ -1964,23 +1959,15 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
> * Allocate a page from the kernel page pool. When not in
> * interrupt context and apply the current process NUMA policy.
> * Returns NULL when no page can be allocated.
> - *
> - * Don't call cpuset_update_task_memory_state() unless
> - * 1) it's ok to take cpuset_sem (can WAIT), and
> - * 2) allocating for current task (not interrupt).
> */
> struct page *alloc_pages_current(gfp_t gfp, unsigned order)
> {
> struct mempolicy *pol = &default_policy;
> struct page *page;
> - unsigned int cpuset_mems_cookie;
>
> if (!in_interrupt() && !(gfp & __GFP_THISNODE))
> pol = get_task_policy(current);
>
> -retry_cpuset:
> - cpuset_mems_cookie = read_mems_allowed_begin();
> -
> /*
> * No reference counting needed for current->mempolicy
> * nor system default_policy
> @@ -1992,9 +1979,6 @@ struct page *alloc_pages_current(gfp_t gfp, unsigned order)
> policy_node(gfp, pol, numa_node_id()),
> policy_nodemask(gfp, pol));
>
> - if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
> - goto retry_cpuset;
> -
> return page;
> }
> EXPORT_SYMBOL(alloc_pages_current);
> --
> 2.12.2
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 5/6] mm, cpuset: always use seqlock when changing task's nodemask
From: Michal Hocko @ 2017-05-23 7:15 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, linux-mm, linux-api, linux-kernel, cgroups,
Li Zefan, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov
In-Reply-To: <20170517081140.30654-6-vbabka@suse.cz>
On Wed 17-05-17 10:11:39, Vlastimil Babka wrote:
> When updating task's mems_allowed and rebinding its mempolicy due to cpuset's
> mems being changed, we currently only take the seqlock for writing when either
> the task has a mempolicy, or the new mems has no intersection with the old
> mems. This should be enough to prevent a parallel allocation seeing no
> available nodes, but the optimization is IMHO unnecessary (cpuset updates
> should not be frequent), and we still potentially risk issues if the
> intersection of new and old nodes has limited amount of free/reclaimable
> memory. Let's just use the seqlock for all tasks.
Agreed
> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> kernel/cgroup/cpuset.c | 29 ++++++++---------------------
> 1 file changed, 8 insertions(+), 21 deletions(-)
>
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index dfd5b420452d..26a1c360a481 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1038,38 +1038,25 @@ static void cpuset_post_attach(void)
> * @tsk: the task to change
> * @newmems: new nodes that the task will be set
> *
> - * In order to avoid seeing no nodes if the old and new nodes are disjoint,
> - * we structure updates as setting all new allowed nodes, then clearing newly
> - * disallowed ones.
> + * We use the mems_allowed_seq seqlock to safely update both tsk->mems_allowed
> + * and rebind an eventual tasks' mempolicy. If the task is allocating in
> + * parallel, it might temporarily see an empty intersection, which results in
> + * a seqlock check and retry before OOM or allocation failure.
> */
> static void cpuset_change_task_nodemask(struct task_struct *tsk,
> nodemask_t *newmems)
> {
> - bool need_loop;
> -
> task_lock(tsk);
> - /*
> - * Determine if a loop is necessary if another thread is doing
> - * read_mems_allowed_begin(). If at least one node remains unchanged and
> - * tsk does not have a mempolicy, then an empty nodemask will not be
> - * possible when mems_allowed is larger than a word.
> - */
> - need_loop = task_has_mempolicy(tsk) ||
> - !nodes_intersects(*newmems, tsk->mems_allowed);
>
> - if (need_loop) {
> - local_irq_disable();
> - write_seqcount_begin(&tsk->mems_allowed_seq);
> - }
> + local_irq_disable();
> + write_seqcount_begin(&tsk->mems_allowed_seq);
>
> nodes_or(tsk->mems_allowed, tsk->mems_allowed, *newmems);
> mpol_rebind_task(tsk, newmems);
> tsk->mems_allowed = *newmems;
>
> - if (need_loop) {
> - write_seqcount_end(&tsk->mems_allowed_seq);
> - local_irq_enable();
> - }
> + write_seqcount_end(&tsk->mems_allowed_seq);
> + local_irq_enable();
>
> task_unlock(tsk);
> }
> --
> 2.12.2
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 4/6] mm, mempolicy: simplify rebinding mempolicies when updating cpusets
From: Michal Hocko @ 2017-05-23 7:11 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, linux-mm, linux-api, linux-kernel, cgroups,
Li Zefan, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov
In-Reply-To: <20170517081140.30654-5-vbabka@suse.cz>
On Wed 17-05-17 10:11:38, Vlastimil Babka wrote:
> Commit c0ff7453bb5c ("cpuset,mm: fix no node to alloc memory when changing
> cpuset's mems") has introduced a two-step protocol when rebinding task's
> mempolicy due to cpuset update, in order to avoid a parallel allocation seeing
> an empty effective nodemask and failing. Later, commit cc9a6c877661 ("cpuset:
> mm: reduce large amounts of memory barrier related damage v3") introduced
> a seqlock protection and removed the synchronization point between the two
> update steps. At that point (or perhaps later), the two-step rebinding became
> unnecessary. Currently it only makes sure that the update first adds new nodes
> in step 1 and then removes nodes in step 2. Without memory barriers the effects
> are questionable, and even then this cannot prevent a parallel zonelist
> iteration checking the nodemask at each step to observe all nodes as unusable
> for allocation. We now fully rely on the seqlock to prevent premature OOMs and
> allocation failures.
>
> We can thus remove the two-step update parts and simplify the code.
>
> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
\o/ This code has been just piling up more complicated code on top of
the code which shouldn't have existed in the first place so I am very
happy to see it go. I hope we can go without rebinding altogether
longterm. Chaging data under feets just asks for problems and this is a
nice example of where it goes.
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> include/linux/mempolicy.h | 6 +--
> include/uapi/linux/mempolicy.h | 8 ----
> kernel/cgroup/cpuset.c | 4 +-
> mm/mempolicy.c | 102 ++++++++---------------------------------
> 4 files changed, 21 insertions(+), 99 deletions(-)
>
> diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h
> index ecb6cbeede5a..3a58b4be1b0c 100644
> --- a/include/linux/mempolicy.h
> +++ b/include/linux/mempolicy.h
> @@ -142,8 +142,7 @@ bool vma_policy_mof(struct vm_area_struct *vma);
>
> extern void numa_default_policy(void);
> extern void numa_policy_init(void);
> -extern void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new,
> - enum mpol_rebind_step step);
> +extern void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new);
> extern void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new);
>
> extern int huge_node(struct vm_area_struct *vma,
> @@ -260,8 +259,7 @@ static inline void numa_default_policy(void)
> }
>
> static inline void mpol_rebind_task(struct task_struct *tsk,
> - const nodemask_t *new,
> - enum mpol_rebind_step step)
> + const nodemask_t *new)
> {
> }
>
> diff --git a/include/uapi/linux/mempolicy.h b/include/uapi/linux/mempolicy.h
> index 9cd8b21dddbe..2a4d89508fec 100644
> --- a/include/uapi/linux/mempolicy.h
> +++ b/include/uapi/linux/mempolicy.h
> @@ -24,13 +24,6 @@ enum {
> MPOL_MAX, /* always last member of enum */
> };
>
> -enum mpol_rebind_step {
> - MPOL_REBIND_ONCE, /* do rebind work at once(not by two step) */
> - MPOL_REBIND_STEP1, /* first step(set all the newly nodes) */
> - MPOL_REBIND_STEP2, /* second step(clean all the disallowed nodes)*/
> - MPOL_REBIND_NSTEP,
> -};
> -
> /* Flags for set_mempolicy */
> #define MPOL_F_STATIC_NODES (1 << 15)
> #define MPOL_F_RELATIVE_NODES (1 << 14)
> @@ -65,7 +58,6 @@ enum mpol_rebind_step {
> */
> #define MPOL_F_SHARED (1 << 0) /* identify shared policies */
> #define MPOL_F_LOCAL (1 << 1) /* preferred local allocation */
> -#define MPOL_F_REBINDING (1 << 2) /* identify policies in rebinding */
> #define MPOL_F_MOF (1 << 3) /* this policy wants migrate on fault */
> #define MPOL_F_MORON (1 << 4) /* Migrate On protnone Reference On Node */
>
> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 0f41292be0fb..dfd5b420452d 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c
> @@ -1063,9 +1063,7 @@ static void cpuset_change_task_nodemask(struct task_struct *tsk,
> }
>
> nodes_or(tsk->mems_allowed, tsk->mems_allowed, *newmems);
> - mpol_rebind_task(tsk, newmems, MPOL_REBIND_STEP1);
> -
> - mpol_rebind_task(tsk, newmems, MPOL_REBIND_STEP2);
> + mpol_rebind_task(tsk, newmems);
> tsk->mems_allowed = *newmems;
>
> if (need_loop) {
> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> index c60807625fd5..047181452040 100644
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -146,22 +146,7 @@ struct mempolicy *get_task_policy(struct task_struct *p)
>
> static const struct mempolicy_operations {
> int (*create)(struct mempolicy *pol, const nodemask_t *nodes);
> - /*
> - * If read-side task has no lock to protect task->mempolicy, write-side
> - * task will rebind the task->mempolicy by two step. The first step is
> - * setting all the newly nodes, and the second step is cleaning all the
> - * disallowed nodes. In this way, we can avoid finding no node to alloc
> - * page.
> - * If we have a lock to protect task->mempolicy in read-side, we do
> - * rebind directly.
> - *
> - * step:
> - * MPOL_REBIND_ONCE - do rebind work at once
> - * MPOL_REBIND_STEP1 - set all the newly nodes
> - * MPOL_REBIND_STEP2 - clean all the disallowed nodes
> - */
> - void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes,
> - enum mpol_rebind_step step);
> + void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes);
> } mpol_ops[MPOL_MAX];
>
> static inline int mpol_store_user_nodemask(const struct mempolicy *pol)
> @@ -304,19 +289,11 @@ void __mpol_put(struct mempolicy *p)
> kmem_cache_free(policy_cache, p);
> }
>
> -static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes,
> - enum mpol_rebind_step step)
> +static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes)
> {
> }
>
> -/*
> - * step:
> - * MPOL_REBIND_ONCE - do rebind work at once
> - * MPOL_REBIND_STEP1 - set all the newly nodes
> - * MPOL_REBIND_STEP2 - clean all the disallowed nodes
> - */
> -static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
> - enum mpol_rebind_step step)
> +static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes)
> {
> nodemask_t tmp;
>
> @@ -325,35 +302,19 @@ static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
> else if (pol->flags & MPOL_F_RELATIVE_NODES)
> mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, nodes);
> else {
> - /*
> - * if step == 1, we use ->w.cpuset_mems_allowed to cache the
> - * result
> - */
> - if (step == MPOL_REBIND_ONCE || step == MPOL_REBIND_STEP1) {
> - nodes_remap(tmp, pol->v.nodes,
> - pol->w.cpuset_mems_allowed, *nodes);
> - pol->w.cpuset_mems_allowed = step ? tmp : *nodes;
> - } else if (step == MPOL_REBIND_STEP2) {
> - tmp = pol->w.cpuset_mems_allowed;
> - pol->w.cpuset_mems_allowed = *nodes;
> - } else
> - BUG();
> + nodes_remap(tmp, pol->v.nodes,pol->w.cpuset_mems_allowed,
> + *nodes);
> + pol->w.cpuset_mems_allowed = tmp;
> }
>
> if (nodes_empty(tmp))
> tmp = *nodes;
>
> - if (step == MPOL_REBIND_STEP1)
> - nodes_or(pol->v.nodes, pol->v.nodes, tmp);
> - else if (step == MPOL_REBIND_ONCE || step == MPOL_REBIND_STEP2)
> - pol->v.nodes = tmp;
> - else
> - BUG();
> + pol->v.nodes = tmp;
> }
>
> static void mpol_rebind_preferred(struct mempolicy *pol,
> - const nodemask_t *nodes,
> - enum mpol_rebind_step step)
> + const nodemask_t *nodes)
> {
> nodemask_t tmp;
>
> @@ -379,42 +340,19 @@ static void mpol_rebind_preferred(struct mempolicy *pol,
> /*
> * mpol_rebind_policy - Migrate a policy to a different set of nodes
> *
> - * If read-side task has no lock to protect task->mempolicy, write-side
> - * task will rebind the task->mempolicy by two step. The first step is
> - * setting all the newly nodes, and the second step is cleaning all the
> - * disallowed nodes. In this way, we can avoid finding no node to alloc
> - * page.
> - * If we have a lock to protect task->mempolicy in read-side, we do
> - * rebind directly.
> - *
> - * step:
> - * MPOL_REBIND_ONCE - do rebind work at once
> - * MPOL_REBIND_STEP1 - set all the newly nodes
> - * MPOL_REBIND_STEP2 - clean all the disallowed nodes
> + * Per-vma policies are protected by mmap_sem. Allocations using per-task
> + * policies are protected by task->mems_allowed_seq to prevent a premature
> + * OOM/allocation failure due to parallel nodemask modification.
> */
> -static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask,
> - enum mpol_rebind_step step)
> +static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask)
> {
> if (!pol)
> return;
> - if (!mpol_store_user_nodemask(pol) && step == MPOL_REBIND_ONCE &&
> + if (!mpol_store_user_nodemask(pol) &&
> nodes_equal(pol->w.cpuset_mems_allowed, *newmask))
> return;
>
> - if (step == MPOL_REBIND_STEP1 && (pol->flags & MPOL_F_REBINDING))
> - return;
> -
> - if (step == MPOL_REBIND_STEP2 && !(pol->flags & MPOL_F_REBINDING))
> - BUG();
> -
> - if (step == MPOL_REBIND_STEP1)
> - pol->flags |= MPOL_F_REBINDING;
> - else if (step == MPOL_REBIND_STEP2)
> - pol->flags &= ~MPOL_F_REBINDING;
> - else if (step >= MPOL_REBIND_NSTEP)
> - BUG();
> -
> - mpol_ops[pol->mode].rebind(pol, newmask, step);
> + mpol_ops[pol->mode].rebind(pol, newmask);
> }
>
> /*
> @@ -424,10 +362,9 @@ static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask,
> * Called with task's alloc_lock held.
> */
>
> -void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new,
> - enum mpol_rebind_step step)
> +void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new)
> {
> - mpol_rebind_policy(tsk->mempolicy, new, step);
> + mpol_rebind_policy(tsk->mempolicy, new);
> }
>
> /*
> @@ -442,7 +379,7 @@ void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new)
>
> down_write(&mm->mmap_sem);
> for (vma = mm->mmap; vma; vma = vma->vm_next)
> - mpol_rebind_policy(vma->vm_policy, new, MPOL_REBIND_ONCE);
> + mpol_rebind_policy(vma->vm_policy, new);
> up_write(&mm->mmap_sem);
> }
>
> @@ -2101,10 +2038,7 @@ struct mempolicy *__mpol_dup(struct mempolicy *old)
>
> if (current_cpuset_is_being_rebound()) {
> nodemask_t mems = cpuset_mems_allowed(current);
> - if (new->flags & MPOL_F_REBINDING)
> - mpol_rebind_policy(new, &mems, MPOL_REBIND_STEP2);
> - else
> - mpol_rebind_policy(new, &mems, MPOL_REBIND_ONCE);
> + mpol_rebind_policy(new, &mems);
> }
> atomic_set(&new->refcnt, 1);
> return new;
> --
> 2.12.2
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer
From: Michal Hocko @ 2017-05-23 7:07 UTC (permalink / raw)
To: Roman Gushchin
Cc: Vladimir Davydov, Johannes Weiner, Tejun Heo, Li Zefan,
Tetsuo Handa, kernel-team, cgroups, linux-doc, linux-kernel,
linux-mm
In-Reply-To: <20170522170116.GB22625@castle>
On Mon 22-05-17 18:01:16, Roman Gushchin wrote:
> On Sat, May 20, 2017 at 09:37:29PM +0300, Vladimir Davydov wrote:
> > Hello Roman,
>
> Hi Vladimir!
>
> >
> > On Thu, May 18, 2017 at 05:28:04PM +0100, Roman Gushchin wrote:
> > ...
> > > +5-2-4. Cgroup-aware OOM Killer
> > > +
> > > +Cgroup v2 memory controller implements a cgroup-aware OOM killer.
> > > +It means that it treats memory cgroups as memory consumers
> > > +rather then individual processes. Under the OOM conditions it tries
> > > +to find an elegible leaf memory cgroup, and kill all processes
> > > +in this cgroup. If it's not possible (e.g. all processes belong
> > > +to the root cgroup), it falls back to the traditional per-process
> > > +behaviour.
> >
> > I agree that the current OOM victim selection algorithm is totally
> > unfair in a system using containers and it has been crying for rework
> > for the last few years now, so it's great to see this finally coming.
> >
> > However, I don't reckon that killing a whole leaf cgroup is always the
> > best practice. It does make sense when cgroups are used for
> > containerizing services or applications, because a service is unlikely
> > to remain operational after one of its processes is gone, but one can
> > also use cgroups to containerize processes started by a user. Kicking a
> > user out for one of her process has gone mad doesn't sound right to me.
>
> I agree, that it's not always a best practise, if you're not allowed
> to change the cgroup configuration (e.g. create new cgroups).
> IMHO, this case is mostly covered by using the v1 cgroup interface,
> which remains unchanged.
But there are features which are v2 only and users might really want to
use it. So I really do not buy this v2-only argument.
> If you do have control over cgroups, you can put processes into
> separate cgroups, and obtain control over OOM victim selection and killing.
Usually you do not have that control because there is a global daemon
doing the placement for you.
> > Another example when the policy you're suggesting fails in my opinion is
> > in case a service (cgroup) consists of sub-services (sub-cgroups) that
> > run processes. The main service may stop working normally if one of its
> > sub-services is killed. So it might make sense to kill not just an
> > individual process or a leaf cgroup, but the whole main service with all
> > its sub-services.
>
> I agree, although I do not pretend for solving all possible
> userspace problems caused by an OOM.
>
> How to react on an OOM - is definitely a policy, which depends
> on the workload. Nothing is changing here from how it's working now,
> except now kernel will choose a victim cgroup, and kill the victim cgroup
> rather than a process.
There is a _big_ difference. The current implementation just tries
to recover from the OOM situation without carying much about the
consequences on the workload. This is the last resort and a services for
the _system_ to get back to sane state. You are trying to make it more
clever and workload aware and that is inevitable going to depend on the
specific workload. I really do think we cannot simply hardcode any
policy into the kernel for this purpose and that is why I would like to
see a discussion about how to do that in a more extensible way. This
might be harder to implement now but it I believe it will turn out
better longerm.
> > And both kinds of workloads (services/applications and individual
> > processes run by users) can co-exist on the same host - consider the
> > default systemd setup, for instance.
> >
> > IMHO it would be better to give users a choice regarding what they
> > really want for a particular cgroup in case of OOM - killing the whole
> > cgroup or one of its descendants. For example, we could introduce a
> > per-cgroup flag that would tell the kernel whether the cgroup can
> > tolerate killing a descendant or not. If it can, the kernel will pick
> > the fattest sub-cgroup or process and check it. If it cannot, it will
> > kill the whole cgroup and all its processes and sub-cgroups.
>
> The last thing we want to do, is to compare processes with cgroups.
> I agree, that we can have some option to disable the cgroup-aware OOM at all,
> mostly for backward-compatibility. But I don't think it should be a
> per-cgroup configuration option, which we will support forever.
I can clearly see a demand for "this is definitely more important
container than others so do not kill" usecases. I can also see demand
for "do not kill this container running for X days". And more are likely
to pop out.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: Define KB, MB, GB, TB in core VM
From: Christoph Hellwig @ 2017-05-23 7:02 UTC (permalink / raw)
To: Andrew Morton; +Cc: Anshuman Khandual, linux-mm, linux-kernel
In-Reply-To: <20170522141149.9ef84bb0713769f4af0383f0@linux-foundation.org>
On Mon, May 22, 2017 at 02:11:49PM -0700, Andrew Morton wrote:
> On Mon, 22 May 2017 16:47:42 +0530 Anshuman Khandual <khandual@linux.vnet.ibm.com> wrote:
>
> > There are many places where we define size either left shifting integers
> > or multiplying 1024s without any generic definition to fall back on. But
> > there are couples of (powerpc and lz4) attempts to define these standard
> > memory sizes. Lets move these definitions to core VM to make sure that
> > all new usage come from these definitions eventually standardizing it
> > across all places.
>
> Grep further - there are many more definitions and some may now
> generate warnings.
>
> Newly including mm.h for these things seems a bit heavyweight. I can't
> immediately think of a more appropriate place. Maybe printk.h or
> kernel.h.
IFF we do these kernel.h is the right place. And please also add the
MiB & co variants for the binary versions right next to the decimal
ones.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [v5 1/1] mm: Adaptive hash table scaling
From: Michal Hocko @ 2017-05-23 6:58 UTC (permalink / raw)
To: Pavel Tatashin; +Cc: akpm, linux-kernel, linux-mm, mpe
In-Reply-To: <1495469329-755807-2-git-send-email-pasha.tatashin@oracle.com>
On Mon 22-05-17 12:08:49, Pavel Tatashin wrote:
> Allow hash tables to scale with memory but at slower pace, when HASH_ADAPT
> is provided every time memory quadruples the sizes of hash tables will only
> double instead of quadrupling as well. This algorithm starts working only
> when memory size reaches a certain point, currently set to 64G.
>
> This is example of dentry hash table size, before and after four various
> memory configurations:
>
> MEMORY SCALE HASH_SIZE
> old new old new
> 8G 13 13 8M 8M
> 16G 13 13 16M 16M
> 32G 13 13 32M 32M
> 64G 13 13 64M 64M
> 128G 13 14 128M 64M
> 256G 13 14 256M 128M
> 512G 13 15 512M 128M
> 1024G 13 15 1024M 256M
> 2048G 13 16 2048M 256M
> 4096G 13 16 4096M 512M
> 8192G 13 17 8192M 512M
> 16384G 13 17 16384M 1024M
> 32768G 13 18 32768M 1024M
> 65536G 13 18 65536M 2048M
>
> Signed-off-by: Pavel Tatashin <pasha.tatashin@oracle.com>
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> mm/page_alloc.c | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 8afa63e81e73..409e0cd35381 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -7169,6 +7169,21 @@ static unsigned long __init arch_reserved_kernel_pages(void)
> #endif
>
> /*
> + * Adaptive scale is meant to reduce sizes of hash tables on large memory
> + * machines. As memory size is increased the scale is also increased but at
> + * slower pace. Starting from ADAPT_SCALE_BASE (64G), every time memory
> + * quadruples the scale is increased by one, which means the size of hash table
> + * only doubles, instead of quadrupling as well.
> + * Because 32-bit systems cannot have large physical memory, where this scaling
> + * makes sense, it is disabled on such platforms.
> + */
> +#if __BITS_PER_LONG > 32
> +#define ADAPT_SCALE_BASE (64ul << 30)
> +#define ADAPT_SCALE_SHIFT 2
> +#define ADAPT_SCALE_NPAGES (ADAPT_SCALE_BASE >> PAGE_SHIFT)
> +#endif
> +
> +/*
> * allocate a large system hash table from bootmem
> * - it is assumed that the hash table must contain an exact power-of-2
> * quantity of entries
> @@ -7199,6 +7214,16 @@ void *__init alloc_large_system_hash(const char *tablename,
> if (PAGE_SHIFT < 20)
> numentries = round_up(numentries, (1<<20)/PAGE_SIZE);
>
> +#if __BITS_PER_LONG > 32
> + if (!high_limit) {
> + unsigned long adapt;
> +
> + for (adapt = ADAPT_SCALE_NPAGES; adapt < numentries;
> + adapt <<= ADAPT_SCALE_SHIFT)
> + scale++;
> + }
> +#endif
> +
> /* limit to 1 bucket per 2^scale bytes of low memory */
> if (scale > PAGE_SHIFT)
> numentries >>= (scale - PAGE_SHIFT);
> --
> 2.13.0
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: dm ioctl: Restore __GFP_HIGH in copy_params()
From: Michal Hocko @ 2017-05-23 6:49 UTC (permalink / raw)
To: Mike Snitzer
Cc: Mikulas Patocka, Junaid Shahid, David Rientjes, Alasdair Kergon,
Andrew Morton, linux-mm, andreslc, gthelen, vbabka, linux-kernel
In-Reply-To: <20170522180415.GA25340@redhat.com>
On Mon 22-05-17 14:04:15, Mike Snitzer wrote:
> On Mon, May 22 2017 at 11:03am -0400,
> Michal Hocko <mhocko@kernel.org> wrote:
>
> > On Mon 22-05-17 10:52:44, Mikulas Patocka wrote:
> > >
> > >
> > > On Mon, 22 May 2017, Michal Hocko wrote:
> > [...]
> > > > I am not sure I understand. OOM killer is invoked for _all_ allocations
> > > > <= PAGE_ALLOC_COSTLY_ORDER that do not have __GFP_NORETRY as long as the
> > > > OOM killer is not disabled (oom_killer_disable) and that only happens
> > > > from the PM suspend path which makes sure that no userspace is active at
> > > > the time. AFAIU this is a userspace triggered path and so the later
> > > > shouldn't apply to it and GFP_KERNEL should be therefore sufficient.
> > > > Relying to a portion of memory reserves to prevent from deadlock seems
> > > > fundamentaly broken to me.
> > > >
> > >
> > > The lvm2 was designed this way - it is broken, but there is not much that
> > > can be done about it - fixing this would mean major rewrite. The only
> > > thing we can do about it is to lower the deadlock probability with
> > > __GFP_HIGH (or PF_MEMALLOC that was used some times ago).
>
> Yes, lvm2 was originally designed to to have access to memory reserves
> to ensure forward progress. But if the mm subsystem has improved to
> allow for the required progress without lvm2 trying to stake a claim on
> those reserves then we'll gladly avoid (ab)using them.
>
> > But let me repeat. GFP_KERNEL allocation for order-0 page will not fail.
>
> OK, but will it be serviced immediately? Not failing isn't useful if it
> never completes.
Well, GFP_KERNEL will not guarantee an immediate success of course.
There is nothing like that. Nor __GFP_HIGH will guarantee that, though,
because reserves can get easily depleted by some workloads. You would
have to use a dedicated memory pool to accomplish what you really need.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 4/6] powerpc/mm: Add devmap support for ppc64
From: Oliver O'Halloran @ 2017-05-23 6:42 UTC (permalink / raw)
To: Aneesh Kumar K.V; +Cc: linuxppc-dev, Linux MM
In-Reply-To: <87efvgqk08.fsf@skywalker.in.ibm.com>
On Tue, May 23, 2017 at 2:23 PM, Aneesh Kumar K.V
<aneesh.kumar@linux.vnet.ibm.com> wrote:
> Oliver O'Halloran <oohall@gmail.com> writes:
>
>> Add support for the devmap bit on PTEs and PMDs for PPC64 Book3S. This
>> is used to differentiate device backed memory from transparent huge
>> pages since they are handled in more or less the same manner by the core
>> mm code.
>>
>> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>> Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
>> ---
>> v1 -> v2: Properly differentiate THP and PMD Devmap entries. The
>> mm core assumes that pmd_trans_huge() and pmd_devmap() are mutually
>> exclusive and v1 had pmd_trans_huge() being true on a devmap pmd.
>>
>> Aneesh, this has been fleshed out substantially since v1. Can you
>> re-review it? Also no explicit gup support is required in this patch
>> since devmap support was added generic GUP as a part of making x86 use
>> the generic version.
>> ---
>> arch/powerpc/include/asm/book3s/64/hash-64k.h | 2 +-
>> arch/powerpc/include/asm/book3s/64/pgtable.h | 37 ++++++++++++++++++++++++++-
>> arch/powerpc/include/asm/book3s/64/radix.h | 2 +-
>> arch/powerpc/mm/hugetlbpage.c | 2 +-
>> arch/powerpc/mm/pgtable-book3s64.c | 4 +--
>> arch/powerpc/mm/pgtable-hash64.c | 4 ++-
>> arch/powerpc/mm/pgtable-radix.c | 3 ++-
>> arch/powerpc/mm/pgtable_64.c | 2 +-
>> 8 files changed, 47 insertions(+), 9 deletions(-)
>>
>> diff --git a/arch/powerpc/include/asm/book3s/64/hash-64k.h b/arch/powerpc/include/asm/book3s/64/hash-64k.h
>> index 9732837aaae8..eaaf613c5347 100644
>> --- a/arch/powerpc/include/asm/book3s/64/hash-64k.h
>> +++ b/arch/powerpc/include/asm/book3s/64/hash-64k.h
>> @@ -180,7 +180,7 @@ static inline void mark_hpte_slot_valid(unsigned char *hpte_slot_array,
>> */
>> static inline int hash__pmd_trans_huge(pmd_t pmd)
>> {
>> - return !!((pmd_val(pmd) & (_PAGE_PTE | H_PAGE_THP_HUGE)) ==
>> + return !!((pmd_val(pmd) & (_PAGE_PTE | H_PAGE_THP_HUGE | _PAGE_DEVMAP)) ==
>> (_PAGE_PTE | H_PAGE_THP_HUGE));
>> }
>
> _PAGE_DEVMAP is not really needed here. We will set H_PAGE_THP_HUGE only
> for thp hugepage w.r.t hash. But putting it here also makes it clear
> that devmap entries are not considered trans huge.
Good point. I'll remove it.
>>
>> diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
>> index 85bc9875c3be..24634e92dd0b 100644
>> --- a/arch/powerpc/include/asm/book3s/64/pgtable.h
>> +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
>> @@ -79,6 +79,9 @@
>>
>> #define _PAGE_SOFT_DIRTY _RPAGE_SW3 /* software: software dirty tracking */
>> #define _PAGE_SPECIAL _RPAGE_SW2 /* software: special page */
>> +#define _PAGE_DEVMAP _RPAGE_SW1
>> +#define __HAVE_ARCH_PTE_DEVMAP
>> +
>> /*
>> * Drivers request for cache inhibited pte mapping using _PAGE_NO_CACHE
>> * Instead of fixing all of them, add an alternate define which
>> @@ -599,6 +602,16 @@ static inline pte_t pte_mkhuge(pte_t pte)
>> return pte;
>> }
>>
>> +static inline pte_t pte_mkdevmap(pte_t pte)
>> +{
>> + return __pte(pte_val(pte) | _PAGE_SPECIAL|_PAGE_DEVMAP);
>> +}
>> +
>> +static inline int pte_devmap(pte_t pte)
>> +{
>> + return !!(pte_raw(pte) & cpu_to_be64(_PAGE_DEVMAP));
>> +}
>> +
>> static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
>> {
>> /* FIXME!! check whether this need to be a conditional */
>> @@ -963,6 +976,9 @@ static inline pte_t *pmdp_ptep(pmd_t *pmd)
>> #define pmd_mk_savedwrite(pmd) pte_pmd(pte_mk_savedwrite(pmd_pte(pmd)))
>> #define pmd_clear_savedwrite(pmd) pte_pmd(pte_clear_savedwrite(pmd_pte(pmd)))
>>
>> +#define pud_pfn(...) (0)
>> +#define pgd_pfn(...) (0)
>> +
>> #ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY
>> #define pmd_soft_dirty(pmd) pte_soft_dirty(pmd_pte(pmd))
>> #define pmd_mksoft_dirty(pmd) pte_pmd(pte_mksoft_dirty(pmd_pte(pmd)))
>> @@ -1137,7 +1153,6 @@ static inline int pmd_move_must_withdraw(struct spinlock *new_pmd_ptl,
>> return true;
>> }
>>
>> -
>> #define arch_needs_pgtable_deposit arch_needs_pgtable_deposit
>> static inline bool arch_needs_pgtable_deposit(void)
>> {
>> @@ -1146,6 +1161,26 @@ static inline bool arch_needs_pgtable_deposit(void)
>> return true;
>> }
>>
>> +static inline pmd_t pmd_mkdevmap(pmd_t pmd)
>> +{
>> + return pte_pmd(pte_mkdevmap(pmd_pte(pmd)));
>> +}
>
>
> We avoided setting _PAGE_SPECIAL on pmd entries. This will set that, we
> may want to check if it is ok. IIRC, we overloaded _PAGE_SPECIAL at some point to indicate thp splitting. But good to double check.
I took a cursory look in arch/powerpc/ and mm/ for usages and didn't
see any usages of _PAGE_SPECIAL for pmds. There's no good reason to
set the flag though so I'll remove it.
>> +
>> +static inline int pmd_devmap(pmd_t pmd)
>> +{
>> + return pte_devmap(pmd_pte(pmd));
>> +}
>> +
>> +static inline int pud_devmap(pud_t pud)
>> +{
>> + return 0;
>> +}
>> +
>> +static inline int pgd_devmap(pgd_t pgd)
>> +{
>> + return 0;
>> +}
>> +
>> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>> #endif /* __ASSEMBLY__ */
>> #endif /* _ASM_POWERPC_BOOK3S_64_PGTABLE_H_ */
>> diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h
>> index ac16d1943022..ba43754e96d2 100644
>> --- a/arch/powerpc/include/asm/book3s/64/radix.h
>> +++ b/arch/powerpc/include/asm/book3s/64/radix.h
>> @@ -252,7 +252,7 @@ static inline int radix__pgd_bad(pgd_t pgd)
>>
>> static inline int radix__pmd_trans_huge(pmd_t pmd)
>> {
>> - return !!(pmd_val(pmd) & _PAGE_PTE);
>> + return (pmd_val(pmd) & (_PAGE_PTE | _PAGE_DEVMAP)) == _PAGE_PTE;
>> }
>>
>> static inline pmd_t radix__pmd_mkhuge(pmd_t pmd)
>> diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c
>> index a4f33de4008e..d9958af5c98e 100644
>> --- a/arch/powerpc/mm/hugetlbpage.c
>> +++ b/arch/powerpc/mm/hugetlbpage.c
>> @@ -963,7 +963,7 @@ pte_t *__find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea,
>> if (pmd_none(pmd))
>> return NULL;
>>
>> - if (pmd_trans_huge(pmd)) {
>> + if (pmd_trans_huge(pmd) || pmd_devmap(pmd)) {
>> if (is_thp)
>> *is_thp = true;
>> ret_pte = (pte_t *) pmdp;
>
>
> Is that correct ? Do we want pmd_devmap to have is_thp set ?
I think so, is_thp is used to differentiate between explicit and
transparent hugepages in the hash fault handler. The management and
fault handling of pmd devmap pages and thp is the same (by design)
while hugepages seem to have their own requirements. Most users of
find_linux_pte_or_hugepte() don't look at is_thp either so it should
be safe.
>> diff --git a/arch/powerpc/mm/pgtable-book3s64.c b/arch/powerpc/mm/pgtable-book3s64.c
>> index 5fcb3dd74c13..31eed8fa8e99 100644
>> --- a/arch/powerpc/mm/pgtable-book3s64.c
>> +++ b/arch/powerpc/mm/pgtable-book3s64.c
>> @@ -32,7 +32,7 @@ int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address,
>> {
>> int changed;
>> #ifdef CONFIG_DEBUG_VM
>> - WARN_ON(!pmd_trans_huge(*pmdp));
>> + WARN_ON(!pmd_trans_huge(*pmdp) && !pmd_devmap(*pmdp));
>> assert_spin_locked(&vma->vm_mm->page_table_lock);
>> #endif
>> changed = !pmd_same(*(pmdp), entry);
>> @@ -59,7 +59,7 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr,
>> #ifdef CONFIG_DEBUG_VM
>> WARN_ON(pte_present(pmd_pte(*pmdp)) && !pte_protnone(pmd_pte(*pmdp)));
>> assert_spin_locked(&mm->page_table_lock);
>> - WARN_ON(!pmd_trans_huge(pmd));
>> + WARN_ON(!(pmd_trans_huge(pmd) || pmd_devmap(pmd)));
>> #endif
>> trace_hugepage_set_pmd(addr, pmd_val(pmd));
>> return set_pte_at(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
>> diff --git a/arch/powerpc/mm/pgtable-hash64.c b/arch/powerpc/mm/pgtable-hash64.c
>> index 8b85a14b08ea..7456cde4dbce 100644
>> --- a/arch/powerpc/mm/pgtable-hash64.c
>> +++ b/arch/powerpc/mm/pgtable-hash64.c
>> @@ -109,7 +109,7 @@ unsigned long hash__pmd_hugepage_update(struct mm_struct *mm, unsigned long addr
>> unsigned long old;
>>
>> #ifdef CONFIG_DEBUG_VM
>> - WARN_ON(!pmd_trans_huge(*pmdp));
>> + WARN_ON(!hash__pmd_trans_huge(*pmdp) && !pmd_devmap(*pmdp));
>> assert_spin_locked(&mm->page_table_lock);
>> #endif
>>
>> @@ -141,6 +141,7 @@ pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addres
>>
>> VM_BUG_ON(address & ~HPAGE_PMD_MASK);
>> VM_BUG_ON(pmd_trans_huge(*pmdp));
>> + VM_BUG_ON(pmd_devmap(*pmdp));
>>
>> pmd = *pmdp;
>> pmd_clear(pmdp);
>> @@ -221,6 +222,7 @@ void hash__pmdp_huge_split_prepare(struct vm_area_struct *vma,
>> {
>> VM_BUG_ON(address & ~HPAGE_PMD_MASK);
>> VM_BUG_ON(REGION_ID(address) != USER_REGION_ID);
>> + VM_BUG_ON(pmd_devmap(*pmdp));
>>
>> /*
>> * We can't mark the pmd none here, because that will cause a race
>> diff --git a/arch/powerpc/mm/pgtable-radix.c b/arch/powerpc/mm/pgtable-radix.c
>> index c28165d8970b..69e28dda81f2 100644
>> --- a/arch/powerpc/mm/pgtable-radix.c
>> +++ b/arch/powerpc/mm/pgtable-radix.c
>> @@ -683,7 +683,7 @@ unsigned long radix__pmd_hugepage_update(struct mm_struct *mm, unsigned long add
>> unsigned long old;
>>
>> #ifdef CONFIG_DEBUG_VM
>> - WARN_ON(!radix__pmd_trans_huge(*pmdp));
>> + WARN_ON(!radix__pmd_trans_huge(*pmdp) && !pmd_devmap(*pmdp));
>> assert_spin_locked(&mm->page_table_lock);
>> #endif
>>
>> @@ -701,6 +701,7 @@ pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addre
>>
>> VM_BUG_ON(address & ~HPAGE_PMD_MASK);
>> VM_BUG_ON(radix__pmd_trans_huge(*pmdp));
>> + VM_BUG_ON(pmd_devmap(*pmdp));
>> /*
>> * khugepaged calls this for normal pmd
>> */
>> diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
>> index db93cf747a03..aefde9bd3110 100644
>> --- a/arch/powerpc/mm/pgtable_64.c
>> +++ b/arch/powerpc/mm/pgtable_64.c
>> @@ -323,7 +323,7 @@ struct page *pud_page(pud_t pud)
>> */
>> struct page *pmd_page(pmd_t pmd)
>> {
>> - if (pmd_trans_huge(pmd) || pmd_huge(pmd))
>> + if (pmd_trans_huge(pmd) || pmd_huge(pmd) || pmd_devmap(pmd))
>> return pte_page(pmd_pte(pmd));
>> return virt_to_page(pmd_page_vaddr(pmd));
>> }
>> --
>> 2.9.3
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 5/6] mm, x86: Add ARCH_HAS_ZONE_DEVICE
From: Ingo Molnar @ 2017-05-23 6:42 UTC (permalink / raw)
To: Oliver O'Halloran; +Cc: linuxppc-dev, linux-mm, x86
In-Reply-To: <20170523040524.13717-5-oohall@gmail.com>
* Oliver O'Halloran <oohall@gmail.com> wrote:
> Currently ZONE_DEVICE depends on X86_64. This is fine for now, but it
> will get unwieldly as new platforms get ZONE_DEVICE support. Moving it
> to an arch selected Kconfig option to save us some trouble in the
> future.
>
> Cc: x86@kernel.org
> Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Thanks,
Ingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: Define KB, MB, GB, TB in core VM
From: kbuild test robot @ 2017-05-23 6:41 UTC (permalink / raw)
To: Anshuman Khandual; +Cc: kbuild-all, linux-mm, linux-kernel, akpm
In-Reply-To: <20170522111742.29433-1-khandual@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 3969 bytes --]
Hi Anshuman,
[auto build test WARNING on linus/master]
[also build test WARNING on v4.12-rc2 next-20170522]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-Define-KB-MB-GB-TB-in-core-VM/20170523-141359
config: i386-tinyconfig (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All warnings (new ones prefixed by >>):
>> arch/x86/kernel/cpu/intel_cacheinfo.c:34:0: warning: "MB" redefined
#define MB(x) ((x) * 1024)
In file included from arch/x86/include/asm/pci.h:4:0,
from include/linux/pci.h:1618,
from arch/x86/kernel/cpu/intel_cacheinfo.c:16:
include/linux/mm.h:2553:0: note: this is the location of the previous definition
#define MB (1UL << 20)
vim +/MB +34 arch/x86/kernel/cpu/intel_cacheinfo.c
cd4d09ec arch/x86/kernel/cpu/intel_cacheinfo.c Borislav Petkov 2016-01-26 18 #include <asm/cpufeature.h>
23ac4ae8 arch/x86/kernel/cpu/intel_cacheinfo.c Andreas Herrmann 2010-09-17 19 #include <asm/amd_nb.h>
dcf39daf arch/x86/kernel/cpu/intel_cacheinfo.c Borislav Petkov 2010-01-22 20 #include <asm/smp.h>
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 21
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 22 #define LVL_1_INST 1
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 23 #define LVL_1_DATA 2
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 24 #define LVL_2 3
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 25 #define LVL_3 4
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 26 #define LVL_TRACE 5
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 27
8bdbd962 arch/x86/kernel/cpu/intel_cacheinfo.c Alan Cox 2009-07-04 28 struct _cache_table {
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 29 unsigned char descriptor;
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 30 char cache_type;
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 31 short size;
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 32 };
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 33
2ca49b2f arch/x86/kernel/cpu/intel_cacheinfo.c Dave Jones 2010-01-04 @34 #define MB(x) ((x) * 1024)
2ca49b2f arch/x86/kernel/cpu/intel_cacheinfo.c Dave Jones 2010-01-04 35
8bdbd962 arch/x86/kernel/cpu/intel_cacheinfo.c Alan Cox 2009-07-04 36 /* All the cache descriptor types we care about (no TLB or
8bdbd962 arch/x86/kernel/cpu/intel_cacheinfo.c Alan Cox 2009-07-04 37 trace cache entries) */
8bdbd962 arch/x86/kernel/cpu/intel_cacheinfo.c Alan Cox 2009-07-04 38
148f9bb8 arch/x86/kernel/cpu/intel_cacheinfo.c Paul Gortmaker 2013-06-18 39 static const struct _cache_table cache_table[] =
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 40 {
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 41 { 0x06, LVL_1_INST, 8 }, /* 4-way set assoc, 32 byte line size */
^1da177e arch/i386/kernel/cpu/intel_cacheinfo.c Linus Torvalds 2005-04-16 42 { 0x08, LVL_1_INST, 16 }, /* 4-way set assoc, 32 byte line size */
:::::: The code at line 34 was first introduced by commit
:::::: 2ca49b2fcf5813571663c3c4c894b78148c43690 x86: Macroise x86 cache descriptors
:::::: TO: Dave Jones <davej@redhat.com>
:::::: CC: Ingo Molnar <mingo@elte.hu>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 6567 bytes --]
^ permalink raw reply
* Re: [PATCH 0/6] refine and rename slub sysfs
From: Michal Hocko @ 2017-05-23 6:39 UTC (permalink / raw)
To: Wei Yang; +Cc: cl, penberg, rientjes, akpm, linux-mm, linux-kernel
In-Reply-To: <20170523032705.GA4275@WeideMBP.lan>
On Tue 23-05-17 11:27:05, Wei Yang wrote:
> On Thu, May 18, 2017 at 11:06:37AM +0200, Michal Hocko wrote:
> >On Wed 17-05-17 22:11:40, Wei Yang wrote:
> >> This patch serial could be divided into two parts.
> >>
> >> First three patches refine and adds slab sysfs.
> >> Second three patches rename slab sysfs.
> >>
> >> 1. Refine slab sysfs
> >>
> >> There are four level slabs:
> >>
> >> CPU
> >> CPU_PARTIAL
> >> PARTIAL
> >> FULL
> >>
> >> And in sysfs, it use show_slab_objects() and cpu_partial_slabs_show() to
> >> reflect the statistics.
> >>
> >> In patch 2, it splits some function in show_slab_objects() which makes sure
> >> only cpu_partial_slabs_show() covers statistics for CPU_PARTIAL slabs.
> >>
> >> After doing so, it would be more clear that show_slab_objects() has totally 9
> >> statistic combinations for three level of slabs. Each slab has three cases
> >> statistic.
> >>
> >> slabs
> >> objects
> >> total_objects
> >>
> >> And when we look at current implementation, some of them are missing. So patch
> >> 2 & 3 add them up.
> >>
> >> 2. Rename sysfs
> >>
> >> The slab statistics in sysfs are
> >>
> >> slabs
> >> objects
> >> total_objects
> >> cpu_slabs
> >> partial
> >> partial_objects
> >> cpu_partial_slabs
> >>
> >> which is a little bit hard for users to understand. The second three patches
> >> rename sysfs file in this pattern.
> >>
> >> xxx_slabs[[_total]_objects]
> >>
> >> Finally it looks Like
> >>
> >> slabs
> >> slabs_objects
> >> slabs_total_objects
> >> cpu_slabs
> >> cpu_slabs_objects
> >> cpu_slabs_total_objects
> >> partial_slabs
> >> partial_slabs_objects
> >> partial_slabs_total_objects
> >> cpu_partial_slabs
> >
> >_Why_ do we need all this?
>
> To have a clear statistics for each slab level.
Is this worth risking breakage of the userspace which consume this data
now? Do you have any user space code which will greatly benefit from the
new data and which couldn't do the same with the current format/output?
If yes this all should be in the changelog.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: dm ioctl: Restore __GFP_HIGH in copy_params()
From: Michal Hocko @ 2017-05-23 6:05 UTC (permalink / raw)
To: David Rientjes
Cc: Mike Snitzer, Mikulas Patocka, Junaid Shahid, Alasdair Kergon,
Andrew Morton, linux-mm, andreslc, gthelen, vbabka, linux-kernel
In-Reply-To: <alpine.DEB.2.10.1705221325200.30407@chino.kir.corp.google.com>
On Mon 22-05-17 13:35:41, David Rientjes wrote:
> On Mon, 22 May 2017, Mike Snitzer wrote:
[...]
> > While adding the __GFP_NOFAIL flag would serve to document expectations
> > I'm left unconvinced that the memory allocator will _not fail_ for an
> > order-0 page -- as Mikulas said most ioctls don't need more than 4K.
>
> __GFP_NOFAIL would make no sense in kvmalloc() calls, ever, it would never
> fallback to vmalloc :)
Sorry, I could have been more specific. You would have to opencode
kvmalloc obviously. It is documented to not support this flag for the
reasons you have mentioned above.
> I'm hoping this can get merged during the 4.12 window to fix the broken
> commit d224e9381897.
I obviously disagree. Relying on memory reserves for _correctness_ is
clearly broken by design, full stop. But it is dm code and you are going
it is responsibility of the respective maintainers to support this code.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox