* mmotm 2017-05-18-14-18 uploaded
From: akpm @ 2017-05-18 21:19 UTC (permalink / raw)
To: mm-commits, linux-kernel, linux-mm, linux-fsdevel, linux-next,
sfr, mhocko, broonie
The mm-of-the-moment snapshot 2017-05-18-14-18 has been uploaded to
http://www.ozlabs.org/~akpm/mmotm/
mmotm-readme.txt says
README for mm-of-the-moment:
http://www.ozlabs.org/~akpm/mmotm/
This is a snapshot of my -mm patch queue. Uploaded at random hopefully
more than once a week.
You will need quilt to apply these patches to the latest Linus release (4.x
or 4.x-rcY). The series file is in broken-out.tar.gz and is duplicated in
http://ozlabs.org/~akpm/mmotm/series
The file broken-out.tar.gz contains two datestamp files: .DATE and
.DATE-yyyy-mm-dd-hh-mm-ss. Both contain the string yyyy-mm-dd-hh-mm-ss,
followed by the base kernel version against which this patch series is to
be applied.
This tree is partially included in linux-next. To see which patches are
included in linux-next, consult the `series' file. Only the patches
within the #NEXT_PATCHES_START/#NEXT_PATCHES_END markers are included in
linux-next.
A git tree which contains the memory management portion of this tree is
maintained at git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git
by Michal Hocko. It contains the patches which are between the
"#NEXT_PATCHES_START mm" and "#NEXT_PATCHES_END" markers, from the series
file, http://www.ozlabs.org/~akpm/mmotm/series.
A full copy of the full kernel tree with the linux-next and mmotm patches
already applied is available through git within an hour of the mmotm
release. Individual mmotm releases are tagged. The master branch always
points to the latest release, so it's constantly rebasing.
http://git.cmpxchg.org/cgit.cgi/linux-mmotm.git/
To develop on top of mmotm git:
$ git remote add mmotm git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git
$ git remote update mmotm
$ git checkout -b topic mmotm/master
<make changes, commit>
$ git send-email mmotm/master.. [...]
To rebase a branch with older patches to a new mmotm release:
$ git remote update mmotm
$ git rebase --onto mmotm/master <topic base> topic
The directory http://www.ozlabs.org/~akpm/mmots/ (mm-of-the-second)
contains daily snapshots of the -mm tree. It is updated more frequently
than mmotm, and is untested.
A git copy of this tree is available at
http://git.cmpxchg.org/cgit.cgi/linux-mmots.git/
and use of this tree is similar to
http://git.cmpxchg.org/cgit.cgi/linux-mmotm.git/, described above.
This mmotm tree contains the following patches against 4.12-rc1:
(patches marked "*" will be included in linux-next)
origin.patch
i-need-old-gcc.patch
* mm-skip-hwpoisoned-pages-when-onlining-pages.patch
* ksm-prevent-crash-after-write_protect_page-fails.patch
* maintainers-greybus-dev-list-is-members-only.patch
* include-linux-gfph-fix-___gfp_nolockdep-value.patch
* frv-declare-jiffies-to-be-located-in-the-data-section.patch
* mm-clarify-why-we-want-kmalloc-before-falling-backto-vmallock.patch
* mm-clarify-why-we-want-kmalloc-before-falling-backto-vmallock-checkpatch-fixes.patch
* arm-arch-arm-include-asm-pageh-needs-personalityh.patch
* mn10300-remove-wrapper-header-for-asm-deviceh.patch
* teach-initramfs_root_uid-and-initramfs_root_gid-that-1-means-current-user.patch
* clarify-help-text-that-compression-applies-to-ramfs-as-well-as-legacy-ramdisk.patch
* sh-intc-delete-an-error-message-for-a-failed-memory-allocation-in-add_virq_to_pirq.patch
* ocfs2-old-mle-put-and-release-after-the-function-dlm_add_migration_mle-called.patch
* ocfs2-old-mle-put-and-release-after-the-function-dlm_add_migration_mle-called-fix.patch
* ocfs2-dlm-optimization-of-code-while-free-dead-node-locks.patch
* ocfs2-dlm-optimization-of-code-while-free-dead-node-locks-checkpatch-fixes.patch
* ocfs2-give-an-obvious-tip-for-dismatch-cluster-names.patch
* block-restore-proc-partitions-to-not-display-non-partitionable-removable-devices.patch
mm.patch
* mm-slub-remove-a-redundant-assignment-in-___slab_alloc.patch
* mm-slub-reset-cpu_slabs-pointer-in-deactivate_slab.patch
* mm-slub-pack-red_left_pad-with-another-int-to-save-a-word.patch
* mm-slub-wrap-cpu_slab-partial-in-config_slub_cpu_partial.patch
* mm-slub-wrap-cpu_slab-partial-in-config_slub_cpu_partial-fix.patch
* mm-slub-wrap-kmem_cache-cpu_partial-in-config-config_slub_cpu_partial.patch
* mm-sparsemem-break-out-of-loops-early.patch
* mark-protection_map-as-__ro_after_init.patch
* mm-vmscan-fix-unsequenced-modification-and-access-warning.patch
* mm-nobootmem-return-0-when-start_pfn-equals-end_pfn.patch
* ksm-introduce-ksm_max_page_sharing-per-page-deduplication-limit.patch
* ksm-fix-use-after-free-with-merge_across_nodes-=-0.patch
* ksm-cleanup-stable_node-chain-collapse-case.patch
* ksm-swap-the-two-output-parameters-of-chain-chain_prune.patch
* ksm-optimize-refile-of-stable_node_dup-at-the-head-of-the-chain.patch
* zram-introduce-zram_entry-to-prepare-dedup-functionality.patch
* zram-implement-deduplication-in-zram.patch
* zram-make-deduplication-feature-optional.patch
* zram-compare-all-the-entries-with-same-checksum-for-deduplication.patch
* zram-count-same-page-write-as-page_stored.patch
* zram-do-not-count-duplicated-pages-as-compressed.patch
* zram-try-harder-to-store-user-data-on-compression-error.patch
* mm-vmstat-standardize-file-operations-variable-names.patch
* mm-thp-swap-delay-splitting-thp-during-swap-out.patch
* mm-thp-swap-delay-splitting-thp-during-swap-out-fix.patch
* mm-thp-swap-unify-swap-slot-free-functions-to-put_swap_page.patch
* mm-thp-swap-move-anonymous-thp-split-logic-to-vmscan.patch
* mm-thp-swap-check-whether-thp-can-be-split-firstly.patch
* mm-thp-swap-enable-thp-swap-optimization-only-if-has-compound-map.patch
* mm-remove-return-value-from-init_currently_empty_zone.patch
* mm-memory_hotplug-use-node-instead-of-zone-in-can_online_high_movable.patch
* mm-drop-page_initialized-check-from-get_nid_for_pfn.patch
* mm-memory_hotplug-get-rid-of-is_zone_device_section.patch
* mm-memory_hotplug-split-up-register_one_node.patch
* mm-memory_hotplug-consider-offline-memblocks-removable.patch
* mm-consider-zone-which-is-not-fully-populated-to-have-holes.patch
* mm-consider-zone-which-is-not-fully-populated-to-have-holes-fix.patch
* mm-compaction-skip-over-holes-in-__reset_isolation_suitable.patch
* mm-__first_valid_page-skip-over-offline-pages.patch
* mm-vmstat-skip-reporting-offline-pages-in-pagetypeinfo.patch
* mm-memory_hotplug-do-not-associate-hotadded-memory-to-zones-until-online.patch
* mm-memory_hotplug-replace-for_device-by-want_memblock-in-arch_add_memory.patch
* mm-memory_hotplug-fix-the-section-mismatch-warning.patch
* mm-memory_hotplug-remove-unused-cruft-after-memory-hotplug-rework.patch
* exit-dont-include-unused-userfaultfd_kh.patch
* userfaultfd-drop-dead-code.patch
* mm-madvise-enable-softhard-offline-of-hugetlb-pages-at-pgd-level.patch
* mm-hugetlb-migration-use-set_huge_pte_at-instead-of-set_pte_at.patch
* mm-follow_page_mask-split-follow_page_mask-to-smaller-functions.patch
* mm-hugetlb-export-hugetlb_entry_migration-helper.patch
* mm-follow_page_mask-add-support-for-hugetlb-pgd-entries.patch
* mm-hugetlb-move-default-definition-of-hugepd_t-earlier-in-the-header.patch
* mm-follow_page_mask-add-support-for-hugepage-directory-entry.patch
* powerpc-hugetlb-add-follow_huge_pd-implementation-for-ppc64.patch
* powerpc-mm-hugetlb-remove-follow_huge_addr-for-powerpc.patch
* powerpc-hugetlb-enable-hugetlb-migration-for-ppc64.patch
* mm-zeroing-hash-tables-in-allocator.patch
* mm-updated-callers-to-use-hash_zero-flag.patch
* mm-adaptive-hash-table-scaling.patch
* mm-adaptive-hash-table-scaling-fix.patch
* mm-hugetlb-cleanup-arch_has_gigantic_page.patch
* powerpc-mm-hugetlb-add-support-for-1g-huge-pages.patch
* mm-page_alloc-mark-bad_range-and-meminit_pfn_in_nid-as-__maybe_unused.patch
* mm-drop-null-return-check-of-pte_offset_map_lock.patch
* mm-page_alloc-return-0-in-case-this-node-has-no-page-within-the-zone.patch
* mm-vmscan-do-not-pass-reclaimed-slab-to-vmpressure.patch
* mm-page_owner-align-with-pageblock_nr-pages.patch
* mm-walk-the-zone-in-pageblock_nr_pages-steps.patch
* mm-kasan-use-kasan_zero_pud-for-p4d-table.patch
* mm-kasan-rename-xxx_is_zero-to-xxx_is_nonzero.patch
* frv-remove-wrapper-header-for-asm-deviceh.patch
* kstrtox-delete-end-of-string-test.patch
* kstrtox-use-unsigned-int-more.patch
* lib-interval_tree_test-allow-the-module-to-be-compiled-in.patch
* lib-interval_tree_test-make-test-options-module-parameters.patch
* lib-interval_tree_test-allow-users-to-limit-scope-of-endpoint.patch
* lib-interval_tree_test-allow-full-tree-search.patch
* fs-epoll-short-circuit-fetching-events-if-thread-has-been-killed.patch
* seq_file-delete-small-value-optimization.patch
* virtually-mapped-stacks-do-not-disable-interrupts.patch
* kexec-move-vmcoreinfo-out-of-the-kernels-bss-section.patch
* powerpc-fadump-use-the-correct-vmcoreinfo_note_size-for-phdr.patch
* powerpc-fadump-use-the-correct-vmcoreinfo_note_size-for-phdr-fix.patch
* kdump-protect-vmcoreinfo-data-under-the-crash-memory.patch
* kexec-kdump-minor-documentation-updates-for-arm64-and-image.patch
* kdump-vmcoreinfo-report-actual-value-of-phys_base.patch
* uapi-fix-linux-sysctlh-userspace-compilation-errors.patch
* bfs-fix-sanity-checks-for-empty-files.patch
* fs-kill-config_percpu_rwsem-some-more.patch
* scripts-gdb-add-lx-fdtdump-command.patch
* procfs-fdinfo-extend-information-about-epoll-target-files.patch
* kcmp-add-kcmp_epoll_tfd-mode-to-compare-epoll-target-files.patch
* kcmp-fs-epoll-wrap-kcmp-code-with-config_checkpoint_restore.patch
* kernel-reboot-add-devm_register_reboot_notifier.patch
* kernel-reboot-add-devm_register_reboot_notifier-fix.patch
* fault-inject-support-systematic-fault-injection.patch
* fault-inject-support-systematic-fault-injection-fix.patch
* fault-inject-automatically-detect-the-number-base-for-fail-nth-write-interface.patch
* fault-inject-parse-as-natural-1-based-value-for-fail-nth-write-interface.patch
* fault-inject-make-fail-nth-read-write-interface-symmetric.patch
* fault-inject-simplify-access-check-for-fail-nth.patch
* fault-inject-simplify-access-check-for-fail-nth-fix.patch
* fault-inject-add-proc-pid-fail-nth.patch
* make-initramfs-honor-config_devtmpfs_mount.patch
* ipc-semc-remove-sem_base-embed-struct-sem.patch
* ipc-merge-ipc_rcu-and-kern_ipc_perm.patch
* include-linux-semh-correctly-document-sem_ctime.patch
linux-next.patch
* sparc64-ng4-memset-32-bits-overflow.patch
* powerpc-sequoia-fix-nand-partitions-not-to-overlap.patch
* lib-crc-ccitt-add-ccitt-false-crc16-variant.patch
mm-add-strictlimit-knob-v2.patch
make-sure-nobodys-leaking-resources.patch
releasing-resources-with-children.patch
kernel-forkc-export-kernel_thread-to-modules.patch
mutex-subsystem-synchro-test-module.patch
slab-leaks3-default-y.patch
workaround-for-a-pci-restoring-bug.patch
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/2] dax: Fix race between colliding PMD & PTE entries
From: Ross Zwisler @ 2017-05-18 21:29 UTC (permalink / raw)
To: Jan Kara
Cc: Ross Zwisler, Andrew Morton, linux-kernel, Darrick J. Wong,
Alexander Viro, Christoph Hellwig, Dan Williams, Dave Hansen,
Matthew Wilcox, linux-fsdevel, linux-mm, linux-nvdimm,
Kirill A . Shutemov, Pawel Lebioda, Dave Jiang, Xiong Zhou,
Eryu Guan, stable
In-Reply-To: <20170518075037.GA9084@quack2.suse.cz>
On Thu, May 18, 2017 at 09:50:37AM +0200, Jan Kara wrote:
> On Wed 17-05-17 11:16:39, Ross Zwisler wrote:
> > We currently have two related PMD vs PTE races in the DAX code. These can
> > both be easily triggered by having two threads reading and writing
> > simultaneously to the same private mapping, with the key being that private
> > mapping reads can be handled with PMDs but private mapping writes are
> > always handled with PTEs so that we can COW.
> >
> > Here is the first race:
> >
> > CPU 0 CPU 1
> >
> > (private mapping write)
> > __handle_mm_fault()
> > create_huge_pmd() - FALLBACK
> > handle_pte_fault()
> > passes check for pmd_devmap()
> >
> > (private mapping read)
> > __handle_mm_fault()
> > create_huge_pmd()
> > dax_iomap_pmd_fault() inserts PMD
> >
> > dax_iomap_pte_fault() does a PTE fault, but we already have a DAX PMD
> > installed in our page tables at this spot.
> >
> >
> > Here's the second race:
> >
> > CPU 0 CPU 1
> >
> > (private mapping write)
> > __handle_mm_fault()
> > create_huge_pmd() - FALLBACK
> > (private mapping read)
> > __handle_mm_fault()
> > passes check for pmd_none()
> > create_huge_pmd()
> >
> > handle_pte_fault()
> > dax_iomap_pte_fault() inserts PTE
> > dax_iomap_pmd_fault() inserts PMD,
> > but we already have a PTE at
> > this spot.
>
> So I don't see how this second scenario can happen. dax_iomap_pmd_fault()
> will call grab_mapping_entry(). That will either find PTE entry in the
> radix tree -> EEXIST and we retry the fault. Or we will not find PTE entry
> -> try to insert PMD entry which collides with the PTE entry -> EEXIST and
> we retry the fault. Am I missing something?
Yep, sorry, I guess I needed a few extra steps in my flow (the initial private
mapping read by CPU 0):
CPU 0 CPU 1
(private mapping read)
__handle_mm_fault()
passes check for pmd_none()
create_huge_pmd()
dax_iomap_pmd_fault() inserts PMD
(private mapping write)
__handle_mm_fault()
create_huge_pmd() - FALLBACK
(private mapping read)
__handle_mm_fault()
passes check for pmd_none()
create_huge_pmd()
handle_pte_fault()
dax_iomap_pte_fault() inserts PTE
dax_iomap_pmd_fault() inserts PMD,
but we already have a PTE at
this spot.
So what happens is that CPU 0 inserts a DAX PMD into the radix tree that has
real storage backing, and all PTE and PMD faults just use that same PMD radix
tree entry for locking and dirty tracking.
> The first scenario seems to be possible. dax_iomap_pmd_fault() will create
> PMD entry in the radix tree. Then dax_iomap_pte_fault() will come, do
> grab_mapping_entry(), there it sees entry is PMD but we are doing PTE fault
> so I'd think that pmd_downgrade = true... But actually the condition there
> doesn't trigger in this case. And that's a catch that although we asked
> grab_mapping_entry() for PTE, we've got PMD back and that screws us later.
Yep, it was a concious decision when implementing the PMD support to allow one
thread to use PMDs and another to use PTEs in the same range, as long as the
thread faulting in PMDs is the first to insert into the radix tree. A PMD
radix tree entry will be inserted and used for locking and dirty tracking, and
each thread or process can fault in either PTEs or PMDs into its own address
space as needed.
We can revisit this, if you think it is incorrect. The option you outline
below would basically mean that if any thread were to fault in a PTE in a
range, all threads and processes would be forced to use PTEs because we would
use PTEs in the radix tree.
This is cleaner...I'm not sure if the use case of having two threads accessing
the same area, one with PTEs and one with PMDs, is actually prevalent. It's
also maybe a bit weird that the current behavior varies based on which thread
faulted first - if the PTE thread faults first, it'll insert a PTE into the
radix tree and everyone will just use PTEs.
> Actually I'm not convinced your patch quite fixes this because
> dax_load_hole() or dax_insert_mapping_entry() will modify the passed entry
> with the assumption that it's PTE entry and so they will likely corrupt the
> entry in the radix tree.
I don't think we can ever call dax_load_hole() if we have a DAX PMD entry in
the radix tree, because we have a block mapping from the filesystem.
For dax_insert_mapping_entry(), we do the right thing. From the comments
above the function:
* If we happen to be trying to insert a PTE and there is a PMD
* already in the tree, we will skip the insertion and just dirty the PMD as
* appropriate. If we happen to be trying to insert a PTE and there is a PMD
* already in the tree, we will skip the insertion and just dirty the PMD as
* appropriate.
> So I think to fix the first case we should rather modify
> grab_mapping_entry() to properly go through the pmd_downgrade path once we
> find PMD entry and we do PTE fault.
>
> What do you think?
That could also work, though I do think the fix as submitted is correct.
I think it comes down to whether we want to keep the behavior where a thread
faulting in a PTEs will use an existing PMD entry in the radix tree, instead
of making all other threads fall back to PTEs.
I think either way solves this issue for the DAX case...but do you understand
how this is solved for other fault handlers? They don't have any isolation
between faults either in the mm/memory.c code, and are susceptible to the same
races. How do they deal with the fact that by the time they get to their PTE
fault handler, a racing PMD fault handler in another thread could have
inserted a PMD into their page tables, and vice versa?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [mmotm:master 61/139] include/asm-generic/memory_model.h:54:52: warning: 'page' is used uninitialized in this function
From: kbuild test robot @ 2017-05-18 23:48 UTC (permalink / raw)
To: Michal Hocko
Cc: kbuild-all, Johannes Weiner, Andrew Morton,
Linux Memory Management List
[-- Attachment #1: Type: text/plain, Size: 3775 bytes --]
tree: git://git.cmpxchg.org/linux-mmotm.git master
head: ecfddce33be51ebedf97f1e03d954c14e575afb6
commit: d9f7aec054d6fdbf8f9c4f37c269bf7fff4c04bd [61/139] mm, vmstat: skip reporting offline pages in pagetypeinfo
config: x86_64-kexec (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
git checkout d9f7aec054d6fdbf8f9c4f37c269bf7fff4c04bd
# save the attached .config to linux build tree
make ARCH=x86_64
All warnings (new ones prefixed by >>):
In file included from arch/x86/include/asm/page.h:75:0,
from arch/x86/include/asm/thread_info.h:11,
from include/linux/thread_info.h:37,
from arch/x86/include/asm/preempt.h:6,
from include/linux/preempt.h:80,
from include/linux/spinlock.h:50,
from include/linux/wait.h:8,
from include/linux/fs.h:5,
from mm/vmstat.c:12:
mm/vmstat.c: In function 'pagetypeinfo_showblockcount_print':
>> include/asm-generic/memory_model.h:54:52: warning: 'page' is used uninitialized in this function [-Wuninitialized]
#define __page_to_pfn(page) (unsigned long)((page) - vmemmap)
^
mm/vmstat.c:1224:16: note: 'page' was declared here
struct page *page;
^~~~
vim +/page +54 include/asm-generic/memory_model.h
a117e66e KAMEZAWA Hiroyuki 2006-03-27 38 ({ unsigned long __pfn = (pfn); \
c5d71243 Rafael J. Wysocki 2008-11-08 39 unsigned long __nid = arch_pfn_to_nid(__pfn); \
a117e66e KAMEZAWA Hiroyuki 2006-03-27 40 NODE_DATA(__nid)->node_mem_map + arch_local_page_offset(__pfn, __nid);\
a117e66e KAMEZAWA Hiroyuki 2006-03-27 41 })
a117e66e KAMEZAWA Hiroyuki 2006-03-27 42
67de6482 Andy Whitcroft 2006-06-23 43 #define __page_to_pfn(pg) \
aa462abe Ian Campbell 2011-08-17 44 ({ const struct page *__pg = (pg); \
a0140c1d KAMEZAWA Hiroyuki 2006-03-27 45 struct pglist_data *__pgdat = NODE_DATA(page_to_nid(__pg)); \
a0140c1d KAMEZAWA Hiroyuki 2006-03-27 46 (unsigned long)(__pg - __pgdat->node_mem_map) + \
a0140c1d KAMEZAWA Hiroyuki 2006-03-27 47 __pgdat->node_start_pfn; \
a117e66e KAMEZAWA Hiroyuki 2006-03-27 48 })
a117e66e KAMEZAWA Hiroyuki 2006-03-27 49
8f6aac41 Christoph Lameter 2007-10-16 50 #elif defined(CONFIG_SPARSEMEM_VMEMMAP)
8f6aac41 Christoph Lameter 2007-10-16 51
af901ca1 Andre Goddard Rosa 2009-11-14 52 /* memmap is virtually contiguous. */
8f6aac41 Christoph Lameter 2007-10-16 53 #define __pfn_to_page(pfn) (vmemmap + (pfn))
32272a26 Martin Schwidefsky 2008-12-25 @54 #define __page_to_pfn(page) (unsigned long)((page) - vmemmap)
8f6aac41 Christoph Lameter 2007-10-16 55
a117e66e KAMEZAWA Hiroyuki 2006-03-27 56 #elif defined(CONFIG_SPARSEMEM)
a117e66e KAMEZAWA Hiroyuki 2006-03-27 57 /*
1a49123b Zhang Yanfei 2013-10-03 58 * Note: section's mem_map is encoded to reflect its start_pfn.
a117e66e KAMEZAWA Hiroyuki 2006-03-27 59 * section[i].section_mem_map == mem_map's address - start_pfn;
a117e66e KAMEZAWA Hiroyuki 2006-03-27 60 */
67de6482 Andy Whitcroft 2006-06-23 61 #define __page_to_pfn(pg) \
aa462abe Ian Campbell 2011-08-17 62 ({ const struct page *__pg = (pg); \
:::::: The code at line 54 was first introduced by commit
:::::: 32272a26974d2027384fd4010cd1780fca425d94 [S390] __page_to_pfn warnings
:::::: TO: Martin Schwidefsky <schwidefsky@de.ibm.com>
:::::: CC: Martin Schwidefsky <schwidefsky@de.ibm.com>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 25361 bytes --]
^ permalink raw reply
* [mmotm:master 61/139] mm/vmstat.c:1224:16: warning: 'page' may be used uninitialized in this function
From: kbuild test robot @ 2017-05-19 0:07 UTC (permalink / raw)
To: Michal Hocko
Cc: kbuild-all, Johannes Weiner, Andrew Morton,
Linux Memory Management List
[-- Attachment #1: Type: text/plain, Size: 3010 bytes --]
tree: git://git.cmpxchg.org/linux-mmotm.git master
head: ecfddce33be51ebedf97f1e03d954c14e575afb6
commit: d9f7aec054d6fdbf8f9c4f37c269bf7fff4c04bd [61/139] mm, vmstat: skip reporting offline pages in pagetypeinfo
config: i386-defconfig (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
git checkout d9f7aec054d6fdbf8f9c4f37c269bf7fff4c04bd
# save the attached .config to linux build tree
make ARCH=i386
Note: it may well be a FALSE warning. FWIW you are at least aware of it now.
http://gcc.gnu.org/wiki/Better_Uninitialized_Warnings
All warnings (new ones prefixed by >>):
mm/vmstat.c: In function 'pagetypeinfo_showblockcount_print':
>> mm/vmstat.c:1224:16: warning: 'page' may be used uninitialized in this function [-Wmaybe-uninitialized]
struct page *page;
^~~~
vim +/page +1224 mm/vmstat.c
467c996c Mel Gorman 2007-10-16 1208
b2bd8598 David Rientjes 2017-05-03 1209 walk_zones_in_node(m, pgdat, true, pagetypeinfo_showfree_print);
467c996c Mel Gorman 2007-10-16 1210
467c996c Mel Gorman 2007-10-16 1211 return 0;
467c996c Mel Gorman 2007-10-16 1212 }
467c996c Mel Gorman 2007-10-16 1213
467c996c Mel Gorman 2007-10-16 1214 static void pagetypeinfo_showblockcount_print(struct seq_file *m,
467c996c Mel Gorman 2007-10-16 1215 pg_data_t *pgdat, struct zone *zone)
467c996c Mel Gorman 2007-10-16 1216 {
467c996c Mel Gorman 2007-10-16 1217 int mtype;
467c996c Mel Gorman 2007-10-16 1218 unsigned long pfn;
467c996c Mel Gorman 2007-10-16 1219 unsigned long start_pfn = zone->zone_start_pfn;
108bcc96 Cody P Schafer 2013-02-22 1220 unsigned long end_pfn = zone_end_pfn(zone);
467c996c Mel Gorman 2007-10-16 1221 unsigned long count[MIGRATE_TYPES] = { 0, };
467c996c Mel Gorman 2007-10-16 1222
467c996c Mel Gorman 2007-10-16 1223 for (pfn = start_pfn; pfn < end_pfn; pfn += pageblock_nr_pages) {
467c996c Mel Gorman 2007-10-16 @1224 struct page *page;
467c996c Mel Gorman 2007-10-16 1225
d9f7aec0 Michal Hocko 2017-05-18 1226 if (!pfn_to_online_page(pfn))
467c996c Mel Gorman 2007-10-16 1227 continue;
467c996c Mel Gorman 2007-10-16 1228
eb33575c Mel Gorman 2009-05-13 1229 /* Watch for unexpected holes punched in the memmap */
eb33575c Mel Gorman 2009-05-13 1230 if (!memmap_valid_within(pfn, page, zone))
e80d6a24 Mel Gorman 2008-08-14 1231 continue;
eb33575c Mel Gorman 2009-05-13 1232
:::::: The code at line 1224 was first introduced by commit
:::::: 467c996c1e1910633fa8e7adc9b052aa3ed5f97c Print out statistics in relation to fragmentation avoidance to /proc/pagetypeinfo
:::::: TO: Mel Gorman <mel@csn.ul.ie>
:::::: CC: Linus Torvalds <torvalds@woody.linux-foundation.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26204 bytes --]
^ permalink raw reply
* Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption
From: Joonsoo Kim @ 2017-05-19 1:53 UTC (permalink / raw)
To: Andrey Ryabinin
Cc: Andrew Morton, Alexander Potapenko, Dmitry Vyukov, kasan-dev,
linux-mm, linux-kernel, Thomas Gleixner, Ingo Molnar,
H . Peter Anvin, kernel-team
In-Reply-To: <ebcc02d9-fa2b-30b1-2260-99cdf7434487@virtuozzo.com>
On Wed, May 17, 2017 at 03:17:13PM +0300, Andrey Ryabinin wrote:
> On 05/16/2017 04:16 AM, js1304@gmail.com wrote:
> > From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> >
> > Hello, all.
> >
> > This is an attempt to recude memory consumption of KASAN. Please see
> > following description to get the more information.
> >
> > 1. What is per-page shadow memory
> >
> > This patch introduces infrastructure to support per-page shadow memory.
> > Per-page shadow memory is the same with original shadow memory except
> > the granualarity. It's one byte shows the shadow value for the page.
> > The purpose of introducing this new shadow memory is to save memory
> > consumption.
> >
> > 2. Problem of current approach
> >
> > Until now, KASAN needs shadow memory for all the range of the memory
> > so the amount of statically allocated memory is so large. It causes
> > the problem that KASAN cannot run on the system with hard memory
> > constraint. Even if KASAN can run, large memory consumption due to
> > KASAN changes behaviour of the workload so we cannot validate
> > the moment that we want to check.
> >
> > 3. How does this patch fix the problem
> >
> > This patch tries to fix the problem by reducing memory consumption for
> > the shadow memory. There are two observations.
> >
>
>
> I think that the best way to deal with your problem is to increase shadow scale size.
>
> You'll need to add tunable to gcc to control shadow size. I expect that gcc has some
> places where 8-shadow scale size is hardcoded, but it should be fixable.
>
> The kernel also have some small amount of code written with KASAN_SHADOW_SCALE_SIZE == 8 in mind,
> which should be easy to fix.
>
> Note that bigger shadow scale size requires bigger alignment of allocated memory and variables.
> However, according to comments in gcc/asan.c gcc already aligns stack and global variables and at
> 32-bytes boundary.
> So we could bump shadow scale up to 32 without increasing current stack consumption.
>
> On a small machine (1Gb) 1/32 of shadow is just 32Mb which is comparable to yours 30Mb, but I expect it to be
> much faster. More importantly, this will require only small amount of simple changes in code, which will be
> a *lot* more easier to maintain.
I agree that it is also a good option to reduce memory consumption.
Nevertheless, there are two reasons that justifies this patchset.
1) With this patchset, memory consumption isn't increased in
proportional to total memory size. Please consider my 4Gb system
example on the below. With increasing shadow scale size to 32, memory
would be consumed by 128M. However, this patchset consumed 50MB. This
difference can be larger if we run KASAN with bigger machine.
2) These two optimization can be applied simulatenously. It is just an
orthogonal feature. If shadow scale size is increased to 32, memory
consumption will be decreased in case of my patchset, too.
Therefore, I think that this patchset is useful in any case.
Note that increasing shadow scale has it's own trade-off. It requires
that the size of slab object is aligned to shadow scale. It will
increase memory consumption due to slab.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [mmotm:master] BUILD REGRESSION ecfddce33be51ebedf97f1e03d954c14e575afb6
From: kbuild test robot @ 2017-05-19 2:08 UTC (permalink / raw)
To: Andrew Morton; +Cc: Johannes Weiner, Linux Memory Management List
git://git.cmpxchg.org/linux-mmotm.git master
ecfddce33be51ebedf97f1e03d954c14e575afb6 pci: test for unexpectedly disabled bridges
include/asm-generic/memory_model.h:54:52: warning: 'page' is used uninitialized in this function [-Wuninitialized]
include/asm-generic/memory_model.h:54:52: warning: 'page' may be used uninitialized in this function [-Wmaybe-uninitialized]
mm/vmstat.c:1224:16: warning: 'page' may be used uninitialized in this function [-Wmaybe-uninitialized]
mm/vmstat.c:1230:8: warning: 'page' may be used uninitialized in this function [-Wmaybe-uninitialized]
Error ids grouped by kconfigs:
recent_errors
a??a??a?? alpha-allyesconfig
a??A A a??a??a?? mm-vmstat.c:warning:page-may-be-used-uninitialized-in-this-function
a??a??a?? arm64-defconfig
a??A A a??a??a?? mm-vmstat.c:warning:page-may-be-used-uninitialized-in-this-function
a??a??a?? arm-exynos_defconfig
a??A A a??a??a?? mm-vmstat.c:warning:page-may-be-used-uninitialized-in-this-function
a??a??a?? arm-multi_v7_defconfig
a??A A a??a??a?? mm-vmstat.c:warning:page-may-be-used-uninitialized-in-this-function
a??a??a?? i386-allmodconfig
a??A A a??a??a?? mm-vmstat.c:warning:page-may-be-used-uninitialized-in-this-function
a??a??a?? i386-defconfig
a??A A a??a??a?? mm-vmstat.c:warning:page-may-be-used-uninitialized-in-this-function
a??a??a?? x86_64-allmodconfig
a??A A a??a??a?? mm-vmstat.c:warning:page-may-be-used-uninitialized-in-this-function
a??a??a?? x86_64-kexec
a??A A a??a??a?? include-asm-generic-memory_model.h:warning:page-is-used-uninitialized-in-this-function
a??a??a?? x86_64-rhel
a??a??a?? include-asm-generic-memory_model.h:warning:page-may-be-used-uninitialized-in-this-function
elapsed time: 238m
configs tested: 112
i386 tinyconfig
x86_64 allmodconfig
arm at91_dt_defconfig
arm allnoconfig
arm efm32_defconfig
arm64 defconfig
arm multi_v5_defconfig
arm sunxi_defconfig
arm64 allnoconfig
arm exynos_defconfig
arm shmobile_defconfig
arm multi_v7_defconfig
powerpc defconfig
s390 default_defconfig
powerpc ppc64_defconfig
powerpc allnoconfig
x86_64 acpi-redef
x86_64 allyesdebian
x86_64 nfsroot
ia64 allnoconfig
ia64 defconfig
ia64 alldefconfig
x86_64 kexec
x86_64 rhel
x86_64 rhel-7.2
i386 randconfig-a0-05181312
mn10300 asb2364_defconfig
openrisc or1ksim_defconfig
um x86_64_defconfig
um i386_defconfig
frv defconfig
tile tilegx_defconfig
c6x evmc6678_defconfig
xtensa common_defconfig
m32r m32104ut_defconfig
xtensa iss_defconfig
m32r opsput_defconfig
m32r usrv_defconfig
m32r mappi3.smp_defconfig
nios2 10m50_defconfig
h8300 h8300h-sim_defconfig
parisc c3000_defconfig
parisc b180_defconfig
parisc defconfig
alpha defconfig
parisc allnoconfig
cris etrax-100lx_v2_defconfig
blackfin TCM-BF537_defconfig
blackfin BF561-EZKIT-SMP_defconfig
blackfin BF533-EZKIT_defconfig
blackfin BF526-EZBRD_defconfig
x86_64 randconfig-x019-201720
x86_64 randconfig-x010-201720
x86_64 randconfig-x015-201720
x86_64 randconfig-x014-201720
x86_64 randconfig-x012-201720
x86_64 randconfig-x018-201720
x86_64 randconfig-x017-201720
x86_64 randconfig-x016-201720
x86_64 randconfig-x013-201720
x86_64 randconfig-x011-201720
m68k sun3_defconfig
m68k multi_defconfig
m68k m5475evb_defconfig
mips jz4740
mips malta_kvm_defconfig
mips 64r6el_defconfig
mips 32r2_defconfig
mips allnoconfig
mips fuloong2e_defconfig
mips txx9
sparc defconfig
sparc64 allnoconfig
sparc64 defconfig
i386 randconfig-x016-201720
i386 randconfig-x017-201720
i386 randconfig-x012-201720
i386 randconfig-x018-201720
i386 randconfig-x010-201720
i386 randconfig-x019-201720
i386 randconfig-x013-201720
i386 randconfig-x014-201720
i386 randconfig-x015-201720
i386 randconfig-x011-201720
sh titan_defconfig
sh rsk7269_defconfig
sh sh7785lcr_32bit_defconfig
sh allnoconfig
i386 randconfig-x074-05150639
i386 randconfig-x076-05150639
i386 randconfig-x070-05150639
i386 randconfig-x077-05150639
i386 randconfig-x078-05150639
i386 randconfig-x075-05150639
i386 randconfig-x072-05150639
i386 randconfig-x071-05150639
i386 randconfig-x079-05150639
i386 randconfig-x073-05150639
i386 allnoconfig
i386 defconfig
i386 alldefconfig
x86_64 randconfig-x006-201720
x86_64 randconfig-x007-201720
x86_64 randconfig-x001-201720
x86_64 randconfig-x004-201720
x86_64 randconfig-x005-201720
x86_64 randconfig-x000-201720
x86_64 randconfig-x008-201720
x86_64 randconfig-x002-201720
x86_64 randconfig-x003-201720
x86_64 randconfig-x009-201720
i386 allmodconfig
Thanks,
Fengguang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] dm ioctl: Restore __GFP_HIGH in copy_params()
From: Junaid Shahid @ 2017-05-19 2:50 UTC (permalink / raw)
To: David Rientjes
Cc: Michal Hocko, Alasdair Kergon, Mike Snitzer, Andrew Morton,
linux-mm, andreslc, gthelen, mpatocka, vbabka, linux-kernel
In-Reply-To: <alpine.DEB.2.10.1705181338090.132717@chino.kir.corp.google.com>
(Adding back the correct linux-mm email address and also adding linux-kernel.)
On Thursday, May 18, 2017 01:41:33 PM David Rientjes wrote:
> On Thu, 18 May 2017, Michal Hocko wrote:
>
> > On Thu 18-05-17 11:50:40, Junaid Shahid wrote:
> > > d224e9381897 (drivers/md/dm-ioctl.c: use kvmalloc rather than opencoded
> > > variant) left out the __GFP_HIGH flag when converting from __vmalloc to
> > > kvmalloc. This can cause the IOCTL to fail in some low memory situations
> > > where it wouldn't have failed earlier. This patch adds it back to avoid
> > > any potential regression.
> >
> > The code previously used __GFP_HIGH only for the vmalloc fallback and
> > that doesn't make that much sense with the current implementation
> > because vmalloc does order-0 pages and those do not really fail and the
> > oom killer is invoked to free memory.
> >
>
> Order-0 pages certainly do fail, there is not an infinite amount of memory
> nor is there a specific exemption to allow order-0 memory to be alloctable
> below watermarks without this gfp flag. OOM kill is the last thing we
> want for these allocations since they are very temporary.
>
> > There is no reason to access memory reserves from this context.
> >
>
> Let's ask Mikulas, who changed this from PF_MEMALLOC to __GFP_HIGH,
> assuming there was a reason to do it in the first place in two different
> ways.
>
> This decision is up to the device mapper maintainers.
>
> > > Signed-off-by: Junaid Shahid <junaids@google.com>
> > > ---
> > > drivers/md/dm-ioctl.c | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c
> > > index 0555b4410e05..bacad7637a56 100644
> > > --- a/drivers/md/dm-ioctl.c
> > > +++ b/drivers/md/dm-ioctl.c
> > > @@ -1715,7 +1715,7 @@ static int copy_params(struct dm_ioctl __user *user, struct dm_ioctl *param_kern
> > > */
> > > dmi = NULL;
> > > noio_flag = memalloc_noio_save();
> > > - dmi = kvmalloc(param_kernel->data_size, GFP_KERNEL);
> > > + dmi = kvmalloc(param_kernel->data_size, GFP_KERNEL | __GFP_HIGH);
> > > memalloc_noio_restore(noio_flag);
> > >
> > > if (!dmi) {
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v4 05/27] btrfs: btrfs_wait_tree_block_writeback can be void return
From: Liu Bo @ 2017-05-19 4:07 UTC (permalink / raw)
To: Jeff Layton
Cc: linux-fsdevel, linux-kernel, linux-btrfs, linux-ext4, linux-cifs,
linux-nfs, linux-mm, jfs-discussion, linux-xfs, cluster-devel,
linux-f2fs-devel, v9fs-developer, linux-nilfs, linux-block,
dhowells, akpm, hch, ross.zwisler, mawilcox, jack, viro, corbet,
neilb, clm, tytso, axboe, josef, hubcap, rpeterso
In-Reply-To: <20170509154930.29524-6-jlayton@redhat.com>
On Tue, May 09, 2017 at 11:49:08AM -0400, Jeff Layton wrote:
> Nothing checks its return value.
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
-liubo
>
> Signed-off-by: Jeff Layton <jlayton@redhat.com>
> ---
> fs/btrfs/disk-io.c | 6 +++---
> fs/btrfs/disk-io.h | 2 +-
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
> index eb1ee7b6f532..8c479bd5534a 100644
> --- a/fs/btrfs/disk-io.c
> +++ b/fs/btrfs/disk-io.c
> @@ -1222,10 +1222,10 @@ int btrfs_write_tree_block(struct extent_buffer *buf)
> buf->start + buf->len - 1);
> }
>
> -int btrfs_wait_tree_block_writeback(struct extent_buffer *buf)
> +void btrfs_wait_tree_block_writeback(struct extent_buffer *buf)
> {
> - return filemap_fdatawait_range(buf->pages[0]->mapping,
> - buf->start, buf->start + buf->len - 1);
> + filemap_fdatawait_range(buf->pages[0]->mapping,
> + buf->start, buf->start + buf->len - 1);
> }
>
> struct extent_buffer *read_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr,
> diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h
> index 2e0ec29bfd69..9cc87835abb5 100644
> --- a/fs/btrfs/disk-io.h
> +++ b/fs/btrfs/disk-io.h
> @@ -127,7 +127,7 @@ int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode,
> extent_submit_bio_hook_t *submit_bio_done);
> unsigned long btrfs_async_submit_limit(struct btrfs_fs_info *info);
> int btrfs_write_tree_block(struct extent_buffer *buf);
> -int btrfs_wait_tree_block_writeback(struct extent_buffer *buf);
> +void btrfs_wait_tree_block_writeback(struct extent_buffer *buf);
> int btrfs_init_log_root_tree(struct btrfs_trans_handle *trans,
> struct btrfs_fs_info *fs_info);
> int btrfs_add_log_tree(struct btrfs_trans_handle *trans,
> --
> 2.9.3
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH] mm/vmstat: add oom_kill counter
From: Konstantin Khlebnikov @ 2017-05-19 6:59 UTC (permalink / raw)
To: linux-mm, Andrew Morton, linux-kernel
Show count of global oom killer invocations in /proc/vmstat
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
include/linux/vm_event_item.h | 1 +
mm/oom_kill.c | 1 +
mm/vmstat.c | 1 +
3 files changed, 3 insertions(+)
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index d84ae90ccd5c..1707e0a7d943 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -41,6 +41,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
PAGEOUTRUN, PGROTATED,
DROP_PAGECACHE, DROP_SLAB,
+ OOM_KILL,
#ifdef CONFIG_NUMA_BALANCING
NUMA_PTE_UPDATES,
NUMA_HUGE_PTE_UPDATES,
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 04c9143a8625..c734c42826cf 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -883,6 +883,7 @@ static void oom_kill_process(struct oom_control *oc, const char *message)
*/
do_send_sig_info(SIGKILL, SEND_SIG_FORCED, victim, true);
mark_oom_victim(victim);
+ count_vm_event(OOM_KILL);
pr_err("Killed process %d (%s) total-vm:%lukB, anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n",
task_pid_nr(victim), victim->comm, K(victim->mm->total_vm),
K(get_mm_counter(victim->mm, MM_ANONPAGES)),
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 76f73670200a..fe80b81a86e0 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1018,6 +1018,7 @@ const char * const vmstat_text[] = {
"drop_pagecache",
"drop_slab",
+ "oom_kill",
#ifdef CONFIG_NUMA_BALANCING
"numa_pte_updates",
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH] ext4: handle the rest of ext4_mb_load_buddy() ENOMEM errors
From: Konstantin Khlebnikov @ 2017-05-19 7:09 UTC (permalink / raw)
To: Andreas Dilger, linux-ext4, Theodore Ts'o, linux-kernel; +Cc: linux-mm
I've got another report about breaking ext4 by ENOMEM error returned from
ext4_mb_load_buddy() caused by memory shortage in memory cgroup.
This time inside ext4_discard_preallocations().
This patch replaces ext4_error() with ext4_warning() where errors returned
from ext4_mb_load_buddy() are not fatal and handled by caller:
* ext4_mb_discard_group_preallocations() - called before generating ENOSPC,
we'll try to discard other group or return ENOSPC into user-space.
* ext4_trim_all_free() - just stop trimming and return ENOMEM from ioctl.
Some callers cannot handle errors, thus __GFP_NOFAIL is used for them:
* ext4_discard_preallocations()
* ext4_mb_discard_lg_preallocations()
The only unclear case is ext4_group_add_blocks(), probably ext4_std_error()
should handle ENOMEM as warning and don't break filesystem.
Fixes: adb7ef600cc9 ("ext4: use __GFP_NOFAIL in ext4_free_blocks()")
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
fs/ext4/mballoc.c | 23 ++++++++++++++---------
1 file changed, 14 insertions(+), 9 deletions(-)
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index 5083bce20ac4..b7928cddd539 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -3887,7 +3887,8 @@ ext4_mb_discard_group_preallocations(struct super_block *sb,
err = ext4_mb_load_buddy(sb, group, &e4b);
if (err) {
- ext4_error(sb, "Error loading buddy information for %u", group);
+ ext4_warning(sb, "Error %d loading buddy information for %u",
+ err, group);
put_bh(bitmap_bh);
return 0;
}
@@ -4044,10 +4045,11 @@ void ext4_discard_preallocations(struct inode *inode)
BUG_ON(pa->pa_type != MB_INODE_PA);
group = ext4_get_group_number(sb, pa->pa_pstart);
- err = ext4_mb_load_buddy(sb, group, &e4b);
+ err = ext4_mb_load_buddy_gfp(sb, group, &e4b,
+ GFP_NOFS|__GFP_NOFAIL);
if (err) {
- ext4_error(sb, "Error loading buddy information for %u",
- group);
+ ext4_error(sb, "Error %d loading buddy information for %u",
+ err, group);
continue;
}
@@ -4303,11 +4305,14 @@ ext4_mb_discard_lg_preallocations(struct super_block *sb,
spin_unlock(&lg->lg_prealloc_lock);
list_for_each_entry_safe(pa, tmp, &discard_list, u.pa_tmp_list) {
+ int err;
group = ext4_get_group_number(sb, pa->pa_pstart);
- if (ext4_mb_load_buddy(sb, group, &e4b)) {
- ext4_error(sb, "Error loading buddy information for %u",
- group);
+ err = ext4_mb_load_buddy_gfp(sb, group, &e4b,
+ GFP_NOFS|__GFP_NOFAIL);
+ if (err) {
+ ext4_error(sb, "Error %d loading buddy information for %u",
+ err, group);
continue;
}
ext4_lock_group(sb, group);
@@ -5127,8 +5132,8 @@ ext4_trim_all_free(struct super_block *sb, ext4_group_t group,
ret = ext4_mb_load_buddy(sb, group, &e4b);
if (ret) {
- ext4_error(sb, "Error in loading buddy "
- "information for %u", group);
+ ext4_warning(sb, "Error %d loading buddy information for %u",
+ ret, group);
return ret;
}
bitmap = e4b.bd_bitmap;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH 07/14] mm: consider zone which is not fully populated to have holes
From: Vlastimil Babka @ 2017-05-19 7:21 UTC (permalink / raw)
To: Michal Hocko
Cc: Andrew Morton, linux-mm, Mel Gorman, Andrea Arcangeli,
Jerome Glisse, Reza Arbab, Yasuaki Ishimatsu, qiuxishi,
Kani Toshimitsu, slaoub, Joonsoo Kim, Andi Kleen, David Rientjes,
Daniel Kiper, Igor Mammedov, Vitaly Kuznetsov, LKML
In-Reply-To: <20170518164210.GD18333@dhcp22.suse.cz>
On 05/18/2017 06:42 PM, Michal Hocko wrote:
> On Thu 18-05-17 18:14:39, Vlastimil Babka wrote:
>> On 05/15/2017 10:58 AM, Michal Hocko wrote:
> [...]
>>> #ifdef CONFIG_MEMORY_HOTPLUG
>>> +/*
>>> + * Return page for the valid pfn only if the page is online. All pfn
>>> + * walkers which rely on the fully initialized page->flags and others
>>> + * should use this rather than pfn_valid && pfn_to_page
>>> + */
>>> +#define pfn_to_online_page(pfn) \
>>> +({ \
>>> + struct page *___page = NULL; \
>>> + \
>>> + if (online_section_nr(pfn_to_section_nr(pfn))) \
>>> + ___page = pfn_to_page(pfn); \
>>> + ___page; \
>>> +})
>>
>> This seems to be already assuming pfn_valid() to be true. There's no
>> "pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS" check and the comment
>> suggests as such, but...
>
> Yes, we should check the validity of the section number. We do not have
> to check whether the section is valid because online sections are a
> subset of those that are valid.
>
>>> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
>>> index 05796ee974f7..c3a146028ba6 100644
>>> --- a/mm/memory_hotplug.c
>>> +++ b/mm/memory_hotplug.c
>>> @@ -929,6 +929,9 @@ static int online_pages_range(unsigned long start_pfn, unsigned long nr_pages,
>>> unsigned long i;
>>> unsigned long onlined_pages = *(unsigned long *)arg;
>>> struct page *page;
>>> +
>>> + online_mem_sections(start_pfn, start_pfn + nr_pages);
>>
>> Shouldn't this be moved *below* the loop that initializes struct pages?
>> In the offline case you do mark sections offline before "tearing" struct
>> pages, so that should be symmetric.
>
> You are right! Andrew, could you fold the following intot the patch?
> ---
> From 0550b61203d6970b47fd79f5e6372dccd143cbec Mon Sep 17 00:00:00 2001
> From: Michal Hocko <mhocko@suse.com>
> Date: Thu, 18 May 2017 18:38:24 +0200
> Subject: [PATCH] fold me "mm: consider zone which is not fully populated to
> have holes"
>
> - check valid section number in pfn_to_online_page - Vlastimil
> - mark sections online after all struct pages are initialized in
> online_pages_range - Vlastimil
>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
Both the patch and fix:
Acked-by: Vlastimil Babka <vbabka@suse.cz>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [mmotm:master 61/139] include/asm-generic/memory_model.h:54:52: warning: 'page' is used uninitialized in this function
From: Michal Hocko @ 2017-05-19 7:22 UTC (permalink / raw)
To: Andrew Morton
Cc: kbuild-all, Johannes Weiner, kbuild test robot,
Linux Memory Management List
In-Reply-To: <201705190738.Nkd5Ar0X%fengguang.wu@intel.com>
Ups, sorry about that. Andrew could you fold the following into
mm-vmstat-skip-reporting-offline-pages-in-pagetypeinfo.patch
---
^ permalink raw reply
* Re: [PATCH 08/14] mm, compaction: skip over holes in __reset_isolation_suitable
From: Vlastimil Babka @ 2017-05-19 7:23 UTC (permalink / raw)
To: Michal Hocko, Andrew Morton
Cc: linux-mm, Mel Gorman, Andrea Arcangeli, Jerome Glisse, Reza Arbab,
Yasuaki Ishimatsu, qiuxishi, Kani Toshimitsu, slaoub, Joonsoo Kim,
Andi Kleen, David Rientjes, Daniel Kiper, Igor Mammedov,
Vitaly Kuznetsov, LKML, Michal Hocko
In-Reply-To: <20170515085827.16474-9-mhocko@kernel.org>
On 05/15/2017 10:58 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> __reset_isolation_suitable walks the whole zone pfn range and it tries
> to jump over holes by checking the zone for each page. It might still
> stumble over offline pages, though. Skip those by checking
> pfn_to_online_page()
>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
> ---
> mm/compaction.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/mm/compaction.c b/mm/compaction.c
> index 613c59e928cb..fb548e4c7bd4 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -236,10 +236,9 @@ static void __reset_isolation_suitable(struct zone *zone)
>
> cond_resched();
>
> - if (!pfn_valid(pfn))
> + page = pfn_to_online_page(pfn);
> + if (!page)
> continue;
> -
> - page = pfn_to_page(pfn);
> if (zone != page_zone(page))
> continue;
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 1/6] mm, page_alloc: fix more premature OOM due to race with cpuset update
From: Michal Hocko @ 2017-05-19 7:37 UTC (permalink / raw)
To: Christoph Lameter
Cc: Vlastimil Babka, linux-mm, linux-kernel, cgroups, Li Zefan,
Mel Gorman, David Rientjes, Hugh Dickins, Andrea Arcangeli,
Anshuman Khandual, Kirill A. Shutemov, linux-api
In-Reply-To: <alpine.DEB.2.20.1705181351120.29348@east.gentwo.org>
On Thu 18-05-17 14:07:45, Cristopher Lameter wrote:
> On Thu, 18 May 2017, Michal Hocko wrote:
>
> > > See above. OOM Kill in a cpuset does not kill an innocent task but a task
> > > that does an allocation in that specific context meaning a task in that
> > > cpuset that also has a memory policty.
> >
> > No, the oom killer will chose the largest task in the specific NUMA
> > domain. If you just fail such an allocation then a page fault would get
> > VM_FAULT_OOM and pagefault_out_of_memory would kill a task regardless of
> > the cpusets.
>
> Ok someone screwed up that code. There still is the determination that we
> have a constrained alloc:
It would be much more easier if you read emails more carefully. In order
to have a constrained OOM you have to have either a non-null nodemask or
zonelist which. And as I've said above you do not have them from the
pagefault_out_of_memory context. The whole point of this discussion is
_that_ failing allocations will not work currently!
> oom_kill:
> /*
> * Check if there were limitations on the allocation (only relevant for
> * NUMA and memcg) that may require different handling.
> */
> constraint = constrained_alloc(oc);
> if (constraint != CONSTRAINT_MEMORY_POLICY)
> oc->nodemask = NULL;
> check_panic_on_oom(oc, constraint);
>
> -- Ok. A constrained failing alloc used to terminate the allocating
> process here. But it falls through to selecting a "bad process"
This behavior is there for ~10 years.
[...]
> Can we restore the old behavior? If I just specify the right memory policy
> I can cause other processes to just be terminated?
Not normally. Because out_of_memory called from the page allocator
context makes sure to kill tasks from the same NUMA domain (see
oom_unkillable_task).
> > > Regardless of that the point earlier was that the moving logic can avoid
> > > creating temporary situations of empty sets of nodes by analysing the
> > > memory policies etc and only performing moves when doing so is safe.
> >
> > How are you going to do that in a raceless way? Moreover the whole
> > discussion is about _failing_ allocations on an empty cpuset and
> > mempolicy intersection.
>
> Again this is only working for processes that are well behaved and it
> never worked in a different way before. There was always the assumption
> that a process does not allocate in the areas that have allocation
> constraints and that the process does not change memory policies nor
> store them somewhere for late etc etc. HPC apps typically allocate memory
> on startup and then go through long times of processing and I/O.
I would call it a bad design which then triggered a lot of work to make
it semi-working over years. This is what Vlastimil tries to address now.
And yes that might mean we would have to do some restrictions on the
semantics. But as you know this is a user visible API and changing
something that has been fundamentally underdefined initially is quite
hard to fix.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] dm ioctl: Restore __GFP_HIGH in copy_params()
From: Michal Hocko @ 2017-05-19 7:46 UTC (permalink / raw)
To: Junaid Shahid
Cc: David Rientjes, Alasdair Kergon, Mike Snitzer, Andrew Morton,
linux-mm, andreslc, gthelen, mpatocka, vbabka, linux-kernel
In-Reply-To: <1508444.i5EqlA1upv@js-desktop.svl.corp.google.com>
On Thu 18-05-17 19:50:46, Junaid Shahid wrote:
> (Adding back the correct linux-mm email address and also adding linux-kernel.)
>
> On Thursday, May 18, 2017 01:41:33 PM David Rientjes wrote:
[...]
> > Let's ask Mikulas, who changed this from PF_MEMALLOC to __GFP_HIGH,
> > assuming there was a reason to do it in the first place in two different
> > ways.
Hmm, the old PF_MEMALLOC used to have the following comment
/*
* Trying to avoid low memory issues when a device is
* suspended.
*/
I am not really sure what that means but __GFP_HIGH certainly have a
different semantic than PF_MEMALLOC. The later grants the full access to
the memory reserves while the prior on partial access. If this is _really_
needed then it deserves a comment explaining why.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer
From: Michal Hocko @ 2017-05-19 8:02 UTC (permalink / raw)
To: Johannes Weiner
Cc: Roman Gushchin, Tejun Heo, Li Zefan, Vladimir Davydov,
Tetsuo Handa, kernel-team, cgroups, linux-doc, linux-kernel,
linux-mm
In-Reply-To: <20170518181117.GA27689@cmpxchg.org>
On Thu 18-05-17 14:11:17, Johannes Weiner wrote:
> On Thu, May 18, 2017 at 07:30:04PM +0200, Michal Hocko wrote:
> > On Thu 18-05-17 17:28:04, Roman Gushchin wrote:
> > > Traditionally, the OOM killer is operating on a process level.
> > > Under oom conditions, it finds a process with the highest oom score
> > > and kills it.
> > >
> > > This behavior doesn't suit well the system with many running
> > > containers. There are two main issues:
> > >
> > > 1) There is no fairness between containers. A small container with
> > > a few large processes will be chosen over a large one with huge
> > > number of small processes.
> > >
> > > 2) Containers often do not expect that some random process inside
> > > will be killed. So, in general, a much safer behavior is
> > > to kill the whole cgroup. Traditionally, this was implemented
> > > in userspace, but doing it in the kernel has some advantages,
> > > especially in a case of a system-wide OOM.
> > >
> > > To address these issues, cgroup-aware OOM killer is introduced.
> > > Under OOM conditions, it looks for a memcg with highest oom score,
> > > and kills all processes inside.
> > >
> > > Memcg oom score is calculated as a size of active and inactive
> > > anon LRU lists, unevictable LRU list and swap size.
> > >
> > > For a cgroup-wide OOM, only cgroups belonging to the subtree of
> > > the OOMing cgroup are considered.
> >
> > While this might make sense for some workloads/setups it is not a
> > generally acceptable policy IMHO. We have discussed that different OOM
> > policies might be interesting few years back at LSFMM but there was no
> > real consensus on how to do that. One possibility was to allow bpf like
> > mechanisms. Could you explore that path?
>
> OOM policy is an orthogonal discussion, though.
>
> The OOM killer's job is to pick a memory consumer to kill. Per default
> the unit of the memory consumer is a process, but cgroups allow
> grouping processes into compound consumers. Extending the OOM killer
> to respect the new definition of "consumer" is not a new policy.
I do not want to play word games here but picking a task or more tasks
is a policy from my POV but that is not all that important. My primary
point is that this new "implementation" is most probably not what people
who use memory cgroups outside of containers want. Why? Mostly because
they do not care that only a part of the memcg is still alive pretty
much like the current global OOM behavior when a single task (or its
children) are gone all of the sudden. Why should I kill the whole user
slice just because one of its processes went wild?
> I don't think it's reasonable to ask the person who's trying to make
> the OOM killer support group-consumers to design a dynamic OOM policy
> framework instead.
>
> All we want is the OOM policy, whatever it is, applied to cgroups.
And I am not dismissing this usecase. I believe it is valid but not
universally applicable when memory cgroups are deployed. That is why
I think that we need a way to define those policies in some sane way.
Our current oom policies are basically random -
/proc/sys/vm/oom_kill_allocating_task resp. /proc/sys/vm/panic_on_oom.
I am not really sure we want another hardcoded one e.g.
/proc/sys/vm/oom_kill_container because even that might turn out not the
great fit for different container usecases. Do we want to kill the
largest container or the one with the largest memory hog? Should some
containers have a higher priority over others? I am pretty sure more
criterion would pop up with more usecases.
That's why I think that the current OOM killer implementation should
stay as a last resort and be process oriented and we should think about
a way to override it for particular usecases. The exact mechanism is not
completely clear to me to be honest.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 09/14] mm: __first_valid_page skip over offline pages
From: Vlastimil Babka @ 2017-05-19 8:05 UTC (permalink / raw)
To: Michal Hocko, Andrew Morton
Cc: linux-mm, Mel Gorman, Andrea Arcangeli, Jerome Glisse, Reza Arbab,
Yasuaki Ishimatsu, qiuxishi, Kani Toshimitsu, slaoub, Joonsoo Kim,
Andi Kleen, David Rientjes, Daniel Kiper, Igor Mammedov,
Vitaly Kuznetsov, LKML, Michal Hocko
In-Reply-To: <20170515085827.16474-10-mhocko@kernel.org>
On 05/15/2017 10:58 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> __first_valid_page skips over invalid pfns in the range but it might
> still stumble over offline pages. At least start_isolate_page_range
> will mark those set_migratetype_isolate. This doesn't represent
> any immediate AFAICS because alloc_contig_range will fail to isolate
> those pages but it relies on not fully initialized page which will
> become a problem later when we stop associating offline pages to zones.
> Use pfn_to_online_page to handle this.
>
> This is more a preparatory patch than a fix.
>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
Iterating over single pages when the whole section is offline seems
rather wasteful, but it should be really rare, so whatever.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
> ---
> mm/page_isolation.c | 26 ++++++++++++++++++--------
> 1 file changed, 18 insertions(+), 8 deletions(-)
>
> diff --git a/mm/page_isolation.c b/mm/page_isolation.c
> index 5092e4ef00c8..3606104893e0 100644
> --- a/mm/page_isolation.c
> +++ b/mm/page_isolation.c
> @@ -138,12 +138,18 @@ static inline struct page *
> __first_valid_page(unsigned long pfn, unsigned long nr_pages)
> {
> int i;
> - for (i = 0; i < nr_pages; i++)
> - if (pfn_valid_within(pfn + i))
> - break;
> - if (unlikely(i == nr_pages))
> - return NULL;
> - return pfn_to_page(pfn + i);
> +
> + for (i = 0; i < nr_pages; i++) {
> + struct page *page;
> +
> + if (!pfn_valid_within(pfn + i))
> + continue;
> + page = pfn_to_online_page(pfn + i);
> + if (!page)
> + continue;
> + return page;
> + }
> + return NULL;
> }
>
> /*
> @@ -184,8 +190,12 @@ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
> undo:
> for (pfn = start_pfn;
> pfn < undo_pfn;
> - pfn += pageblock_nr_pages)
> - unset_migratetype_isolate(pfn_to_page(pfn), migratetype);
> + pfn += pageblock_nr_pages) {
> + struct page *page = pfn_to_online_page(pfn);
> + if (!page)
> + continue;
> + unset_migratetype_isolate(page, migratetype);
> + }
>
> return -EBUSY;
> }
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v6 05/15] lockdep: Implement crossrelease feature
From: Byungchul Park @ 2017-05-19 8:07 UTC (permalink / raw)
To: peterz, mingo
Cc: tglx, walken, boqun.feng, kirill, linux-kernel, linux-mm,
iamjoonsoo.kim, akpm, willy, npiggin, kernel-team
In-Reply-To: <1489479542-27030-6-git-send-email-byungchul.park@lge.com>
On Tue, Mar 14, 2017 at 05:18:52PM +0900, Byungchul Park wrote:
> Lockdep is a runtime locking correctness validator that detects and
> reports a deadlock or its possibility by checking dependencies between
> locks. It's useful since it does not report just an actual deadlock but
> also the possibility of a deadlock that has not actually happened yet.
> That enables problems to be fixed before they affect real systems.
>
> However, this facility is only applicable to typical locks, such as
> spinlocks and mutexes, which are normally released within the context in
> which they were acquired. However, synchronization primitives like page
> locks or completions, which are allowed to be released in any context,
> also create dependencies and can cause a deadlock. So lockdep should
> track these locks to do a better job. The 'crossrelease' implementation
> makes these primitives also be tracked.
Excuse me but I have a question...
Only for maskable irq, can I assume that hardirq are prevented within
hardirq context? I remember that nested interrupts were allowed in the
past but not recommanded. But what about now? I'm curious about the
overall direction of kernel and current status. It would be very
appriciated if you answer it.
Thank you.
Byungchul
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 10/14] mm, vmstat: skip reporting offline pages in pagetypeinfo
From: Vlastimil Babka @ 2017-05-19 8:17 UTC (permalink / raw)
To: Michal Hocko, Andrew Morton
Cc: linux-mm, Mel Gorman, Andrea Arcangeli, Jerome Glisse, Reza Arbab,
Yasuaki Ishimatsu, qiuxishi, Kani Toshimitsu, slaoub, Joonsoo Kim,
Andi Kleen, David Rientjes, Daniel Kiper, Igor Mammedov,
Vitaly Kuznetsov, LKML, Michal Hocko
In-Reply-To: <20170515085827.16474-11-mhocko@kernel.org>
On 05/15/2017 10:58 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> pagetypeinfo_showblockcount_print skips over invalid pfns but it would
> report pages which are offline because those have a valid pfn. Their
> migrate type is misleading at best. Now that we have pfn_to_online_page()
> we can use it instead of pfn_valid() and fix this.
>
> Noticed-by: Joonsoo Kim <js1304@gmail.com>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
(with the followup fix)
Acked-by: Vlastimil Babka <vbabka@suse.cz>
> ---
> mm/vmstat.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 571d3ec05566..c432e581f9a9 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -1223,11 +1223,9 @@ static void pagetypeinfo_showblockcount_print(struct seq_file *m,
> for (pfn = start_pfn; pfn < end_pfn; pfn += pageblock_nr_pages) {
> struct page *page;
>
> - if (!pfn_valid(pfn))
> + if (!pfn_to_online_page(pfn))
> continue;
>
> - page = pfn_to_page(pfn);
> -
> /* Watch for unexpected holes punched in the memmap */
> if (!memmap_valid_within(pfn, page, zone))
> continue;
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm/vmstat: add oom_kill counter
From: Konstantin Khlebnikov @ 2017-05-19 8:22 UTC (permalink / raw)
To: linux-mm, Andrew Morton, linux-kernel
In-Reply-To: <149517718482.32770.939520643229572472.stgit@buzz>
On 19.05.2017 09:59, Konstantin Khlebnikov wrote:
> Show count of global oom killer invocations in /proc/vmstat
Oops, this actually counts memcg kills too. Will redo.
>
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> ---
> include/linux/vm_event_item.h | 1 +
> mm/oom_kill.c | 1 +
> mm/vmstat.c | 1 +
> 3 files changed, 3 insertions(+)
>
> diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
> index d84ae90ccd5c..1707e0a7d943 100644
> --- a/include/linux/vm_event_item.h
> +++ b/include/linux/vm_event_item.h
> @@ -41,6 +41,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
> KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
> PAGEOUTRUN, PGROTATED,
> DROP_PAGECACHE, DROP_SLAB,
> + OOM_KILL,
> #ifdef CONFIG_NUMA_BALANCING
> NUMA_PTE_UPDATES,
> NUMA_HUGE_PTE_UPDATES,
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 04c9143a8625..c734c42826cf 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -883,6 +883,7 @@ static void oom_kill_process(struct oom_control *oc, const char *message)
> */
> do_send_sig_info(SIGKILL, SEND_SIG_FORCED, victim, true);
> mark_oom_victim(victim);
> + count_vm_event(OOM_KILL);
> pr_err("Killed process %d (%s) total-vm:%lukB, anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n",
> task_pid_nr(victim), victim->comm, K(victim->mm->total_vm),
> K(get_mm_counter(victim->mm, MM_ANONPAGES)),
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 76f73670200a..fe80b81a86e0 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -1018,6 +1018,7 @@ const char * const vmstat_text[] = {
>
> "drop_pagecache",
> "drop_slab",
> + "oom_kill",
>
> #ifdef CONFIG_NUMA_BALANCING
> "numa_pte_updates",
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 11/14] mm, memory_hotplug: do not associate hotadded memory to zones until online
From: Vlastimil Babka @ 2017-05-19 8:56 UTC (permalink / raw)
To: Michal Hocko, Andrew Morton
Cc: linux-mm, Mel Gorman, Andrea Arcangeli, Jerome Glisse, Reza Arbab,
Yasuaki Ishimatsu, qiuxishi, Kani Toshimitsu, slaoub, Joonsoo Kim,
Andi Kleen, David Rientjes, Daniel Kiper, Igor Mammedov,
Vitaly Kuznetsov, LKML, Michal Hocko, Dan Williams,
Heiko Carstens, Martin Schwidefsky
In-Reply-To: <20170515085827.16474-12-mhocko@kernel.org>
On 05/15/2017 10:58 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> The current memory hotplug implementation relies on having all the
> struct pages associate with a zone/node during the physical hotplug phase
> (arch_add_memory->__add_pages->__add_section->__add_zone). In the vast
> majority of cases this means that they are added to ZONE_NORMAL. This
> has been so since 9d99aaa31f59 ("[PATCH] x86_64: Support memory hotadd
> without sparsemem") and it wasn't a big deal back then because movable
> onlining didn't exist yet.
>
> Much later memory hotplug wanted to (ab)use ZONE_MOVABLE for movable
> onlining 511c2aba8f07 ("mm, memory-hotplug: dynamic configure movable
> memory and portion memory") and then things got more complicated. Rather
> than reconsidering the zone association which was no longer needed
> (because the memory hotplug already depended on SPARSEMEM) a convoluted
> semantic of zone shifting has been developed. Only the currently last
> memblock or the one adjacent to the zone_movable can be onlined movable.
> This essentially means that the online type changes as the new memblocks
> are added.
>
> Let's simulate memory hot online manually
> $ echo 0x100000000 > /sys/devices/system/memory/probe
> $ grep . /sys/devices/system/memory/memory32/valid_zones
> Normal Movable
>
> $ echo $((0x100000000+(128<<20))) > /sys/devices/system/memory/probe
> $ grep . /sys/devices/system/memory/memory3?/valid_zones
> /sys/devices/system/memory/memory32/valid_zones:Normal
> /sys/devices/system/memory/memory33/valid_zones:Normal Movable
>
> $ echo $((0x100000000+2*(128<<20))) > /sys/devices/system/memory/probe
> $ grep . /sys/devices/system/memory/memory3?/valid_zones
> /sys/devices/system/memory/memory32/valid_zones:Normal
> /sys/devices/system/memory/memory33/valid_zones:Normal
> /sys/devices/system/memory/memory34/valid_zones:Normal Movable
>
> $ echo online_movable > /sys/devices/system/memory/memory34/state
> $ grep . /sys/devices/system/memory/memory3?/valid_zones
> /sys/devices/system/memory/memory32/valid_zones:Normal
> /sys/devices/system/memory/memory33/valid_zones:Normal Movable
> /sys/devices/system/memory/memory34/valid_zones:Movable Normal
>
> This is an awkward semantic because an udev event is sent as soon as the
> block is onlined and an udev handler might want to online it based on
> some policy (e.g. association with a node) but it will inherently race
> with new blocks showing up.
>
> This patch changes the physical online phase to not associate pages
> with any zone at all. All the pages are just marked reserved and wait
> for the onlining phase to be associated with the zone as per the online
> request. There are only two requirements
> - existing ZONE_NORMAL and ZONE_MOVABLE cannot overlap
> - ZONE_NORMAL precedes ZONE_MOVABLE in physical addresses
> the later on is not an inherent requirement and can be changed in the
> future. It preserves the current behavior and made the code slightly
> simpler. This is subject to change in future.
>
> This means that the same physical online steps as above will lead to the
> following state:
> Normal Movable
>
> /sys/devices/system/memory/memory32/valid_zones:Normal Movable
> /sys/devices/system/memory/memory33/valid_zones:Normal Movable
>
> /sys/devices/system/memory/memory32/valid_zones:Normal Movable
> /sys/devices/system/memory/memory33/valid_zones:Normal Movable
> /sys/devices/system/memory/memory34/valid_zones:Normal Movable
>
> /sys/devices/system/memory/memory32/valid_zones:Normal Movable
> /sys/devices/system/memory/memory33/valid_zones:Normal Movable
> /sys/devices/system/memory/memory34/valid_zones:Movable
>
> Implementation:
> The current move_pfn_range is reimplemented to check the above
> requirements (allow_online_pfn_range) and then updates the respective
> zone (move_pfn_range_to_zone), the pgdat and links all the pages in the
> pfn range with the zone/node. __add_pages is updated to not require the
> zone and only initializes sections in the range. This allowed to
> simplify the arch_add_memory code (s390 could get rid of quite some
> of code).
>
> devm_memremap_pages is the only user of arch_add_memory which relies
> on the zone association because it only hooks into the memory hotplug
> only half way. It uses it to associate the new memory with ZONE_DEVICE
> but doesn't allow it to be {on,off}lined via sysfs. This means that this
> particular code path has to call move_pfn_range_to_zone explicitly.
>
> The original zone shifting code is kept in place and will be removed in
> the follow up patch for an easier review.
>
> Please note that this patch also changes the original behavior when
> offlining a memory block adjacent to another zone (Normal vs. Movable)
> used to allow to change its movable type. This will be handled later.
>
> Changes since v1
> - we have to associate the page with the node early (in __add_section),
> because pfn_to_node depends on struct page containing this
> information - based on testing by Reza Arbab
> - resize_{zone,pgdat}_range has to check whether they are popoulated -
> Reza Arbab
> - fix devm_memremap_pages to use pfn rather than physical address -
> JA(C)rA'me Glisse
> - move_pfn_range has to check for intersection with zone_movable rather
> than to rely on allow_online_pfn_range(MMOP_ONLINE_MOVABLE) for
> MMOP_ONLINE_KEEP
>
> Changes since v2
> - fix show_valid_zones nr_pages calculation
> - allow_online_pfn_range has to check managed pages rather than present
> - zone_intersects fix bogus check
> - fix zone_intersects + few nits as per Vlastimil
>
> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
> Cc: linux-arch@vger.kernel.org
> Tested-by: Dan Williams <dan.j.williams@intel.com>
> Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com> # For s390 bits
> Tested-by: Reza Arbab <arbab@linux.vnet.ibm.com>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: mm, something wring in page_lock_anon_vma_read()?
From: Xishi Qiu @ 2017-05-19 8:52 UTC (permalink / raw)
To: Andrew Morton, Tejun Heo, Michal Hocko, Johannes Weiner,
Mel Gorman, Michal Hocko, Vlastimil Babka, Minchan Kim,
David Rientjes, Joonsoo Kim, aarcange, sumeet.keswani,
Rik van Riel, Hugh Dickins
Cc: Linux MM, LKML, zhong jiang
In-Reply-To: <591D6D79.7030704@huawei.com>
On 2017/5/18 17:46, Xishi Qiu wrote:
> Hi, my system triggers this bug, and the vmcore shows the anon_vma seems be freed.
> The kernel is RHEL 7.2, and the bug is hard to reproduce, so I don't know if it
> exists in mainline, any reply is welcome!
>
When we alloc anon_vma, we will init the value of anon_vma->root,
so can we set anon_vma->root to NULL when calling
anon_vma_free -> kmem_cache_free(anon_vma_cachep, anon_vma);
anon_vma_free()
...
anon_vma->root = NULL;
kmem_cache_free(anon_vma_cachep, anon_vma);
I find if we do this above, system boot failed, why?
Thanks,
Xishi Qiu
> [35030.332666] general protection fault: 0000 [#1] SMP
> [35030.333016] Modules linked in: veth ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c rtos_kbox_panic(OE) ipmi_devintf ipmi_si ipmi_msghandler signo_catch(O) cirrus syscopyarea sysfillrect sysimgblt ttm crc32_pclmul ghash_clmulni_intel drm_kms_helper aesni_intel ppdev drm lrw gf128mul parport_pc glue_helper ablk_helper serio_raw cryptd i2c_piix4 parport pcspkr sg floppy i2c_core dm_mod sha512_generic ip_tables sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_console virtio_scsi virtio_net ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring virtio ata_piix libata ext4 mbcache
> [35030.333016] jbd2
> [35030.333016] CPU: 3 PID: 48 Comm: kswapd0 Tainted: G OE ---- ------- 3.10.0-327.36.58.4.x86_64 #1
> [35030.333016] Hardware name: OpenStack Foundation OpenStack Nova, BIOS rel-1.8.1-0-g4adadbd-20160826_044443-hghoulaslx112 04/01/2014
> [35030.333016] task: ffff8801b2d20000 ti: ffff8801b4c38000 task.ti: ffff8801b4c38000
> [35030.333016] RIP: 0010:[<ffffffff810acac5>] [<ffffffff810acac5>] down_read_trylock+0x5/0x50
> [35030.333016] RSP: 0000:ffff8801b4c3ba90 EFLAGS: 00010282
> [35030.333016] RAX: 0000000000000000 RBX: ffff8801b3e2a100 RCX: 0000000000000000
> [35030.333016] RDX: 0000000000000000 RSI: 0000000000000000 RDI: deb604d497705c5d
> [35030.333016] RBP: ffff8801b4c3bab8 R08: ffffea0002c34460 R09: ffff8801b3d7e8a0
> [35030.333016] R10: 0000000000000004 R11: fff00000fe000000 R12: ffff8801b3e2a101
> [35030.333016] R13: ffffea0002c34440 R14: deb604d497705c5d R15: ffffea0002c34440
> [35030.333016] FS: 0000000000000000(0000) GS:ffff8801bed80000(0000) knlGS:0000000000000000
> [35030.333016] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [35030.333016] CR2: 000000c422011080 CR3: 0000000001976000 CR4: 00000000001407e0
> [35030.333016] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [35030.333016] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [35030.333016] Stack:
> [35030.333016] ffffffff811b2795 ffffea0002c34440 0000000000000000 000000000000000f
> [35030.333016] 0000000000000001 ffff8801b4c3bb30 ffffffff811b2a17 ffff8800a712d640
> [35030.333016] 000000000c4229e2 ffff8801b4c3bb80 0000000100000000 000000000c41fe38
> [35030.333016] Call Trace:
> [35030.333016] [<ffffffff811b2795>] ? page_lock_anon_vma_read+0x55/0x110
> [35030.333016] [<ffffffff811b2a17>] page_referenced+0x1c7/0x350
> [35030.333016] [<ffffffff8118d9b4>] shrink_active_list+0x1e4/0x400
> [35030.333016] [<ffffffff8118e08d>] shrink_lruvec+0x4bd/0x770
> [35030.333016] [<ffffffff8118e3b6>] shrink_zone+0x76/0x1a0
> [35030.333016] [<ffffffff8118f6cc>] balance_pgdat+0x49c/0x610
> [35030.333016] [<ffffffff8118f9b3>] kswapd+0x173/0x450
> [35030.333016] [<ffffffff810a8a00>] ? wake_up_atomic_t+0x30/0x30
> [35030.333016] [<ffffffff8118f840>] ? balance_pgdat+0x610/0x610
> [35030.333016] [<ffffffff810a79bf>] kthread+0xcf/0xe0
> [35030.333016] [<ffffffff810a78f0>] ? kthread_create_on_node+0x120/0x120
> [35030.333016] [<ffffffff81665bd8>] ret_from_fork+0x58/0x90
> [35030.333016] [<ffffffff810a78f0>] ? kthread_create_on_node+0x120/0x120
> [35030.333016] Code: 00 ba ff ff ff ff 48 89 d8 f0 48 0f c1 10 79 05 e8 31 06 27 00 5b 5d c3 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 <48> 8b 07 48 89 c2 48 83 c2 01 7e 07 f0 48 0f b1 17 75 f0 48 f7
> [35030.333016] RIP [<ffffffff810acac5>] down_read_trylock+0x5/0x50
> [35030.333016] RSP <ffff8801b4c3ba90>
> [35030.333016] ------------[ cut here ]------------
>
> struct page {
> flags = 9007194960298056,
> mapping = 0xffff8801b3e2a101,
> {
> {
> index = 34324593617,
> freelist = 0x7fde7bbd1,
> pfmemalloc = 209,
> thp_mmu_gather = {
> counter = -35144751
> },
> pmd_huge_pte = 0x7fde7bbd1
> },
> {
> counters = 8589934592,
> {
> {
> _mapcount = {
> counter = 0
> },
> {
> inuse = 0,
> objects = 0,
> frozen = 0
> },
> units = 0
> },
> _count = {
> counter = 2
> }
> }
> }
> },
> {
> lru = {
> next = 0xdead000000100100,
> prev = 0xdead000000200200
> },
> {
> next = 0xdead000000100100,
> pages = 2097664,
> pobjects = -559087616
> },
> list = {
> next = 0xdead000000100100,
> prev = 0xdead000000200200
> },
> slab_page = 0xdead000000100100
> },
> {
> private = 0,
> ptl = {
> {
> rlock = {
> raw_lock = {
> {
> head_tail = 0,
> tickets = {
> head = 0,
> tail = 0
> }
> }
> }
> }
> }
> },
> slab_cache = 0x0,
> first_page = 0x0
> }
> }
>
>
>
> crash> struct anon_vma 0xffff8801b3e2a100
> struct anon_vma {
> root = 0xdeb604d497705c55,
> rwsem = {
> count = -8192007903225070328,
> wait_lock = {
> raw_lock = {
> {
> head_tail = 2955503940,
> tickets = {
> head = 26948,
> tail = 45097
> }
> }
> }
> },
> wait_list = {
> next = 0x559f9107c1b47439,
> prev = 0x3de13f709bfa043b
> }
> },
> refcount = {
> counter = -13243516
> },
> rb_root = {
> rb_node = 0x11dd18f9ce0bb2e9
> }
> }
>
> This address 0xffff8801b3e2a100 can not find in "kmem -S anon_vma"
>
> The page flags is
> crash> kmem -g 0x1FFFFF00080048
> FLAGS: 1fffff00080048
> PAGE-FLAG BIT VALUE
> PG_uptodate 3 0000008
> PG_active 6 0000040
> PG_swapbacked 19 0080000
>
>
> .
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm/vmstat: add oom_kill counter
From: Vlastimil Babka @ 2017-05-19 9:05 UTC (permalink / raw)
To: Konstantin Khlebnikov, linux-mm, Andrew Morton, linux-kernel,
Michal Hocko
In-Reply-To: <149517718482.32770.939520643229572472.stgit@buzz>
On 05/19/2017 08:59 AM, Konstantin Khlebnikov wrote:
> Show count of global oom killer invocations in /proc/vmstat
Maybe some more rationale why is that useful?
Vlastimil
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> ---
> include/linux/vm_event_item.h | 1 +
> mm/oom_kill.c | 1 +
> mm/vmstat.c | 1 +
> 3 files changed, 3 insertions(+)
>
> diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
> index d84ae90ccd5c..1707e0a7d943 100644
> --- a/include/linux/vm_event_item.h
> +++ b/include/linux/vm_event_item.h
> @@ -41,6 +41,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
> KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
> PAGEOUTRUN, PGROTATED,
> DROP_PAGECACHE, DROP_SLAB,
> + OOM_KILL,
> #ifdef CONFIG_NUMA_BALANCING
> NUMA_PTE_UPDATES,
> NUMA_HUGE_PTE_UPDATES,
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 04c9143a8625..c734c42826cf 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -883,6 +883,7 @@ static void oom_kill_process(struct oom_control *oc, const char *message)
> */
> do_send_sig_info(SIGKILL, SEND_SIG_FORCED, victim, true);
> mark_oom_victim(victim);
> + count_vm_event(OOM_KILL);
> pr_err("Killed process %d (%s) total-vm:%lukB, anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n",
> task_pid_nr(victim), victim->comm, K(victim->mm->total_vm),
> K(get_mm_counter(victim->mm, MM_ANONPAGES)),
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 76f73670200a..fe80b81a86e0 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -1018,6 +1018,7 @@ const char * const vmstat_text[] = {
>
> "drop_pagecache",
> "drop_slab",
> + "oom_kill",
>
> #ifdef CONFIG_NUMA_BALANCING
> "numa_pte_updates",
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm/vmstat: add oom_kill counter
From: Konstantin Khlebnikov @ 2017-05-19 9:10 UTC (permalink / raw)
To: Vlastimil Babka, linux-mm, Andrew Morton, linux-kernel,
Michal Hocko
In-Reply-To: <7683805d-e0ac-3ab9-0a73-47eea945436d@suse.cz>
On 19.05.2017 12:05, Vlastimil Babka wrote:
> On 05/19/2017 08:59 AM, Konstantin Khlebnikov wrote:
>> Show count of global oom killer invocations in /proc/vmstat
>
> Maybe some more rationale why is that useful?
Currently the only way to detect oom kill is grepping kernel logs.
This counter makes such monitoring much easier.
>
> Vlastimil
>
>> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
>> ---
>> include/linux/vm_event_item.h | 1 +
>> mm/oom_kill.c | 1 +
>> mm/vmstat.c | 1 +
>> 3 files changed, 3 insertions(+)
>>
>> diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
>> index d84ae90ccd5c..1707e0a7d943 100644
>> --- a/include/linux/vm_event_item.h
>> +++ b/include/linux/vm_event_item.h
>> @@ -41,6 +41,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
>> KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
>> PAGEOUTRUN, PGROTATED,
>> DROP_PAGECACHE, DROP_SLAB,
>> + OOM_KILL,
>> #ifdef CONFIG_NUMA_BALANCING
>> NUMA_PTE_UPDATES,
>> NUMA_HUGE_PTE_UPDATES,
>> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
>> index 04c9143a8625..c734c42826cf 100644
>> --- a/mm/oom_kill.c
>> +++ b/mm/oom_kill.c
>> @@ -883,6 +883,7 @@ static void oom_kill_process(struct oom_control *oc, const char *message)
>> */
>> do_send_sig_info(SIGKILL, SEND_SIG_FORCED, victim, true);
>> mark_oom_victim(victim);
>> + count_vm_event(OOM_KILL);
>> pr_err("Killed process %d (%s) total-vm:%lukB, anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n",
>> task_pid_nr(victim), victim->comm, K(victim->mm->total_vm),
>> K(get_mm_counter(victim->mm, MM_ANONPAGES)),
>> diff --git a/mm/vmstat.c b/mm/vmstat.c
>> index 76f73670200a..fe80b81a86e0 100644
>> --- a/mm/vmstat.c
>> +++ b/mm/vmstat.c
>> @@ -1018,6 +1018,7 @@ const char * const vmstat_text[] = {
>>
>> "drop_pagecache",
>> "drop_slab",
>> + "oom_kill",
>>
>> #ifdef CONFIG_NUMA_BALANCING
>> "numa_pte_updates",
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org. For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: mm, something wring in page_lock_anon_vma_read()?
From: Xishi Qiu @ 2017-05-19 9:44 UTC (permalink / raw)
To: Andrew Morton, Tejun Heo, Michal Hocko, Johannes Weiner,
Mel Gorman, Michal Hocko, Vlastimil Babka, Minchan Kim,
David Rientjes, Joonsoo Kim, aarcange, sumeet.keswani,
Rik van Riel, Hugh Dickins
Cc: Linux MM, LKML, zhong jiang
In-Reply-To: <591EB25C.9080901@huawei.com>
On 2017/5/19 16:52, Xishi Qiu wrote:
> On 2017/5/18 17:46, Xishi Qiu wrote:
>
>> Hi, my system triggers this bug, and the vmcore shows the anon_vma seems be freed.
>> The kernel is RHEL 7.2, and the bug is hard to reproduce, so I don't know if it
>> exists in mainline, any reply is welcome!
>>
>
> When we alloc anon_vma, we will init the value of anon_vma->root,
> so can we set anon_vma->root to NULL when calling
> anon_vma_free -> kmem_cache_free(anon_vma_cachep, anon_vma);
>
> anon_vma_free()
> ...
> anon_vma->root = NULL;
> kmem_cache_free(anon_vma_cachep, anon_vma);
>
> I find if we do this above, system boot failed, why?
>
If anon_vma was freed, we should not to access the root_anon_vma, because it maybe also
freed(e.g. anon_vma == root_anon_vma), right?
page_lock_anon_vma_read()
...
anon_vma = (struct anon_vma *) (anon_mapping - PAGE_MAPPING_ANON);
root_anon_vma = ACCESS_ONCE(anon_vma->root);
if (down_read_trylock(&root_anon_vma->rwsem)) { // it's not safe
...
if (!atomic_inc_not_zero(&anon_vma->refcount)) { // check anon_vma was not freed
...
anon_vma_lock_read(anon_vma); // it's safe
...
> Thanks,
> Xishi Qiu
>
>> [35030.332666] general protection fault: 0000 [#1] SMP
>> [35030.333016] Modules linked in: veth ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c rtos_kbox_panic(OE) ipmi_devintf ipmi_si ipmi_msghandler signo_catch(O) cirrus syscopyarea sysfillrect sysimgblt ttm crc32_pclmul ghash_clmulni_intel drm_kms_helper aesni_intel ppdev drm lrw gf128mul parport_pc glue_helper ablk_helper serio_raw cryptd i2c_piix4 parport pcspkr sg floppy i2c_core dm_mod sha512_generic ip_tables sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_console virtio_scsi virtio_net ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring virtio ata_piix libata ext4 mbcache
>> [35030.333016] jbd2
>> [35030.333016] CPU: 3 PID: 48 Comm: kswapd0 Tainted: G OE ---- ------- 3.10.0-327.36.58.4.x86_64 #1
>> [35030.333016] Hardware name: OpenStack Foundation OpenStack Nova, BIOS rel-1.8.1-0-g4adadbd-20160826_044443-hghoulaslx112 04/01/2014
>> [35030.333016] task: ffff8801b2d20000 ti: ffff8801b4c38000 task.ti: ffff8801b4c38000
>> [35030.333016] RIP: 0010:[<ffffffff810acac5>] [<ffffffff810acac5>] down_read_trylock+0x5/0x50
>> [35030.333016] RSP: 0000:ffff8801b4c3ba90 EFLAGS: 00010282
>> [35030.333016] RAX: 0000000000000000 RBX: ffff8801b3e2a100 RCX: 0000000000000000
>> [35030.333016] RDX: 0000000000000000 RSI: 0000000000000000 RDI: deb604d497705c5d
>> [35030.333016] RBP: ffff8801b4c3bab8 R08: ffffea0002c34460 R09: ffff8801b3d7e8a0
>> [35030.333016] R10: 0000000000000004 R11: fff00000fe000000 R12: ffff8801b3e2a101
>> [35030.333016] R13: ffffea0002c34440 R14: deb604d497705c5d R15: ffffea0002c34440
>> [35030.333016] FS: 0000000000000000(0000) GS:ffff8801bed80000(0000) knlGS:0000000000000000
>> [35030.333016] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [35030.333016] CR2: 000000c422011080 CR3: 0000000001976000 CR4: 00000000001407e0
>> [35030.333016] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [35030.333016] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [35030.333016] Stack:
>> [35030.333016] ffffffff811b2795 ffffea0002c34440 0000000000000000 000000000000000f
>> [35030.333016] 0000000000000001 ffff8801b4c3bb30 ffffffff811b2a17 ffff8800a712d640
>> [35030.333016] 000000000c4229e2 ffff8801b4c3bb80 0000000100000000 000000000c41fe38
>> [35030.333016] Call Trace:
>> [35030.333016] [<ffffffff811b2795>] ? page_lock_anon_vma_read+0x55/0x110
>> [35030.333016] [<ffffffff811b2a17>] page_referenced+0x1c7/0x350
>> [35030.333016] [<ffffffff8118d9b4>] shrink_active_list+0x1e4/0x400
>> [35030.333016] [<ffffffff8118e08d>] shrink_lruvec+0x4bd/0x770
>> [35030.333016] [<ffffffff8118e3b6>] shrink_zone+0x76/0x1a0
>> [35030.333016] [<ffffffff8118f6cc>] balance_pgdat+0x49c/0x610
>> [35030.333016] [<ffffffff8118f9b3>] kswapd+0x173/0x450
>> [35030.333016] [<ffffffff810a8a00>] ? wake_up_atomic_t+0x30/0x30
>> [35030.333016] [<ffffffff8118f840>] ? balance_pgdat+0x610/0x610
>> [35030.333016] [<ffffffff810a79bf>] kthread+0xcf/0xe0
>> [35030.333016] [<ffffffff810a78f0>] ? kthread_create_on_node+0x120/0x120
>> [35030.333016] [<ffffffff81665bd8>] ret_from_fork+0x58/0x90
>> [35030.333016] [<ffffffff810a78f0>] ? kthread_create_on_node+0x120/0x120
>> [35030.333016] Code: 00 ba ff ff ff ff 48 89 d8 f0 48 0f c1 10 79 05 e8 31 06 27 00 5b 5d c3 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 <48> 8b 07 48 89 c2 48 83 c2 01 7e 07 f0 48 0f b1 17 75 f0 48 f7
>> [35030.333016] RIP [<ffffffff810acac5>] down_read_trylock+0x5/0x50
>> [35030.333016] RSP <ffff8801b4c3ba90>
>> [35030.333016] ------------[ cut here ]------------
>>
>> struct page {
>> flags = 9007194960298056,
>> mapping = 0xffff8801b3e2a101,
>> {
>> {
>> index = 34324593617,
>> freelist = 0x7fde7bbd1,
>> pfmemalloc = 209,
>> thp_mmu_gather = {
>> counter = -35144751
>> },
>> pmd_huge_pte = 0x7fde7bbd1
>> },
>> {
>> counters = 8589934592,
>> {
>> {
>> _mapcount = {
>> counter = 0
>> },
>> {
>> inuse = 0,
>> objects = 0,
>> frozen = 0
>> },
>> units = 0
>> },
>> _count = {
>> counter = 2
>> }
>> }
>> }
>> },
>> {
>> lru = {
>> next = 0xdead000000100100,
>> prev = 0xdead000000200200
>> },
>> {
>> next = 0xdead000000100100,
>> pages = 2097664,
>> pobjects = -559087616
>> },
>> list = {
>> next = 0xdead000000100100,
>> prev = 0xdead000000200200
>> },
>> slab_page = 0xdead000000100100
>> },
>> {
>> private = 0,
>> ptl = {
>> {
>> rlock = {
>> raw_lock = {
>> {
>> head_tail = 0,
>> tickets = {
>> head = 0,
>> tail = 0
>> }
>> }
>> }
>> }
>> }
>> },
>> slab_cache = 0x0,
>> first_page = 0x0
>> }
>> }
>>
>>
>>
>> crash> struct anon_vma 0xffff8801b3e2a100
>> struct anon_vma {
>> root = 0xdeb604d497705c55,
>> rwsem = {
>> count = -8192007903225070328,
>> wait_lock = {
>> raw_lock = {
>> {
>> head_tail = 2955503940,
>> tickets = {
>> head = 26948,
>> tail = 45097
>> }
>> }
>> }
>> },
>> wait_list = {
>> next = 0x559f9107c1b47439,
>> prev = 0x3de13f709bfa043b
>> }
>> },
>> refcount = {
>> counter = -13243516
>> },
>> rb_root = {
>> rb_node = 0x11dd18f9ce0bb2e9
>> }
>> }
>>
>> This address 0xffff8801b3e2a100 can not find in "kmem -S anon_vma"
>>
>> The page flags is
>> crash> kmem -g 0x1FFFFF00080048
>> FLAGS: 1fffff00080048
>> PAGE-FLAG BIT VALUE
>> PG_uptodate 3 0000008
>> PG_active 6 0000040
>> PG_swapbacked 19 0080000
>>
>>
>> .
>>
>
>
>
>
> .
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox