* Re: [PATCH v2 1/2] mm: Uncharge poisoned pages
From: Michal Hocko @ 2017-05-09 9:18 UTC (permalink / raw)
To: Naoya Horiguchi, akpm@linux-foundation.org
Cc: Andi Kleen, Johannes Weiner, Laurent Dufour,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Vladimir Davydov
In-Reply-To: <20170508025827.GA4913@hori1.linux.bs1.fc.nec.co.jp>
On Mon 08-05-17 02:58:36, Naoya Horiguchi wrote:
> On Tue, May 02, 2017 at 08:55:07PM +0200, Michal Hocko wrote:
> > On Tue 02-05-17 16:59:30, Laurent Dufour wrote:
> > > On 28/04/2017 15:48, Michal Hocko wrote:
> > [...]
> > > > This is getting quite hairy. What is the expected page count of the
> > > > hwpoison page?
> >
> > OK, so from the quick check of the hwpoison code it seems that the ref
> > count will be > 1 (from get_hwpoison_page).
> >
> > > > I guess we would need to update the VM_BUG_ON in the
> > > > memcg uncharge code to ignore the page count of hwpoison pages if it can
> > > > be arbitrary.
> > >
> > > Based on the experiment I did, page count == 2 when isolate_lru_page()
> > > succeeds, even in the case of a poisoned page.
> >
> > that would make some sense to me. The page should have been already
> > unmapped therefore but memory_failure increases the ref count and 1 is
> > for isolate_lru_page().
>
> # sorry for late reply, I was on holidays last week...
>
> Right, and the refcount taken for memory_failure is not freed after
> memory_failure() returns. unpoison_memory() does free the refcount.
OK, from the charge POV this would be safe because we clear page->memcg
so it wouldn't get uncharged more times.
> > > In my case I think this
> > > is because the page is still used by the process which is calling madvise().
> > >
> > > I'm wondering if I'm looking at the right place. May be the poisoned
> > > page should remain attach to the memory_cgroup until no one is using it.
> > > In that case this means that something should be done when the page is
> > > off-lined... I've to dig further here.
> >
> > No, AFAIU the page will not drop the reference count down to 0 in most
> > cases. Maybe there are some scenarios where this can happen but I would
> > expect that the poisoned page will be mapped and in use most of the time
> > and won't drop down 0. And then we should really uncharge it because it
> > will pin the memcg and make it unfreeable which doesn't seem to be what
> > we want. So does the following work reasonable? Andi, Johannes, what do
> > you think? I cannot say I would be really comfortable touching hwpoison
> > code as I really do not understand the workflow. Maybe we want to move
> > this uncharge down to memory_failure() right before we report success?
>
> memory_failure() can be called for any types of page (including slab or
> any kernel/driver pages), and the reported problem seems happen only on
> in-use user pages, so uncharging in delete_from_lru_cache() as done below
> looks better to me.
Yeah, we do see problems only for LRU/page cache pages but my
understanding is that error_states (e.g. me_kernel for the kernel
memory) might change in the future and then we wouldn't catch the same
bug, no?
> > ---
> > From 8bf0791bcf35996a859b6d33fb5494e5b53de49d Mon Sep 17 00:00:00 2001
> > From: Michal Hocko <mhocko@suse.com>
> > Date: Tue, 2 May 2017 20:32:24 +0200
> > Subject: [PATCH] hwpoison, memcg: forcibly uncharge LRU pages
> >
> > Laurent Dufour has noticed that hwpoinsoned pages are kept charged. In
> > his particular case he has hit a bad_page("page still charged to cgroup")
> > when onlining a hwpoison page.
>
> > While this looks like something that shouldn't
> > happen in the first place because onlining hwpages and returning them to
> > the page allocator makes only little sense it shows a real problem.
> >
> > hwpoison pages do not get freed usually so we do not uncharge them (at
> > least not since 0a31bc97c80c ("mm: memcontrol: rewrite uncharge API")).
> > Each charge pins memcg (since e8ea14cc6ead ("mm: memcontrol: take a css
> > reference for each charged page")) as well and so the mem_cgroup and the
> > associated state will never go away. Fix this leak by forcibly
> > uncharging a LRU hwpoisoned page in delete_from_lru_cache(). We also
> > have to tweak uncharge_list because it cannot rely on zero ref count
> > for these pages.
> >
> > Fixes: 0a31bc97c80c ("mm: memcontrol: rewrite uncharge API")
> > Reported-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
> > Signed-off-by: Michal Hocko <mhocko@suse.com>
>
> Reviewed-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Thanks! I will wait a day or two for Johannes and repost the patch.
Andrew could you drop
http://www.ozlabs.org/~akpm/mmotm/broken-out/mm-uncharge-poisoned-pages.patch
in the mean time, please?
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH RFC] hugetlbfs 'noautofill' mount option
From: Christoph Hellwig @ 2017-05-09 8:58 UTC (permalink / raw)
To: prakash.sangappa; +Cc: Dave Hansen, linux-kernel, linux-mm
In-Reply-To: <476ea1b6-36d1-bc86-fa99-b727e3c2650d@oracle.com>
On Mon, May 08, 2017 at 03:12:42PM -0700, prakash.sangappa wrote:
> Regarding #3 as a general feature, do we want to
> consider this and the complexity associated with the
> implementation?
We have to. Given that no one has exclusive access to hugetlbfs
a mount option is fundamentally the wrong interface.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: fix the memory leak after collapsing the huge page fails
From: Vlastimil Babka @ 2017-05-09 8:42 UTC (permalink / raw)
To: zhongjiang, akpm; +Cc: mgorman, hannes, kirill.shutemov, linux-mm
In-Reply-To: <1494317557-49680-1-git-send-email-zhongjiang@huawei.com>
On 05/09/2017 10:12 AM, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
>
> Current, when we prepare a huge page to collapse, due to some
> reasons, it can fail to collapse. At the moment, we should
> release the preallocate huge page.
Yeah, looks like the leak is there...
> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> ---
> mm/khugepaged.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index 7cb9c88..3f5749e 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -1080,6 +1080,7 @@ static void collapse_huge_page(struct mm_struct *mm,
> result = SCAN_SUCCEED;
> out_up_write:
> up_write(&mm->mmap_sem);
> + put_page(new_page);
This doesn't seem correct.
- the put_page() will be called also on success, so a premature free?
- the out_nolock: case should be also handled
- collapse_shmem() seems to have the same problem
> out_nolock:
> trace_mm_collapse_huge_page(mm, isolated, result);
> return;
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH] mm: fix the memory leak after collapsing the huge page fails
From: zhongjiang @ 2017-05-09 8:12 UTC (permalink / raw)
To: akpm; +Cc: mgorman, hannes, vbabka, kirill.shutemov, linux-mm
From: zhong jiang <zhongjiang@huawei.com>
Current, when we prepare a huge page to collapse, due to some
reasons, it can fail to collapse. At the moment, we should
release the preallocate huge page.
Signed-off-by: zhong jiang <zhongjiang@huawei.com>
---
mm/khugepaged.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 7cb9c88..3f5749e 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1080,6 +1080,7 @@ static void collapse_huge_page(struct mm_struct *mm,
result = SCAN_SUCCEED;
out_up_write:
up_write(&mm->mmap_sem);
+ put_page(new_page);
out_nolock:
trace_mm_collapse_huge_page(mm, isolated, result);
return;
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Balbir Singh @ 2017-05-09 7:51 UTC (permalink / raw)
To: Michal Hocko, Benjamin Herrenschmidt
Cc: linux-mm, akpm, khandual, aneesh.kumar, paulmck, srikar, haren,
jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <20170505145238.GE31461@dhcp22.suse.cz>
On Fri, 2017-05-05 at 16:52 +0200, Michal Hocko wrote:
> On Thu 04-05-17 17:49:21, Benjamin Herrenschmidt wrote:
> > On Thu, 2017-05-04 at 14:52 +0200, Michal Hocko wrote:
> > > But the direct reclaim would be effective only _after_ all other nodes
> > > are full.
> > >
> > > I thought that kswapd reclaim is a problem because the HW doesn't
> > > support aging properly but as the direct reclaim works then what is the
> > > actual problem?
> >
> > Ageing isn't isn't completely broken. The ATS MMU supports
> > dirty/accessed just fine.
> >
> > However the TLB invalidations are quite expensive with a GPU so too
> > much harvesting is detrimental, and the GPU tends to check pages out
> > using a special "read with intend to write" mode, which means it almost
> > always set the dirty bit if the page is writable to begin with.
>
> This sounds pretty much like a HW specific details which is not the
> right criterion to design general CDM around.
I think Ben answered several of these questions. NUMA we felt was the best
representation of such memory, but it has limitations in that we'd like
to isolate some default algorithms that run on all nodes marked N_MEMORY.
Do you see that as a concern? Would you like to see a generic policy
like Ben said to handle node attributes like reclaim, autonuma, etc?
>
> So let me repeat the fundamental question. Is the only difference from
> cpuless nodes the fact that the node should be invisible to processes
> unless they specify an explicit node mask? If yes then we are talking
> about policy in the kernel and that sounds like a big no-no to me.
> Moreover cpusets already support exclusive numa nodes AFAIR.
Why do you see this as a policy, it's a mechanism of isolating nodes,
the nodes themselves are then used using mempolicy.
>
> I am either missing something important here, and the discussion so far
> hasn't helped to be honest, or this whole CDM effort tries to build a
> generic interface around a _specific_ piece of HW. The matter is worse
> by the fact that the described usecases are so vague that it is hard to
> build a good picture whether this is generic enough that a new/different
> HW will still fit into this picture.
The use case is similar to HMM, except that we've got coherent memory.
We treat is as important and want to isolate normal allocations, unless
the allocation is explicitly specified. CPUsets provide an isolation
mechanism, but we see autonuma for example moving pages away when there
is an access from the system side. With reclaim, it would be better to
use the fallback list first then swap. Again the use case is:
I'm trying to do a FAQ version here
Isolate memory - why?
- CDM memory is not meant for normal usage, applications can request for it
explictly. Oflload their compute to the device where the memory is
(the offload is via a user space API like CUDA/openCL/...)
How do we isolate - NUMA or HMM?
- Since the memory is coherent, NUMA provides the mechanism to isolate to
a large extent via mempolicy. With NUMA we also get autonuma/kswapd/etc
running. Something we would like to avoid. NUMA gives the application
a transparent view of memory, in the sense that all mm features work,
like direct page cache allocation in coherent device memory, limiting
memory via cgroups if required, etc. With CPUSets, its
possible for us to isolate allocation. One challenge is that the
admin on the system may use them differently and applications need to
be aware of running in the right cpuset to allocate memory from the
CDM node. Putting all applications in the cpuset with the CDM node is
not the right thing to do, which means the application needs to move itself
to the right cpuset before requesting for CDM memory. It's not impossible
to use CPUsets, just hard to configure correctly.
- With HMM, we would need a HMM variant HMM-CDM, so that we are not marking
the pages as unavailable, page cache cannot do directly to coherent memory.
Audit of mm paths is required. Most of the other things should work.
User access to HMM-CDM memory behind ZONE_DEVICE is via a device driver.
Why do we need migration?
- Depending on where the memory is being accessed from, we would like to
migrate pages between system and coherent device memory. HMM provides
DMA offload capability that is useful in both cases.
What is the larger picture - end to end?
- Applications can allocate memory on the device or in system memory,
offload the compute via user space API. Migration can be used for performance
if required since it helps to keep the memory local to the compute.
Ben/Jerome/John/others did I get the FAQ right?
>From my side, I want to ensure that the decision HMM-CDM or NUMA-CDM is based
on our design and understanding, as opposed to the reason that the
use case is not clear or in sufficient. I'd be happy if we said, we understand
the use case and believe that HMM-CDM is better from the mm's perspective as
its better because... as opposed to isolating NUMA attributes because ....
or vice-versa.
Thanks for the review,
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [virtio-dev] Re: [PATCH v9 2/5] virtio-balloon: VIRTIO_BALLOON_F_BALLOON_CHUNKS
From: Wei Wang @ 2017-05-09 2:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-dev@lists.oasis-open.org, linux-kernel@vger.kernel.org,
qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org,
kvm@vger.kernel.org, linux-mm@kvack.org, david@redhat.com,
Hansen, Dave, cornelia.huck@de.ibm.com, akpm@linux-foundation.org,
mgorman@techsingularity.net, aarcange@redhat.com,
amit.shah@redhat.com, pbonzini@redhat.com,
liliang.opensource@gmail.com
In-Reply-To: <20170508203533-mutt-send-email-mst@kernel.org>
On 05/09/2017 01:40 AM, Michael S. Tsirkin wrote:
> On Sun, May 07, 2017 at 04:19:28AM +0000, Wang, Wei W wrote:
>> On 05/06/2017 06:26 AM, Michael S. Tsirkin wrote:
>>> On Thu, Apr 27, 2017 at 02:31:49PM +0800, Wei Wang wrote:
>>>> On 04/27/2017 07:20 AM, Michael S. Tsirkin wrote:
>>>>> On Wed, Apr 26, 2017 at 11:03:34AM +0000, Wang, Wei W wrote:
>>>>>> Hi Michael, could you please give some feedback?
>>>>> I'm sorry, I'm not sure feedback on what you are requesting.
>>>> Oh, just some trivial things (e.g. use a field in the header,
>>>> hdr->chunks to indicate the number of chunks in the payload) that
>>>> wasn't confirmed.
>>>>
>>>> I will prepare the new version with fixing the agreed issues, and we
>>>> can continue to discuss those parts if you still find them improper.
>>>>
>>>>
>>>>> The interface looks reasonable now, even though there's a way to
>>>>> make it even simpler if we can limit chunk size to 2G (in fact 4G -
>>>>> 1). Do you think we can live with this limitation?
>>>> Yes, I think we can. So, is it good to change to use the previous
>>>> 64-bit chunk format (52-bit base + 12-bit size)?
>>> This isn't what I meant. virtio ring has descriptors with a 64 bit address and 32 bit
>>> size.
>>>
>>> If size < 4g is not a significant limitation, why not just use that to pass
>>> address/size in a standard s/g list, possibly using INDIRECT?
>> OK, I see your point, thanks. Post the two options here for an analysis:
>> Option1 (what we have now):
>> struct virtio_balloon_page_chunk {
>> __le64 chunk_num;
>> struct virtio_balloon_page_chunk_entry entry[];
>> };
>> Option2:
>> struct virtio_balloon_page_chunk {
>> __le64 chunk_num;
>> struct scatterlist entry[];
>> };
> This isn't what I meant really :) I meant vring_desc.
OK. Repost the code change:
Option2:
struct virtio_balloon_page_chunk {
__le64 chunk_num;
struct ving_desc entry[];
};
We pre-allocate a table of desc, and each desc is used to hold a chunk.
In that case, the virtqueue_add() function, which deals with sg, is not
usable for us. We may need to add a new one,
virtqueue_add_indirect_desc(),
to add a pre-allocated indirect descriptor table to vring.
>
>> I don't have an issue to change it to Option2, but I would prefer Option1,
>> because I think there is no be obvious difference between the two options,
>> while Option1 appears to have little advantages here:
>> 1) "struct virtio_balloon_page_chunk_entry" has smaller size than
>> "struct scatterlist", so the same size of allocated page chunk buffer
>> can hold more entry[] using Option1;
>> 2) INDIRECT needs on demand kmalloc();
> Within alloc_indirect? We can fix that with a separate patch.
>
>
>> 3) no 4G size limit;
> Do you see lots of >=4g chunks in practice?
It wouldn't be much in practice, but we still need the extra code to
handle the case - break larger chunks into less-than 4g ones.
Best,
Wei
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 1/2] mm: Uncharge poisoned pages
From: Balbir Singh @ 2017-05-09 1:41 UTC (permalink / raw)
To: Laurent Dufour, Michal Hocko, Andi Kleen, Johannes Weiner
Cc: Naoya Horiguchi, linux-kernel, linux-mm, akpm, Vladimir Davydov
In-Reply-To: <03a7ec34-106e-3eb6-0b05-f77a40a2d6b9@linux.vnet.ibm.com>
On Mon, 2017-05-08 at 12:42 +0200, Laurent Dufour wrote:
> Sorry Balbir,
>
> You pointed this out since the beginning but I missed your comment.
> My mistake.
>
No worries, as long as the right thing gets in
Balbir Singh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* mmotm 2017-05-08-16-30 uploaded
From: akpm @ 2017-05-08 23:30 UTC (permalink / raw)
To: mm-commits, linux-kernel, linux-mm, linux-fsdevel, linux-next,
sfr, mhocko, broonie
The mm-of-the-moment snapshot 2017-05-08-16-30 has been uploaded to
http://www.ozlabs.org/~akpm/mmotm/
mmotm-readme.txt says
README for mm-of-the-moment:
http://www.ozlabs.org/~akpm/mmotm/
This is a snapshot of my -mm patch queue. Uploaded at random hopefully
more than once a week.
You will need quilt to apply these patches to the latest Linus release (4.x
or 4.x-rcY). The series file is in broken-out.tar.gz and is duplicated in
http://ozlabs.org/~akpm/mmotm/series
The file broken-out.tar.gz contains two datestamp files: .DATE and
.DATE-yyyy-mm-dd-hh-mm-ss. Both contain the string yyyy-mm-dd-hh-mm-ss,
followed by the base kernel version against which this patch series is to
be applied.
This tree is partially included in linux-next. To see which patches are
included in linux-next, consult the `series' file. Only the patches
within the #NEXT_PATCHES_START/#NEXT_PATCHES_END markers are included in
linux-next.
A git tree which contains the memory management portion of this tree is
maintained at git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git
by Michal Hocko. It contains the patches which are between the
"#NEXT_PATCHES_START mm" and "#NEXT_PATCHES_END" markers, from the series
file, http://www.ozlabs.org/~akpm/mmotm/series.
A full copy of the full kernel tree with the linux-next and mmotm patches
already applied is available through git within an hour of the mmotm
release. Individual mmotm releases are tagged. The master branch always
points to the latest release, so it's constantly rebasing.
http://git.cmpxchg.org/cgit.cgi/linux-mmotm.git/
To develop on top of mmotm git:
$ git remote add mmotm git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git
$ git remote update mmotm
$ git checkout -b topic mmotm/master
<make changes, commit>
$ git send-email mmotm/master.. [...]
To rebase a branch with older patches to a new mmotm release:
$ git remote update mmotm
$ git rebase --onto mmotm/master <topic base> topic
The directory http://www.ozlabs.org/~akpm/mmots/ (mm-of-the-second)
contains daily snapshots of the -mm tree. It is updated more frequently
than mmotm, and is untested.
A git copy of this tree is available at
http://git.cmpxchg.org/cgit.cgi/linux-mmots.git/
and use of this tree is similar to
http://git.cmpxchg.org/cgit.cgi/linux-mmotm.git/, described above.
This mmotm tree contains the following patches against 4.11:
(patches marked "*" will be included in linux-next)
origin.patch
i-need-old-gcc.patch
* mm-compaction-reorder-fields-in-struct-compact_control.patch
* mm-compaction-remove-redundant-watermark-check-in-compact_finished.patch
* mm-page_alloc-split-smallest-stolen-page-in-fallback.patch
* mm-page_alloc-count-movable-pages-when-stealing-from-pageblock.patch
* mm-compaction-change-migrate_async_suitable-to-suitable_migration_source.patch
* mm-compaction-add-migratetype-to-compact_control.patch
* mm-compaction-restrict-async-compaction-to-pageblocks-of-same-migratetype.patch
* mm-compaction-finish-whole-pageblock-to-reduce-fragmentation.patch
* proc-remove-cast-from-memory-allocation.patch
* proc-sysctl-fix-the-int-overflow-for-jiffies-conversion.patch
* drivers-virt-use-get_user_pages_unlocked.patch
* jiffiesh-declare-jiffies-and-jiffies_64-with-____cacheline_aligned_in_smp.patch
* make-help-add-tools-help-target.patch
* locking-hung_task-defer-showing-held-locks.patch
* vmci-fix-a-couple-integer-overflow-tests.patch
* c2port-checking-for-null-instead-of-is_err.patch
* revert-lib-test_sortc-make-it-explicitly-non-modular.patch
* lib-add-module-support-to-array-based-sort-tests.patch
* lib-add-module-support-to-linked-list-sorting-tests.patch
* firmware-makefile-force-recompilation-if-makefile-changes.patch
* checkpatch-remove-obsolete-config_experimental-checks.patch
* checkpatch-add-ability-to-find-bad-uses-of-vsprintf-%pfoo-extensions.patch
* checkpatch-improve-embedded_function_name-test.patch
* checkpatch-allow-space-leading-blank-lines-in-email-headers.patch
* checkpatch-avoid-suggesting-struct-definitions-should-be-const.patch
* checkpatch-improve-multistatement_macro_use_do_while-test.patch
* checkpatch-clarify-the-embedded_function_name-message.patch
* checkpatch-special-audit-for-revert-commit-line.patch
* checkpatch-improve-kalloc-with-multiplication-and-sizeof-test.patch
* checkpatch-add-typedefsfile.patch
* checkpatch-improve-the-embedded-function-name-test-for-patch-contexts.patch
* checkpatch-improve-the-suspect_code_indent-test.patch
* reiserfs-use-designated-initializers.patch
* fork-free-vmapped-stacks-in-cache-when-cpus-are-offline.patch
* cpumask-make-nr_cpumask_bits-unsigned.patch
* crash-move-crashkernel-parsing-and-vmcore-related-code-under-config_crash_core.patch
* ia64-reuse-append_elf_note-and-final_note-functions.patch
* powerpc-fadump-remove-dependency-with-config_kexec.patch
* powerpc-fadump-reuse-crashkernel-parameter-for-fadump-memory-reservation.patch
* powerpc-fadump-update-documentation-about-crashkernel-parameter-reuse.patch
* pidns-disable-pid-allocation-if-pid_ns_prepare_proc-is-failed-in-alloc_pid.patch
* ns-allow-ns_entries-to-have-custom-symlink-content.patch
* pidns-expose-task-pid_ns_for_children-to-userspace.patch
* taskstats-add-e-u-stime-for-tgid-command.patch
* kcov-simplify-interrupt-check.patch
* fault-inject-use-correct-check-for-interrupts.patch
* zlib-inflate-fix-potential-buffer-overflow.patch
* initramfs-provide-a-way-to-ignore-image-provided-by-bootloader.patch
* initramfs-use-vfs_stat-lstat-directly.patch
* ipc-shm-some-shmat-cleanups.patch
* sysvipc-cacheline-align-kern_ipc_perm.patch
* mm-introduce-kvalloc-helpers.patch
* mm-vmalloc-properly-track-vmalloc-users.patch
* mm-support-__gfp_repeat-in-kvmalloc_node-for-32kb.patch
* rhashtable-simplify-a-strange-allocation-pattern.patch
* ila-simplify-a-strange-allocation-pattern.patch
* xattr-zero-out-memory-copied-to-userspace-in-getxattr.patch
* treewide-use-kvalloc-rather-than-opencoded-variants.patch
* net-use-kvmalloc-with-__gfp_repeat-rather-than-open-coded-variant.patch
* md-use-kvmalloc-rather-than-opencoded-variant.patch
* bcache-use-kvmalloc.patch
* mm-swap-use-kvzalloc-to-allocate-some-swap-data-structure.patch
* mm-vmalloc-use-__gfp_highmem-implicitly.patch
* scripts-spellingtxt-add-memory-pattern-and-fix-typos.patch
* scripts-spellingtxt-add-regsiter-register-spelling-mistake.patch
* scripts-spellingtxt-add-intialised-pattern-and-fix-typo-instances.patch
* treewide-correct-diffrent-and-banlance-typos.patch
* treewide-move-set_memory_-functions-away-from-cacheflushh.patch
* arm-use-set_memoryh-header.patch
* arm64-use-set_memoryh-header.patch
* s390-use-set_memoryh-header.patch
* x86-use-set_memoryh-header.patch
* agp-use-set_memoryh-header.patch
* drm-use-set_memoryh-header.patch
* intel_th-use-set_memoryh-header.patch
* watchdog-hpwdt-use-set_memoryh-header.patch
* bpf-use-set_memoryh-header.patch
* module-use-set_memoryh-header.patch
* pm-hibernate-use-set_memoryh-header.patch
* alsa-use-set_memoryh-header.patch
* misc-sram-use-set_memoryh-header.patch
* video-vermilion-use-set_memoryh-header.patch
* drivers-staging-media-atomisp-pci-atomisp2-use-set_memoryh.patch
* treewide-decouple-cacheflushh-and-set_memoryh.patch
* kref-remove-warn_on-for-null-release-functions.patch
* megasas-remove-expensive-inline-from-megasas_return_cmd.patch
* remove-expensive-warn_on-in-pagefault_disabled_dec.patch
* fs-remove-set-but-not-checked-aop_flag_uninterruptible-flag.patch
* docs-vm-transhuge-fix-few-trivial-typos.patch
* format-security-move-static-strings-to-const.patch
* fs-f2fs-use-ktime_get_real_seconds-for-sit_info-times.patch
* trace-make-trace_hwlat-timestamp-y2038-safe.patch
* fs-cifs-replace-current_time-by-other-appropriate-apis.patch
* fs-ceph-current_time-with-ktime_get_real_ts.patch
* fs-ufs-use-ktime_get_real_ts64-for-birthtime.patch
* fs-ubifs-replace-current_time_sec-with-current_time.patch
* lustre-replace-current_time-macro.patch
* apparmorfs-replace-current_time-with-current_time.patch
* gfs2-replace-current_time-with-current_time.patch
* time-delete-current_time_sec-and-current_time.patch
* mm-huge_memory-use-zap_deposited_table-more.patch
* mm-huge_memory-deposit-a-pgtable-for-dax-pmd-faults-when-required.patch
* mm-prevent-potential-recursive-reclaim-due-to-clearing-pf_memalloc.patch
* mm-introduce-memalloc_noreclaim_saverestore.patch
* treewide-convert-pf_memalloc-manipulations-to-new-helpers.patch
* mtd-nand-nandsim-convert-to-memalloc_noreclaim_.patch
* dax-add-tracepoints-to-dax_iomap_pte_fault.patch
* dax-add-tracepoints-to-dax_pfn_mkwrite.patch
* dax-add-tracepoints-to-dax_load_hole.patch
* dax-add-tracepoints-to-dax_writeback_mapping_range.patch
* dax-add-tracepoint-to-dax_writeback_one.patch
* dax-add-tracepoint-to-dax_insert_mapping.patch
* selftests-vm-add-a-test-for-virtual-address-range-mapping.patch
* drivers-staging-ccree-ssi_hashc-fix-build-with-gcc-444.patch
* mm-uncharge-poisoned-pages.patch
* mm-skip-hwpoisoned-pages-when-onlining-pages.patch
* mm-vmstat-remove-spurious-warn-during-zoneinfo-print.patch
* gcov-support-gcc-71.patch
* gcov-support-gcc-71-v2.patch
* gcov-support-gcc-71-v2-checkpatch-fixes.patch
* mm-khugepaged-add-missed-tracepoint-for-collapse_huge_page_swapin.patch
* arm-arch-arm-include-asm-pageh-needs-personalityh.patch
* ocfs2-old-mle-put-and-release-after-the-function-dlm_add_migration_mle-called.patch
* ocfs2-old-mle-put-and-release-after-the-function-dlm_add_migration_mle-called-fix.patch
* ocfs2-dlm-optimization-of-code-while-free-dead-node-locks.patch
* ocfs2-dlm-optimization-of-code-while-free-dead-node-locks-checkpatch-fixes.patch
* block-restore-proc-partitions-to-not-display-non-partitionable-removable-devices.patch
mm.patch
* mm-page_alloc-return-0-in-case-this-node-has-no-page-within-the-zone.patch
* mm-vmscan-do-not-pass-reclaimed-slab-to-vmpressure.patch
* mm-page_owner-align-with-pageblock_nr-pages.patch
* mm-walk-the-zone-in-pageblock_nr_pages-steps.patch
* kdump-vmcoreinfo-report-actual-value-of-phys_base.patch
* uapi-fix-linux-sysctlh-userspace-compilation-errors.patch
* scripts-gdb-add-lx-fdtdump-command.patch
* kernel-reboot-add-devm_register_reboot_notifier.patch
* kernel-reboot-add-devm_register_reboot_notifier-fix.patch
* fault-inject-support-systematic-fault-injection.patch
* fault-inject-support-systematic-fault-injection-fix.patch
* fault-inject-automatically-detect-the-number-base-for-fail-nth-write-interface.patch
* fault-inject-parse-as-natural-1-based-value-for-fail-nth-write-interface.patch
* fault-inject-make-fail-nth-read-write-interface-symmetric.patch
* fault-inject-simplify-access-check-for-fail-nth.patch
* fault-inject-simplify-access-check-for-fail-nth-fix.patch
* fault-inject-add-proc-pid-fail-nth.patch
linux-next.patch
* imx7-fix-kconfig-warning-and-build-errors.patch
* sparc64-ng4-memset-32-bits-overflow.patch
* mm-zeroing-hash-tables-in-allocator.patch
* mm-updated-callers-to-use-hash_zero-flag.patch
* mm-adaptive-hash-table-scaling.patch
* time-delete-current_fs_time-function.patch
* lib-crc-ccitt-add-ccitt-false-crc16-variant.patch
mm-add-strictlimit-knob-v2.patch
make-sure-nobodys-leaking-resources.patch
releasing-resources-with-children.patch
kernel-forkc-export-kernel_thread-to-modules.patch
mutex-subsystem-synchro-test-module.patch
slab-leaks3-default-y.patch
workaround-for-a-pci-restoring-bug.patch
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH RFC] hugetlbfs 'noautofill' mount option
From: prakash.sangappa @ 2017-05-08 22:12 UTC (permalink / raw)
To: Dave Hansen, linux-kernel, linux-mm
In-Reply-To: <48a544c4-61b3-acaf-0386-649f073602b6@intel.com>
On 05/08/2017 08:58 AM, Dave Hansen wrote:
>
> It depends on how you define the feature. I think you have three choices:
>
> 1. "Error" on page fault. Require all access to be pre-faulted.
> 2. Allow faults, but "Error" if page cache has to be allocated
> 3. Allow faults and page cache allocations, but error on filesystem
> backing storage allocation.
>
> All of those are useful in some cases. But the implementations probably
> happen in different places:
>
> #1 can be implemented in core mm code
> #2 can be implemented in the VFS
> #3 needs filesystem involvement
Sure it will depend on how we define the feature.
However, I am not clear about how useful are #1 & #2
as a general feature with respect to filesystems, but I
assume we could find some use cases for them.
Regarding #3 as a general feature, do we want to
consider this and the complexity associated with the
implementation?
>
>> In case of hugetlbfs it is much straight forward. Since this
>> filesystem is not like a normal filesystems and and the file sizes
>> are multiple of huge pages. The hole will be a multiple of the huge
>> page size. For this reason then should the advise be specific to
>> hugetlbfs?
> Let me paraphrase: it's simpler to implement it if it's specific to
> hugetlbfs, thus we should implement it only for hugetlbfs, and keep it
> specific to hugetlbfs.
>
> The bigger question is: do we want to continue adding to the complexity
> of hugetlbfs and increase its divergence from the core mm?
Ok,
The purpose of hugetlbfs is to enable applications to be
able to use memory in huge page sizes. Expecting that there
will be no change to its purpose other then this. The filesystem
API fallocate(), with the recent addition for hole punching support
to free pages, allows explicit control on page
allocation / deallocation which is useful.
It seems that the 'noautofill' feature is what is missing, with
regards to applications having explicit control on memory page
allocations using hugetlbfs. Even though the description for this
feature is not to fill holes in files, given it is filesystem semantic, but
actually the intent is to indicate not allocating pages implicitly as
the application is primarily dealing with memory allocation and
deallocation here. Is this a good enough justification?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/1] mm: Rework slab bitmasks
From: kbuild test robot @ 2017-05-08 20:48 UTC (permalink / raw)
To: Igor Stoppa; +Cc: kbuild-all, mhocko, linux-mm
In-Reply-To: <20170508132022.15488-2-igor.stoppa@huawei.com>
[-- Attachment #1: Type: text/plain, Size: 3466 bytes --]
Hi Igor,
[auto build test WARNING on linus/master]
[also build test WARNING on v4.11]
[cannot apply to next-20170508]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Igor-Stoppa/mm-Rework-slab-bitmasks/20170508-214149
config: arm-allmodconfig (attached as .config)
compiler: arm-linux-gnueabi-gcc (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm
All warnings (new ones prefixed by >>):
>> drivers/gpu/drm/exynos/exynos5433_drm_decon.c:331:0: warning: "BIT_VAL" redefined
#define BIT_VAL(x, e, s) (((x) & ((1 << ((e) - (s) + 1)) - 1)) << (s))
In file included from include/drm/drmP.h:54:0,
from drivers/gpu/drm/exynos/exynos_drm_drv.h:18,
from drivers/gpu/drm/exynos/exynos5433_drm_decon.c:24:
include/linux/slab.h:19:0: note: this is the location of the previous definition
#define BIT_VAL(val, shift) __BIT_VAL(val, shift)
vim +/BIT_VAL +331 drivers/gpu/drm/exynos/exynos5433_drm_decon.c
b2192073 Andrzej Hajda 2015-10-20 315 decon_set_bits(ctx, DECON_SHADOWCON, SHADOWCON_Wx_PROTECT(win),
b2192073 Andrzej Hajda 2015-10-20 316 protect ? ~0 : 0);
c8466a91 Joonyoung Shim 2015-06-12 317 }
c8466a91 Joonyoung Shim 2015-06-12 318
d29c2c14 Marek Szyprowski 2016-01-05 319 static void decon_atomic_begin(struct exynos_drm_crtc *crtc)
cc5a7b35 Hyungwon Hwang 2015-08-27 320 {
cc5a7b35 Hyungwon Hwang 2015-08-27 321 struct decon_context *ctx = crtc->ctx;
d29c2c14 Marek Szyprowski 2016-01-05 322 int i;
cc5a7b35 Hyungwon Hwang 2015-08-27 323
7b6bb6ed Andrzej Hajda 2015-10-20 324 if (test_bit(BIT_SUSPENDED, &ctx->flags))
cc5a7b35 Hyungwon Hwang 2015-08-27 325 return;
cc5a7b35 Hyungwon Hwang 2015-08-27 326
d29c2c14 Marek Szyprowski 2016-01-05 327 for (i = ctx->first_win; i < WINDOWS_NR; i++)
d29c2c14 Marek Szyprowski 2016-01-05 328 decon_shadow_protect_win(ctx, i, true);
cc5a7b35 Hyungwon Hwang 2015-08-27 329 }
cc5a7b35 Hyungwon Hwang 2015-08-27 330
b8182832 Andrzej Hajda 2015-10-20 @331 #define BIT_VAL(x, e, s) (((x) & ((1 << ((e) - (s) + 1)) - 1)) << (s))
b8182832 Andrzej Hajda 2015-10-20 332 #define COORDINATE_X(x) BIT_VAL((x), 23, 12)
b8182832 Andrzej Hajda 2015-10-20 333 #define COORDINATE_Y(x) BIT_VAL((x), 11, 0)
b8182832 Andrzej Hajda 2015-10-20 334
1e1d1393 Gustavo Padovan 2015-08-03 335 static void decon_update_plane(struct exynos_drm_crtc *crtc,
1e1d1393 Gustavo Padovan 2015-08-03 336 struct exynos_drm_plane *plane)
c8466a91 Joonyoung Shim 2015-06-12 337 {
0114f404 Marek Szyprowski 2015-11-30 338 struct exynos_drm_plane_state *state =
0114f404 Marek Szyprowski 2015-11-30 339 to_exynos_plane_state(plane->base.state);
:::::: The code at line 331 was first introduced by commit
:::::: b8182832c5a9d9ce645d53be84e5db07f8aa5302 drm/exynos/decon5433: add support for DECON-TV
:::::: TO: Andrzej Hajda <a.hajda@samsung.com>
:::::: CC: Inki Dae <inki.dae@samsung.com>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 62197 bytes --]
^ permalink raw reply
* Re: [mm/usercopy] 517e1fbeb6: kernel BUG at arch/x86/mm/physaddr.c:78!
From: Kees Cook @ 2017-05-08 19:15 UTC (permalink / raw)
To: Laura Abbott; +Cc: x86@kernel.org, LKP, kernel test robot, LKML, Linux-MM, wfg
In-Reply-To: <851cb32e-e9da-a710-6e4d-ed2e8790ae70@redhat.com>
On Mon, May 8, 2017 at 11:41 AM, Laura Abbott <labbott@redhat.com> wrote:
> On 05/07/2017 07:51 AM, Kees Cook wrote:
>> On Sun, May 7, 2017 at 2:06 AM, kernel test robot
>> <fengguang.wu@intel.com> wrote:
>>> Greetings,
>>>
>>> 0day kernel testing robot got the below dmesg and the first bad commit is
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
>>>
>>> commit 517e1fbeb65f5eade8d14f46ac365db6c75aea9b
>>> Author: Laura Abbott <labbott@redhat.com>
>>> AuthorDate: Tue Apr 4 14:09:00 2017 -0700
>>> Commit: Kees Cook <keescook@chromium.org>
>>> CommitDate: Wed Apr 5 12:30:18 2017 -0700
>>>
>>> mm/usercopy: Drop extra is_vmalloc_or_module() check
>>>
>>> Previously virt_addr_valid() was insufficient to validate if virt_to_page()
>>> could be called on an address on arm64. This has since been fixed up so
>>> there is no need for the extra check. Drop it.
>>>
>>> Signed-off-by: Laura Abbott <labbott@redhat.com>
>>> Acked-by: Mark Rutland <mark.rutland@arm.com>
>>> Signed-off-by: Kees Cook <keescook@chromium.org>
>>
>> This appears to be from CONFIG_DEBUG_VIRTUAL on __phys_addr, used by
>> hardened usercopy, probably during virt_addr_valid(). I'll take a
>> closer look on Monday...
>>
>> -Kees
>>
>
> So this looks like a strange edge case/bug on x86 32-bit.
> virt_addr_valid is returning true on vmalloc addresses because
> __vmalloc_start_set is never getting set because the below
> configuration uses CONFIG_NEED_MULTIPLE_NODES=y and that variable
> only gets set with CONFIG_NEED_MULTIPLE_NODES=n currently. If
> I set it in arch/x86/mm/numa_32.c, it seems to work:
>
> Thanks,
> Laura
>
>
> diff --git a/arch/x86/mm/numa_32.c b/arch/x86/mm/numa_32.c
> index 6b7ce62..aca6295 100644
> --- a/arch/x86/mm/numa_32.c
> +++ b/arch/x86/mm/numa_32.c
> @@ -100,5 +100,6 @@ void __init initmem_init(void)
> printk(KERN_DEBUG "High memory starts at vaddr %08lx\n",
> (ulong) pfn_to_kaddr(highstart_pfn));
>
> + __vmalloc_start_set = true;
> setup_bootmem_allocator();
> }
Ah, nice catch. Can you send this as a normal patch for Ingo to apply?
-Kees
--
Kees Cook
Pixel Security
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [mm/usercopy] 517e1fbeb6: kernel BUG at arch/x86/mm/physaddr.c:78!
From: Laura Abbott @ 2017-05-08 18:41 UTC (permalink / raw)
To: Kees Cook, x86@kernel.org; +Cc: LKP, kernel test robot, LKML, Linux-MM, wfg
In-Reply-To: <CAGXu5jKwONoDb=LdAYEk99QKSV=TUqfyiQkMZK2AVxGwhyp0uw@mail.gmail.com>
On 05/07/2017 07:51 AM, Kees Cook wrote:
> On Sun, May 7, 2017 at 2:06 AM, kernel test robot
> <fengguang.wu@intel.com> wrote:
>> Greetings,
>>
>> 0day kernel testing robot got the below dmesg and the first bad commit is
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
>>
>> commit 517e1fbeb65f5eade8d14f46ac365db6c75aea9b
>> Author: Laura Abbott <labbott@redhat.com>
>> AuthorDate: Tue Apr 4 14:09:00 2017 -0700
>> Commit: Kees Cook <keescook@chromium.org>
>> CommitDate: Wed Apr 5 12:30:18 2017 -0700
>>
>> mm/usercopy: Drop extra is_vmalloc_or_module() check
>>
>> Previously virt_addr_valid() was insufficient to validate if virt_to_page()
>> could be called on an address on arm64. This has since been fixed up so
>> there is no need for the extra check. Drop it.
>>
>> Signed-off-by: Laura Abbott <labbott@redhat.com>
>> Acked-by: Mark Rutland <mark.rutland@arm.com>
>> Signed-off-by: Kees Cook <keescook@chromium.org>
>
> This appears to be from CONFIG_DEBUG_VIRTUAL on __phys_addr, used by
> hardened usercopy, probably during virt_addr_valid(). I'll take a
> closer look on Monday...
>
> -Kees
>
So this looks like a strange edge case/bug on x86 32-bit.
virt_addr_valid is returning true on vmalloc addresses because
__vmalloc_start_set is never getting set because the below
configuration uses CONFIG_NEED_MULTIPLE_NODES=y and that variable
only gets set with CONFIG_NEED_MULTIPLE_NODES=n currently. If
I set it in arch/x86/mm/numa_32.c, it seems to work:
Thanks,
Laura
diff --git a/arch/x86/mm/numa_32.c b/arch/x86/mm/numa_32.c
index 6b7ce62..aca6295 100644
--- a/arch/x86/mm/numa_32.c
+++ b/arch/x86/mm/numa_32.c
@@ -100,5 +100,6 @@ void __init initmem_init(void)
printk(KERN_DEBUG "High memory starts at vaddr %08lx\n",
(ulong) pfn_to_kaddr(highstart_pfn));
+ __vmalloc_start_set = true;
setup_bootmem_allocator();
}
>>
>> 96dc4f9fb6 usercopy: Move enum for arch_within_stack_frames()
>> 517e1fbeb6 mm/usercopy: Drop extra is_vmalloc_or_module() check
>> 13e0988140 docs: complete bumping minimal GNU Make version to 3.81
>> 9e597e815f Add linux-next specific files for 20170505
>> +------------------------------------------------------+------------+------------+------------+---------------+
>> | | 96dc4f9fb6 | 517e1fbeb6 | 13e0988140 | next-20170505 |
>> +------------------------------------------------------+------------+------------+------------+---------------+
>> | boot_successes | 35 | 3 | 6 | 0 |
>> | boot_failures | 0 | 12 | 13 | 18 |
>> | kernel_BUG_at_arch/x86/mm/physaddr.c | 0 | 12 | 13 | 13 |
>> | invalid_opcode:#[##] | 0 | 12 | 13 | 13 |
>> | EIP:__phys_addr | 0 | 12 | 13 | 13 |
>> | Kernel_panic-not_syncing:Fatal_exception | 0 | 12 | 13 | 13 |
>> | WARNING:at_kernel/cpu.c:#lockdep_assert_hotplug_held | 0 | 0 | 0 | 18 |
>> | EIP:lockdep_assert_hotplug_held | 0 | 0 | 0 | 18 |
>> +------------------------------------------------------+------------+------------+------------+---------------+
>>
>> [main] Setsockopt(1 22 80d3000 4) on fd 47 [1:5:1]
>> [ 18.665929] sock: process `trinity-main' is using obsolete setsockopt SO_BSDCOMPAT
>> [main] Setsockopt(1 e 80d3000 90) on fd 49 [1:2:1]
>> [main] Setsockopt(10e 5 80d3000 4) on fd 52 [16:3:16]
>> [ 18.668412] ------------[ cut here ]------------
>> [ 18.668824] kernel BUG at arch/x86/mm/physaddr.c:78!
>> [ 18.669424] invalid opcode: 0000 [#1] SMP
>> [ 18.669776] CPU: 0 PID: 754 Comm: trinity-main Not tainted 4.11.0-rc2-00002-g517e1fb #1
>> [ 18.670469] task: 4ca52e80 task.stack: 4c572000
>> [ 18.670860] EIP: __phys_addr+0x120/0x130
>> [ 18.671189] EFLAGS: 00010202 CPU: 0
>> [ 18.671482] EAX: 0000ff01 EBX: 50851020 ECX: 00000000 EDX: 00000001
>> [ 18.672025] ESI: 0000ff01 EDI: 10851020 EBP: 4c573e70 ESP: 4c573e60
>> [ 18.672557] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
>> [ 18.673025] CR0: 80050033 CR2: 084da000 CR3: 0c65c4a0 CR4: 001406f0
>> [ 18.673560] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>> [ 18.674100] DR6: fffe0ff0 DR7: 00000400
>> [ 18.674420] Call Trace:
>> [ 18.674632] __check_object_size+0xff/0x42f
>> [ 18.674988] ? __might_sleep+0x8e/0x130
>> [ 18.675310] __get_filter+0xaa/0x130
>> [ 18.675612] sk_attach_filter+0x15/0x90
>> [ 18.675937] sock_setsockopt+0x6b3/0x960
>> [ 18.676263] SyS_socketcall+0x773/0x810
>> [ 18.676585] ? __do_page_fault+0x36c/0x730
>> [ 18.676932] do_int80_syscall_32+0x8a/0x230
>> [ 18.677307] ? prepare_exit_to_usermode+0x38/0x60
>> [ 18.677712] entry_INT80_32+0x2f/0x2f
>> [ 18.678034] EIP: 0x37688a42
>> [ 18.678278] EFLAGS: 00000202 CPU: 0
>> [ 18.678580] EAX: ffffffda EBX: 0000000e ECX: 3fc2da40 EDX: 3fc2dac0
>> [ 18.679099] ESI: 00000004 EDI: 00000035 EBP: 3753f1ac ESP: 3fc2da3c
>> [ 18.679618] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
>> [ 18.680069] Code: 00 00 e0 ff 2d 00 20 00 00 39 c3 0f 83 47 ff ff ff c7 04 24 00 00 00 00 31 c9 ba 01 00 00 00 b8 98 e7 1a 42 e8 22 3e 0d 00 0f 0b <0f> 0b 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00 55 89 e5 53 3e
>> [ 18.681652] EIP: __phys_addr+0x120/0x130 SS:ESP: 0068:4c573e60
>> [ 18.682174] ---[ end trace bbf34582d6d63d7a ]---
>> [ 18.682636] Kernel panic - not syncing: Fatal exception
>>
>> # HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
>> git bisect start 773f7f5cf2d18eb40343d1e4e9a49062739e0425 a351e9b9fc24e982ec2f0e76379a49826036da12 --
>> git bisect bad 39af3d3d90897d17d79bc655068cf09a717a0e68 # 12:26 B 0 4 15 0 Merge 'mellanox/queue-next' into devel-spot-201705070851
>> git bisect bad 32f465722603afc8d3d90ad9fb999095afe11205 # 12:42 B 0 11 22 0 Merge 'linux-review/David-Ahern/net-reducing-memory-footprint-of-network-devices/20170507-031536' into devel-spot-201705070851
>> git bisect bad 1cbccce1b4565d60c4d9a5bc3aaf8d63b5b9224f # 12:53 B 0 11 22 0 Merge 'linux-review/Geliang-Tang/yam-use-memdup_user/20170507-045454' into devel-spot-201705070851
>> git bisect bad 408133c058c5492c03ff9f3827ccdb65b42cb842 # 13:06 B 0 11 22 0 Merge 'linux-review/Christophe-JAILLET/firmware-Google-VPD-Fix-memory-allocation-error-handling/20170507-064549' into devel-spot-201705070851
>> git bisect bad d5f6ce59cba315fc39f8bdd594d9a6ec7633be45 # 13:14 B 0 1 12 0 Merge 'linux-review/Geert-Uytterhoeven/signal-Export-signal_wake_up_state-to-modules/20170507-082935' into devel-spot-201705070851
>> git bisect good 163f34fcdf2791ac0e609d59440a9ef90d2bf3d2 # 13:34 G 11 0 0 0 0day base guard for 'devel-spot-201705070851'
>> git bisect good ddd92361062a7eb9708eb6c633346c35d0d67d2f # 13:45 G 11 0 0 0 Merge 'linux-review/Geliang-Tang/platform-x86-toshiba_acpi-use-memdup_user_nul/20170507-083752' into devel-spot-201705070851
>> git bisect bad a3719f34fdb664ffcfaec2160ef20fca7becf2ee # 13:57 B 0 11 22 0 Merge branch 'generic' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs
>> git bisect good 5d15af6778b8e4ed1fd41b040283af278e7a9a72 # 14:11 G 11 0 0 0 Merge branch 'tipc-refactor-socket-receive-functions'
>> git bisect good 7c8c03bfc7b9f5211d8a69eab7fee99c9fb4f449 # 14:21 G 11 0 0 0 Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
>> git bisect bad 8d65b08debc7e62b2c6032d7fe7389d895b92cbc # 14:30 B 0 11 22 0 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
>> git bisect good b68e7e952f24527de62f4768b1cead91f92f5f6e # 14:40 G 11 0 0 0 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
>> git bisect bad 5b13475a5e12c49c24422ba1bd9998521dec1d4e # 14:51 B 0 11 22 0 Merge branch 'work.iov_iter' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
>> git bisect good 0cb300623e3bb460fd9853bbde2fd1973e3bbcd8 # 15:01 G 11 0 0 0 usb: gadget.h: be consistent at kernel doc macros
>> git bisect good 3a7d2fd16c57a1ef47dc2891171514231c9c7c6e # 15:21 G 11 0 0 0 pstore: Solve lockdep warning by moving inode locks
>> git bisect good c58d4055c054fc6dc72f1be8bc71bd6fff209e48 # 15:35 G 11 0 0 0 Merge tag 'docs-4.12' of git://git.lwn.net/linux
>> git bisect bad 6fd4e7f7744bd7859ca3cae19c4613252ebb6bff # 15:43 B 0 11 22 0 Merge branch 'for-next' of git://git.samba.org/sfrench/cifs-2.6
>> git bisect bad 5958cc49ed2961a059d92ae55afeeaba64a783a0 # 15:51 B 0 1 12 0 Merge tag 'usercopy-v4.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux
>> git bisect bad 517e1fbeb65f5eade8d14f46ac365db6c75aea9b # 16:05 B 0 11 22 0 mm/usercopy: Drop extra is_vmalloc_or_module() check
>> git bisect good 96dc4f9fb64690fc34410415fd1fc609cf803f61 # 16:14 G 11 0 0 0 usercopy: Move enum for arch_within_stack_frames()
>> # first bad commit: [517e1fbeb65f5eade8d14f46ac365db6c75aea9b] mm/usercopy: Drop extra is_vmalloc_or_module() check
>> git bisect good 96dc4f9fb64690fc34410415fd1fc609cf803f61 # 16:17 G 31 0 0 0 usercopy: Move enum for arch_within_stack_frames()
>> # extra tests with CONFIG_DEBUG_INFO_REDUCED
>> git bisect bad 517e1fbeb65f5eade8d14f46ac365db6c75aea9b # 16:31 B 0 11 22 0 mm/usercopy: Drop extra is_vmalloc_or_module() check
>> # extra tests on HEAD of linux-devel/devel-spot-201705070851
>> git bisect bad 773f7f5cf2d18eb40343d1e4e9a49062739e0425 # 16:32 B 0 22 37 0 0day head guard for 'devel-spot-201705070851'
>> # extra tests on tree/branch linus/master
>> git bisect bad 13e0988140374123bead1dd27c287354cb95108e # 16:43 B 0 11 22 0 docs: complete bumping minimal GNU Make version to 3.81
>> # extra tests with first bad commit reverted
>> git bisect good 688e95d3e3571e6b1c08da62fc402f1c1c3d5542 # 16:53 G 10 0 0 0 Revert "mm/usercopy: Drop extra is_vmalloc_or_module() check"
>> # extra tests on tree/branch linux-next/master
>> git bisect bad 9e597e815f68867c70d1b70cb2b037b92a8ec12b # 17:06 B 0 9 27 7 Add linux-next specific files for 20170505
>>
>> ---
>> 0-DAY kernel test infrastructure Open Source Technology Center
>> https://lists.01.org/pipermail/lkp Intel Corporation
>
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [virtio-dev] Re: [PATCH v9 2/5] virtio-balloon: VIRTIO_BALLOON_F_BALLOON_CHUNKS
From: Michael S. Tsirkin @ 2017-05-08 17:40 UTC (permalink / raw)
To: Wang, Wei W
Cc: virtio-dev@lists.oasis-open.org, linux-kernel@vger.kernel.org,
qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org,
kvm@vger.kernel.org, linux-mm@kvack.org, david@redhat.com,
Hansen, Dave, cornelia.huck@de.ibm.com, akpm@linux-foundation.org,
mgorman@techsingularity.net, aarcange@redhat.com,
amit.shah@redhat.com, pbonzini@redhat.com,
liliang.opensource@gmail.com
In-Reply-To: <286AC319A985734F985F78AFA26841F7391FFBB0@shsmsx102.ccr.corp.intel.com>
On Sun, May 07, 2017 at 04:19:28AM +0000, Wang, Wei W wrote:
> On 05/06/2017 06:26 AM, Michael S. Tsirkin wrote:
> > On Thu, Apr 27, 2017 at 02:31:49PM +0800, Wei Wang wrote:
> > > On 04/27/2017 07:20 AM, Michael S. Tsirkin wrote:
> > > > On Wed, Apr 26, 2017 at 11:03:34AM +0000, Wang, Wei W wrote:
> > > > > Hi Michael, could you please give some feedback?
> > > > I'm sorry, I'm not sure feedback on what you are requesting.
> > > Oh, just some trivial things (e.g. use a field in the header,
> > > hdr->chunks to indicate the number of chunks in the payload) that
> > > wasn't confirmed.
> > >
> > > I will prepare the new version with fixing the agreed issues, and we
> > > can continue to discuss those parts if you still find them improper.
> > >
> > >
> > > >
> > > > The interface looks reasonable now, even though there's a way to
> > > > make it even simpler if we can limit chunk size to 2G (in fact 4G -
> > > > 1). Do you think we can live with this limitation?
> > > Yes, I think we can. So, is it good to change to use the previous
> > > 64-bit chunk format (52-bit base + 12-bit size)?
> >
> > This isn't what I meant. virtio ring has descriptors with a 64 bit address and 32 bit
> > size.
> >
> > If size < 4g is not a significant limitation, why not just use that to pass
> > address/size in a standard s/g list, possibly using INDIRECT?
>
> OK, I see your point, thanks. Post the two options here for an analysis:
> Option1 (what we have now):
> struct virtio_balloon_page_chunk {
> __le64 chunk_num;
> struct virtio_balloon_page_chunk_entry entry[];
> };
> Option2:
> struct virtio_balloon_page_chunk {
> __le64 chunk_num;
> struct scatterlist entry[];
> };
This isn't what I meant really :) I meant vring_desc.
> I don't have an issue to change it to Option2, but I would prefer Option1,
> because I think there is no be obvious difference between the two options,
> while Option1 appears to have little advantages here:
> 1) "struct virtio_balloon_page_chunk_entry" has smaller size than
> "struct scatterlist", so the same size of allocated page chunk buffer
> can hold more entry[] using Option1;
> 2) INDIRECT needs on demand kmalloc();
Within alloc_indirect? We can fix that with a separate patch.
> 3) no 4G size limit;
Do you see lots of >=4g chunks in practice?
> What do you think?
>
> Best,
> Wei
>
>
OTOH using existing vring APIs handles things like DMA transparently.
--
MST
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 1/2] dax: prevent invalidation of mapped DAX entries
From: Ross Zwisler @ 2017-05-08 17:08 UTC (permalink / raw)
To: Jan Kara
Cc: Ross Zwisler, Andrew Morton, linux-kernel, Alexander Viro,
Alexey Kuznetsov, Andrey Ryabinin, Anna Schumaker,
Christoph Hellwig, Dan Williams, Darrick J. Wong,
Eric Van Hensbergen, Jens Axboe, Johannes Weiner,
Konrad Rzeszutek Wilk, Latchesar Ionkov, linux-cifs,
linux-fsdevel, linux-mm, linux-nfs, linux-nvdimm, Matthew Wilcox,
Ron Minnich, samba-technical, Steve French, Trond Myklebust,
v9fs-developer
In-Reply-To: <20170505072912.GA25424@quack2.suse.cz>
On Fri, May 05, 2017 at 09:29:12AM +0200, Jan Kara wrote:
> On Thu 04-05-17 13:59:09, Ross Zwisler wrote:
> > dax_invalidate_mapping_entry() currently removes DAX exceptional entries
> > only if they are clean and unlocked. This is done via:
> >
> > invalidate_mapping_pages()
> > invalidate_exceptional_entry()
> > dax_invalidate_mapping_entry()
> >
> > However, for page cache pages removed in invalidate_mapping_pages() there
> > is an additional criteria which is that the page must not be mapped. This
> > is noted in the comments above invalidate_mapping_pages() and is checked in
> > invalidate_inode_page().
> >
> > For DAX entries this means that we can can end up in a situation where a
> > DAX exceptional entry, either a huge zero page or a regular DAX entry,
> > could end up mapped but without an associated radix tree entry. This is
> > inconsistent with the rest of the DAX code and with what happens in the
> > page cache case.
> >
> > We aren't able to unmap the DAX exceptional entry because according to its
> > comments invalidate_mapping_pages() isn't allowed to block, and
> > unmap_mapping_range() takes a write lock on the mapping->i_mmap_rwsem.
> >
> > We could potentially do an rmap walk to see if each of the entries actually
> > has any active mappings before we remove it, but this might end up being
> > very expensive and doesn't currently look to be worth it.
> >
> > So, just remove dax_invalidate_mapping_entry() and leave the DAX entries in
> > the radix tree.
> >
> > Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
> > Fixes: c6dcf52c23d2 ("mm: Invalidate DAX radix tree entries only if appropriate")
> > Reported-by: Jan Kara <jack@suse.cz>
> > Reviewed-by: Jan Kara <jack@suse.cz>
> > Cc: <stable@vger.kernel.org> [4.10+]
>
> Ah, I've just sent out a series which contains these two patches and
> another two patches which change the entry locking to fix the last spotted
> race... So either just take my last two patches on top of these two or
> take my series as a whole.
Sounds good. You added a better comment in invalidate_inode_pages2_range(), so
let's just use your version of this series.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 00/10] x86 TLB flush cleanups, moving toward PCID support
From: Nadav Amit @ 2017-05-08 16:36 UTC (permalink / raw)
To: Andy Lutomirski
Cc: X86 ML, linux-kernel@vger.kernel.org, Borislav Petkov,
Linus Torvalds, Andrew Morton, Mel Gorman, linux-mm@kvack.org
In-Reply-To: <cover.1494160201.git.luto@kernel.org>
> On May 7, 2017, at 5:38 AM, Andy Lutomirski <luto@kernel.org> wrote:
>
> As I've been working on polishing my PCID code, a major problem I've
> encountered is that there are too many x86 TLB flushing code paths and
> that they have too many inconsequential differences. The result was
> that earlier versions of the PCID code were a colossal mess and very
> difficult to understand.
>
> This series goes a long way toward cleaning up the mess. With all the
> patches applied, there is a single function that contains the meat of
> the code to flush the TLB on a given CPU, and all the tlb flushing
> APIs call it for both local and remote CPUs.
>
> This series should only adversely affect the kernel in a couple of
> minor ways:
>
> - It makes smp_mb() unconditional when flushing TLBs. We used to
> use the TLB flush itself to mostly avoid smp_mb() on the initiating
> CPU.
>
> - On UP kernels, we lose the dubious optimization of inlining nerfed
> variants of all the TLB flush APIs. This bloats the kernel a tiny
> bit, although it should increase performance, since the SMP
> versions were better.
>
> Patch 10 in here is a little bit off topic. It's a cleanup that's
> also needed before PCID can go in, but it's not directly about
> TLB flushing.
>
> Thoughts?
In general I like the changes. I needed to hack Linux TLB shootdowns for
a research project just because I could not handle the code otherwise.
I ended up doing some of changes that you have done.
I just have two general comments:
- You may want to consider merging the kernel mappings invalidation
with the userspace mappings invalidations as well, since there are
still code redundancies.
- Don’t expect too much from concurrent TLB invalidations. In many
cases the IPI latency dominates the overhead from my experience.
Regards,
Nadav
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH RFC] hugetlbfs 'noautofill' mount option
From: Dave Hansen @ 2017-05-08 15:58 UTC (permalink / raw)
To: Prakash Sangappa, linux-kernel, linux-mm
In-Reply-To: <7677d20e-5d53-1fb7-5dac-425edda70b7b@oracle.com>
On 05/03/2017 12:02 PM, Prakash Sangappa wrote:
>>> If we do consider a new madvise()option, will it be acceptable
>>> since this will be specifically for hugetlbfs file mappings?
>> Ideally, it would be something that is *not* specifically for
>> hugetlbfs. MADV_NOAUTOFILL, for instance, could be defined to
>> SIGSEGV whenever memory is touched that was not populated with
>> MADV_WILLNEED, mlock(), etc...
>
> If this is a generic advice type, necessary support will have to be
> implemented in various filesystems which can support this.
Yep.
> The proposed behavior for 'noautofill' was to not fill holes in
> files(like sparse files). In the page fault path, mm would not know
> if the mmapped address on which the fault occurred, is over a hole in
> the file or just that the page is not available in the page cache.
It depends on how you define the feature. I think you have three choices:
1. "Error" on page fault. Require all access to be pre-faulted.
2. Allow faults, but "Error" if page cache has to be allocated
3. Allow faults and page cache allocations, but error on filesystem
backing storage allocation.
All of those are useful in some cases. But the implementations probably
happen in different places:
#1 can be implemented in core mm code
#2 can be implemented in the VFS
#3 needs filesystem involvement
> The underlying filesystem would be called and it determines if it is
> a hole and that is where it would fail and not fill the hole, if this
> support is added. Normally, filesystem which support sparse
> files(holes in file) automatically fill the hole when accessed. Then
> there is the issue of file system block size and page size. If the
> block sizes are smaller then page size, it could mean the noautofill
> would only work if the hole size is equal to or a multiple of, page
> size?
It depends on how you define the feature whether this is true.
> In case of hugetlbfs it is much straight forward. Since this
> filesystem is not like a normal filesystems and and the file sizes
> are multiple of huge pages. The hole will be a multiple of the huge
> page size. For this reason then should the advise be specific to
> hugetlbfs?
Let me paraphrase: it's simpler to implement it if it's specific to
hugetlbfs, thus we should implement it only for hugetlbfs, and keep it
specific to hugetlbfs.
The bigger question is: do we want to continue adding to the complexity
of hugetlbfs and increase its divergence from the core mm?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 03/10] x86/mm: Make the batched unmap TLB flush API more generic
From: Dave Hansen @ 2017-05-08 15:34 UTC (permalink / raw)
To: Andy Lutomirski, X86 ML
Cc: linux-kernel@vger.kernel.org, Borislav Petkov, Linus Torvalds,
Andrew Morton, Mel Gorman, linux-mm@kvack.org, Rik van Riel,
Nadav Amit, Michal Hocko, Sasha Levin
In-Reply-To: <983c5ee661d8fe8a70c596c4e77076d11ce3f80a.1494160201.git.luto@kernel.org>
On 05/07/2017 05:38 AM, Andy Lutomirski wrote:
> diff --git a/mm/rmap.c b/mm/rmap.c
> index f6838015810f..2e568c82f477 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -579,25 +579,12 @@ void page_unlock_anon_vma_read(struct anon_vma *anon_vma)
> void try_to_unmap_flush(void)
> {
> struct tlbflush_unmap_batch *tlb_ubc = ¤t->tlb_ubc;
> - int cpu;
>
> if (!tlb_ubc->flush_required)
> return;
>
> - cpu = get_cpu();
> -
> - if (cpumask_test_cpu(cpu, &tlb_ubc->cpumask)) {
> - count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
> - local_flush_tlb();
> - trace_tlb_flush(TLB_LOCAL_SHOOTDOWN, TLB_FLUSH_ALL);
> - }
> -
> - if (cpumask_any_but(&tlb_ubc->cpumask, cpu) < nr_cpu_ids)
> - flush_tlb_others(&tlb_ubc->cpumask, NULL, 0, TLB_FLUSH_ALL);
> - cpumask_clear(&tlb_ubc->cpumask);
> tlb_ubc->flush_required = false;
> tlb_ubc->writable = false;
> - put_cpu();
> }
>
> /* Flush iff there are potentially writable TLB entries that can race with IO */
> @@ -613,7 +600,7 @@ static void set_tlb_ubc_flush_pending(struct mm_struct *mm, bool writable)
> {
> struct tlbflush_unmap_batch *tlb_ubc = ¤t->tlb_ubc;
>
> - cpumask_or(&tlb_ubc->cpumask, &tlb_ubc->cpumask, mm_cpumask(mm));
> + arch_tlbbatch_add_mm(&tlb_ubc->arch, mm);
> tlb_ubc->flush_required = true;
>
> /*
Looking at this patch in isolation, how can this be safe? It removes
TLB flushes from the generic code. Do other patches in the series fix
this up?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Laura Abbott @ 2017-05-08 15:25 UTC (permalink / raw)
To: Igor Stoppa, Michal Hocko
Cc: linux-mm, linux-kernel, kernel-hardening@lists.openwall.com
In-Reply-To: <0b55343e-4305-a9f1-2b17-51c3c734aea6@huawei.com>
On 05/05/2017 03:42 AM, Igor Stoppa wrote:
> On 04/05/17 19:49, Laura Abbott wrote:
>> [adding kernel-hardening since I think there would be interest]
>
> thank you, I overlooked this
>
>
>> BPF takes the approach of calling set_memory_ro to mark regions as
>> read only. I'm certainly over simplifying but it sounds like this
>> is mostly a mechanism to have this happen mostly automatically.
>> Can you provide any more details about tradeoffs of the two approaches?
>
> I am not sure I understand the question ...
> For what I can understand, the bpf is marking as read only something
> that spans across various pages, which is fine.
> The payload to be protected is already organized in such pages.
>
> But in the case I have in mind, I have various, heterogeneous chunks of
> data, coming from various subsystems, not necessarily page aligned.
> And, even if they were page aligned, most likely they would be far
> smaller than a page, even a 4k page.
>
> The first problem I see, is how to compact them into pages, ensuring
> that no rwdata manages to infiltrate the range.
>
> The actual mechanism for marking pages as read only is not relevant at
> this point, if I understand your question correctly, since set_memory_ro
> is walking the pages it receives as parameter.
>
Thanks for clarifying, this makes sense. I also saw some replies up
thread that also answered some my questions.
>> arm and arm64 have the added complexity of using larger
>> page sizes on the linear map so dynamic mapping/unmapping generally
>> doesn't work.
>
> Do you mean that a page could be 16MB and therefore it would not be
> possible to get a smaller chunk?
>
Roughly yes.
PAGE_SIZE is still 4K/16K/64K but the underlying page table mappings
may use larger mappings (2MB, 32M, 512M, etc.). The ARM architecture
has a break-before-make requirement which requires old mappings be
fully torn down and invalidated to avoid TLB conflicts. This is nearly
impossible to do correctly on live page tables so the current policy
is to not break down larger mappings.
>> arm64 supports DEBUG_PAGEALLOC by mapping with only
>> pages but this is generally only wanted as a debug mechanism.
>> I don't know if you've given this any thought at all.
>
> Since the beginning I have thought about this feature as an opt-in
> feature. I am aware that it can have drawbacks, but I think it would be
> valuable as debugging tool even where it's not feasible to keep it
> always-on.
>
> OTOH on certain systems it can be sufficiently appealing to be kept on,
> even if it eats up some more memory.
I'd rather see this designed as being mandatory from the start and then
provide a mechanism to turn it off if necessary. The uptake and
coverage from opt-in features tends to be very low based on past experience.
Thanks,
Laura
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH 1/1] mm: Rework slab bitmasks
From: Igor Stoppa @ 2017-05-08 13:20 UTC (permalink / raw)
To: mhocko; +Cc: linux-mm, Igor Stoppa
In-Reply-To: <20170508132022.15488-1-igor.stoppa@huawei.com>
The bitmasks defined in the slab header can be made more readable by
using the BIT() macro.
Furthermore, several conditional definitions can be collapsed, by
expressing their value as a function of the configuration paramter that
controles them, using the macro IS_ENABLED().
Signed-off-by: Igor Stoppa <igor.stoppa@huawei.com>
---
include/linux/slab.h | 71 +++++++++++++++++++++++-----------------------------
1 file changed, 31 insertions(+), 40 deletions(-)
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 3c37a8c..f7e55f0 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -15,18 +15,24 @@
#include <linux/types.h>
#include <linux/workqueue.h>
+#define __BIT_VAL(val, shift) (((unsigned long)(val != 0)) << (shift))
+#define BIT_VAL(val, shift) __BIT_VAL(val, shift)
+#define BIT_CFG(cfg, shift) __BIT_VAL(IS_ENABLED(CONFIG_##cfg), shift)
+#define BIT_DBG(dbg, shift) BIT_CFG(DEBUG_##dbg, shift)
+#define BIT_DBG_SLAB(shift) BIT_DBG(SLAB, shift)
/*
* Flags to pass to kmem_cache_create().
* The ones marked DEBUG are only valid if CONFIG_DEBUG_SLAB is set.
*/
-#define SLAB_CONSISTENCY_CHECKS 0x00000100UL /* DEBUG: Perform (expensive) checks on alloc/free */
-#define SLAB_RED_ZONE 0x00000400UL /* DEBUG: Red zone objs in a cache */
-#define SLAB_POISON 0x00000800UL /* DEBUG: Poison objects */
-#define SLAB_HWCACHE_ALIGN 0x00002000UL /* Align objs on cache lines */
-#define SLAB_CACHE_DMA 0x00004000UL /* Use GFP_DMA memory */
-#define SLAB_STORE_USER 0x00010000UL /* DEBUG: Store the last owner for bug hunting */
-#define SLAB_PANIC 0x00040000UL /* Panic if kmem_cache_create() fails */
+
+#define SLAB_CONSISTENCY_CHECKS BIT_DBG_SLAB(8) /* Perform (expensive) checks on alloc/free */
+#define SLAB_RED_ZONE BIT_DBG_SLAB(10) /* Red zone objs in a cache */
+#define SLAB_POISON BIT_DBG_SLAB(11) /* Poison objects */
+#define SLAB_HWCACHE_ALIGN BIT(13) /* Align objs on cache lines */
+#define SLAB_CACHE_DMA BIT(14) /* Use GFP_DMA memory */
+#define SLAB_STORE_USER BIT_DBG_SLAB(16) /* Store the last owner for bug hunting */
+#define SLAB_PANIC BIT(18) /* Panic if kmem_cache_create() fails */
/*
* SLAB_DESTROY_BY_RCU - **WARNING** READ THIS!
*
@@ -62,44 +68,29 @@
* rcu_read_lock before reading the address, then rcu_read_unlock after
* taking the spinlock within the structure expected at that address.
*/
-#define SLAB_DESTROY_BY_RCU 0x00080000UL /* Defer freeing slabs to RCU */
-#define SLAB_MEM_SPREAD 0x00100000UL /* Spread some memory over cpuset */
-#define SLAB_TRACE 0x00200000UL /* Trace allocations and frees */
+#define SLAB_DESTROY_BY_RCU BIT(19) /* Defer freeing slabs to RCU */
+#define SLAB_MEM_SPREAD BIT(20) /* Spread some memory over cpuset */
+#define SLAB_TRACE BIT(21) /* Trace allocations and frees */
/* Flag to prevent checks on free */
-#ifdef CONFIG_DEBUG_OBJECTS
-# define SLAB_DEBUG_OBJECTS 0x00400000UL
-#else
-# define SLAB_DEBUG_OBJECTS 0x00000000UL
-#endif
+# define SLAB_DEBUG_OBJECTS BIT_DBG(OBJECTS, 22)
-#define SLAB_NOLEAKTRACE 0x00800000UL /* Avoid kmemleak tracing */
+#define SLAB_NOLEAKTRACE BIT(23) /* Avoid kmemleak tracing */
/* Don't track use of uninitialized memory */
-#ifdef CONFIG_KMEMCHECK
-# define SLAB_NOTRACK 0x01000000UL
-#else
-# define SLAB_NOTRACK 0x00000000UL
-#endif
-#ifdef CONFIG_FAILSLAB
-# define SLAB_FAILSLAB 0x02000000UL /* Fault injection mark */
-#else
-# define SLAB_FAILSLAB 0x00000000UL
-#endif
-#if defined(CONFIG_MEMCG) && !defined(CONFIG_SLOB)
-# define SLAB_ACCOUNT 0x04000000UL /* Account to memcg */
-#else
-# define SLAB_ACCOUNT 0x00000000UL
-#endif
+# define SLAB_NOTRACK BIT_CFG(KMEMCHECK, 24)
-#ifdef CONFIG_KASAN
-#define SLAB_KASAN 0x08000000UL
-#else
-#define SLAB_KASAN 0x00000000UL
-#endif
+/* Fault injection mark */
+# define SLAB_FAILSLAB BIT_CFG(FAILSLAB, 25)
+
+/* Account to memcg */
+# define SLAB_ACCOUNT BIT_VAL(IS_ENABLED(CONFIG_MEMCG) && \
+ !IS_ENABLED(CONFIG_SLOB), 26)
+
+#define SLAB_KASAN BIT_CFG(KASAN, 27)
/* The following flags affect the page allocator grouping pages by mobility */
-#define SLAB_RECLAIM_ACCOUNT 0x00020000UL /* Objects are reclaimable */
+#define SLAB_RECLAIM_ACCOUNT BIT(17) /* Objects are reclaimable */
#define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */
/*
* ZERO_SIZE_PTR will be returned for zero sized kmalloc requests.
@@ -246,9 +237,9 @@ static inline const char *__check_heap_object(const void *ptr,
#endif
/* Maximum allocatable size */
-#define KMALLOC_MAX_SIZE (1UL << KMALLOC_SHIFT_MAX)
+#define KMALLOC_MAX_SIZE BIT(KMALLOC_SHIFT_MAX)
/* Maximum size for which we actually use a slab cache */
-#define KMALLOC_MAX_CACHE_SIZE (1UL << KMALLOC_SHIFT_HIGH)
+#define KMALLOC_MAX_CACHE_SIZE BIT(KMALLOC_SHIFT_HIGH)
/* Maximum order allocatable via the slab allocagtor */
#define KMALLOC_MAX_ORDER (KMALLOC_SHIFT_MAX - PAGE_SHIFT)
@@ -256,7 +247,7 @@ static inline const char *__check_heap_object(const void *ptr,
* Kmalloc subsystem.
*/
#ifndef KMALLOC_MIN_SIZE
-#define KMALLOC_MIN_SIZE (1 << KMALLOC_SHIFT_LOW)
+#define KMALLOC_MIN_SIZE BIT(KMALLOC_SHIFT_LOW)
#endif
/*
--
2.9.3
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH 0/1] mm: Use BIT macro in SLAB bitmaps
From: Igor Stoppa @ 2017-05-08 13:20 UTC (permalink / raw)
To: mhocko; +Cc: linux-mm, Igor Stoppa
The file include/linux/slab.h can be simplified by moving to use the
macro BIT() and making other bitmaps depend on their correspactive
master-toggle configuration option.
Igor Stoppa (1):
Rework slab bitmasks
include/linux/slab.h | 71 +++++++++++++++++++++++-----------------------------
1 file changed, 31 insertions(+), 40 deletions(-)
--
2.9.3
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v5 15/32] efi: Update efi_mem_type() to return an error rather than 0
From: Tom Lendacky @ 2017-05-08 13:20 UTC (permalink / raw)
To: Borislav Petkov
Cc: linux-arch, linux-efi, kvm, linux-doc, x86, kexec, linux-kernel,
kasan-dev, linux-mm, iommu, Rik van Riel,
Radim Krčmář, Toshimitsu Kani, Arnd Bergmann,
Jonathan Corbet, Matt Fleming, Michael S. Tsirkin, Joerg Roedel,
Konrad Rzeszutek Wilk, Paolo Bonzini, Larry Woodman,
Brijesh Singh, Ingo Molnar, Andy Lutomirski, H. Peter Anvin,
Andrey Ryabinin, Alexander Potapenko, Dave Young, Thomas Gleixner,
Dmitry Vyukov
In-Reply-To: <20170507171822.x7grrqg2tcvbv6j5@pd.tnic>
On 5/7/2017 12:18 PM, Borislav Petkov wrote:
> On Tue, Apr 18, 2017 at 04:19:00PM -0500, Tom Lendacky wrote:
>> The efi_mem_type() function currently returns a 0, which maps to
>> EFI_RESERVED_TYPE, if the function is unable to find a memmap entry for
>> the supplied physical address. Returning EFI_RESERVED_TYPE implies that
>> a memmap entry exists, when it doesn't. Instead of returning 0, change
>> the function to return a negative error value when no memmap entry is
>> found.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
>> ---
>
> ...
>
>> diff --git a/include/linux/efi.h b/include/linux/efi.h
>> index cd768a1..a27bb3f 100644
>> --- a/include/linux/efi.h
>> +++ b/include/linux/efi.h
>> @@ -973,7 +973,7 @@ static inline void efi_esrt_init(void) { }
>> extern int efi_config_parse_tables(void *config_tables, int count, int sz,
>> efi_config_table_type_t *arch_tables);
>> extern u64 efi_get_iobase (void);
>> -extern u32 efi_mem_type (unsigned long phys_addr);
>> +extern int efi_mem_type (unsigned long phys_addr);
>
> WARNING: space prohibited between function name and open parenthesis '('
> #101: FILE: include/linux/efi.h:976:
> +extern int efi_mem_type (unsigned long phys_addr);
>
> Please integrate scripts/checkpatch.pl in your patch creation workflow.
> Some of the warnings/errors *actually* make sense.
I do/did run scripts/checkpatch.pl against all my patches. In this case
I chose to keep the space in order to stay consistent with some of the
surrounding functions. No problem though, I can remove the space.
Thanks,
Tom
>
> I know, the other function prototypes have a space too but that's not
> our coding style. Looks like this trickled in from ia64, from looking at
> arch/ia64/kernel/efi.c.
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Antw: Q: si_code for SIGBUS caused by mmap() write error
From: Ulrich Windl @ 2017-05-08 13:09 UTC (permalink / raw)
To: linux-mm@kvack.org; +Cc: linux-kernel
In-Reply-To: <59106DF0020000A100026216@gwsmtp1.uni-regensburg.de>
<Stoermeldung.Infrastruktur-Ukr@rz.uni-regensburg.de>
I'm sorry for using that address as sender in my previous message; it was an oversight! The CC: address was correct, however. You can drop above address from your replies.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [v3 9/9] s390: teach platforms not to zero struct pages memory
From: Heiko Carstens @ 2017-05-08 11:36 UTC (permalink / raw)
To: Pavel Tatashin
Cc: linux-kernel, sparclinux, linux-mm, linuxppc-dev, linux-s390,
borntraeger, davem
In-Reply-To: <1494003796-748672-10-git-send-email-pasha.tatashin@oracle.com>
On Fri, May 05, 2017 at 01:03:16PM -0400, Pavel Tatashin wrote:
> If we are using deferred struct page initialization feature, most of
> "struct page"es are getting initialized after other CPUs are started, and
> hence we are benefiting from doing this job in parallel. However, we are
> still zeroing all the memory that is allocated for "struct pages" using the
> boot CPU. This patch solves this problem, by deferring zeroing "struct
> pages" to only when they are initialized on s390 platforms.
>
> Signed-off-by: Pavel Tatashin <pasha.tatashin@oracle.com>
> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
> ---
> arch/s390/mm/vmem.c | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c
> index 9c75214..ffe9ba1 100644
> --- a/arch/s390/mm/vmem.c
> +++ b/arch/s390/mm/vmem.c
> @@ -252,7 +252,7 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node)
> void *new_page;
>
> new_page = vmemmap_alloc_block(PMD_SIZE, node,
> - true);
> + VMEMMAP_ZERO);
> if (!new_page)
> goto out;
> pmd_val(*pm_dir) = __pa(new_page) | sgt_prot;
If you add the hunk below then this is
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c
index ffe9ba1aec8b..bf88a8b9c24d 100644
--- a/arch/s390/mm/vmem.c
+++ b/arch/s390/mm/vmem.c
@@ -272,7 +272,7 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node)
if (pte_none(*pt_dir)) {
void *new_page;
- new_page = vmemmap_alloc_block(PAGE_SIZE, node, true);
+ new_page = vmemmap_alloc_block(PAGE_SIZE, node, VMEMMAP_ZERO);
if (!new_page)
goto out;
pte_val(*pt_dir) = __pa(new_page) | pgt_prot;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH v2 1/2] mm: Uncharge poisoned pages
From: Laurent Dufour @ 2017-05-08 10:42 UTC (permalink / raw)
To: Balbir Singh, Michal Hocko, Andi Kleen, Johannes Weiner
Cc: Naoya Horiguchi, linux-kernel, linux-mm, akpm, Vladimir Davydov
In-Reply-To: <1493860869.8082.1.camel@gmail.com>
On 04/05/2017 03:21, Balbir Singh wrote:
>> @@ -5527,7 +5527,7 @@ static void uncharge_list(struct list_head *page_list)
>> next = page->lru.next;
>>
>> VM_BUG_ON_PAGE(PageLRU(page), page);
>> - VM_BUG_ON_PAGE(page_count(page), page);
>> + VM_BUG_ON_PAGE(!PageHWPoison(page) && page_count(page), page);
>>
>> if (!page->mem_cgroup)
>> continue;
>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>> index 8a6bd3a9eb1e..4497d9619bb4 100644
>> --- a/mm/memory-failure.c
>> +++ b/mm/memory-failure.c
>> @@ -541,6 +541,13 @@ static int delete_from_lru_cache(struct page *p)
>> */
>> ClearPageActive(p);
>> ClearPageUnevictable(p);
>> +
>> + /*
>> + * Poisoned page might never drop its ref count to 0 so we have to
>> + * uncharge it manually from its memcg.
>> + */
>> + mem_cgroup_uncharge(p);
>> +
>
> Yep, that is the right fix
>
> https://lkml.org/lkml/2017/4/26/133
Sorry Balbir,
You pointed this out since the beginning but I missed your comment.
My mistake.
Thanks,
Laurent.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v3 3/3] arm64: Silence first allocation with CONFIG_ARM64_MODULE_PLTS=y
From: Will Deacon @ 2017-05-08 10:07 UTC (permalink / raw)
To: Florian Fainelli
Cc: linux-arm-kernel, Russell King, Catalin Marinas, Ard Biesheuvel,
Andrew Morton, Michal Hocko, zijun_hu, Kirill A. Shutemov,
Andrey Ryabinin, Chris Wilson, open list,
open list:MEMORY MANAGEMENT, angus
In-Reply-To: <3af577ca-8f01-7a1c-997c-4c04914b4633@gmail.com>
On Fri, May 05, 2017 at 02:07:28PM -0700, Florian Fainelli wrote:
> On 05/03/2017 04:18 AM, Will Deacon wrote:
> > On Thu, Apr 27, 2017 at 11:19:02AM -0700, Florian Fainelli wrote:
> >> When CONFIG_ARM64_MODULE_PLTS is enabled, the first allocation using the
> >> module space fails, because the module is too big, and then the module
> >> allocation is attempted from vmalloc space. Silence the first allocation
> >> failure in that case by setting __GFP_NOWARN.
> >>
> >> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> >> ---
> >> arch/arm64/kernel/module.c | 7 ++++++-
> >> 1 file changed, 6 insertions(+), 1 deletion(-)
> >
> > I'm not sure what the merge plan is for these, but the arm64 bit here
> > looks fine to me:
> >
> > Acked-by: Will Deacon <will.deacon@arm.com>
>
> Thanks, not sure either, would you or Catalin want to pick this series?
We'd need an Ack from Russell on the arch/arm/ part before we could take
this series.
Will
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox