* Re: CONFIG_ISOLATION=y (was: [PATCH 0/6] support "dataplane" mode for nohz_full)
From: Peter Zijlstra @ 2015-05-12 12:39 UTC (permalink / raw)
To: Ingo Molnar
Cc: Chris Metcalf, Steven Rostedt, Frederic Weisbecker, Andrew Morton,
paulmck-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8, Gilad Ben Yossef,
Rik van Riel, Tejun Heo, Thomas Gleixner, Christoph Lameter,
Srivatsa S. Bhat, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150512123440.GA16959-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
On Tue, May 12, 2015 at 02:34:40PM +0200, Ingo Molnar wrote:
> Yes, that's what I meant: CONFIG_ISOLATION would trigger what is
> NO_HZ_FULL today - we could possibly even remove CONFIG_NO_HZ_FULL as
> an individual Kconfig option?
Ah, as a rename of nohz_full, sure that might work.
^ permalink raw reply
* Re: CONFIG_ISOLATION=y (was: [PATCH 0/6] support "dataplane" mode for nohz_full)
From: Ingo Molnar @ 2015-05-12 12:34 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Chris Metcalf, Steven Rostedt, Frederic Weisbecker, Andrew Morton,
paulmck, Gilad Ben Yossef, Rik van Riel, Tejun Heo,
Thomas Gleixner, Christoph Lameter, Srivatsa S. Bhat, linux-doc,
linux-api, linux-kernel
In-Reply-To: <20150512114809.GL21418@twins.programming.kicks-ass.net>
* Peter Zijlstra <peterz@infradead.org> wrote:
> On Tue, May 12, 2015 at 11:10:32AM +0200, Ingo Molnar wrote:
> >
> > So I'd vote for Frederic's CONFIG_ISOLATION=y, mostly because this
> > is a high level kernel feature, so it won't conflict with
> > isolation concepts in lower level subsystems such as IOMMU
> > isolation - and other higher level features like scheduler
> > isolation are basically another partial implementation we want to
> > merge with all this...
>
> But why do we need a CONFIG flag for something that has no content?
>
> That is, I do not see anything much; except the 'I want to stay in
> userspace and kill me otherwise' flag, and I'm not sure that
> warrants a CONFIG flag like this.
>
> Other than that, its all a combination of NOHZ_FULL and
> cpusets/isolcpus and whatnot.
Yes, that's what I meant: CONFIG_ISOLATION would trigger what is
NO_HZ_FULL today - we could possibly even remove CONFIG_NO_HZ_FULL as
an individual Kconfig option?
CONFIG_ISOLATION=y would express the guarantee from the kernel that
it's possible for user-space to configure itself to run undisturbed -
instead of the current inconsistent set of options and facilities.
A bit like CONFIG_PREEMPT_RT is more than just preemptable spinlocks,
it also tries to offer various facilities and tune the defaults to
turn the kernel hard-rt.
Does that make sense to you?
Thanks,
Ingo
^ permalink raw reply
* Re: [PATCH v2 1/3] pagemap: add mmap-exclusive bit for marking pages mapped only here
From: Mark Williamson @ 2015-05-12 12:05 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: linux-mm, Naoya Horiguchi, kernel list, Andrew Morton,
Pavel Emelyanov, Linux API, Andy Lutomirski, Vlastimil Babka,
Pavel Machek, Mark Seaborn, Kirill A. Shutemov, Linus Torvalds,
Daniel James, Finn Grimwood
In-Reply-To: <20150512094303.24768.10282.stgit@buzz>
Hi Konstantin,
I hope you won't mind me thinking out loud here on the idea of adding
a flag to the v2 pagemap fields... From a kernel PoV, I agree that
this seems like the cleanest approach. However, with my application
developer hat on:
1. I was hoping we'd be able to backport a compatible fix to older
kernels that might adopt the pagemap permissions change. Using the V2
format flags rules out doing this for kernels that are too old to have
soft-dirty, I think.
2. From our software's PoV, I feel it's worth noting that it doesn't
strictly fix ABI compatibility, though I realise that's probably not
your primary concern here. We'll need to modify our code to write the
clear_refs file but that change is OK for us if it's the preferred
solution.
In the patches I've been playing with, I was considering putting the
Exclusive flag in the now-unused PFN field of the pagemap entries.
Since we're specifically trying to work around for the lack of PFN
information, would there be any appetite for mirroring this flag
unconditionally into the now-empty PFN field (i.e. whether using v1 or
v2 flags) when accessed by an unprivileged process?
I realise it's ugly from a kernel PoV and I feel a little bad for
suggesting it - but it would address points 1 and 2 for us (our
existing code just looks for changes in the pagemap entry, so sticking
the flag in there would cause it to do the right thing).
I'm sorry to raise application-specific issues at this point; I
appreciate that your primary concern is to improve the kernel and
technically I like the approach that you've taken! I'll try and
provide more code-oriented feedback once I've tried out the changes.
Thanks,
Mark
On Tue, May 12, 2015 at 10:43 AM, Konstantin Khlebnikov
<khlebnikov@yandex-team.ru> wrote:
> This patch sets bit 56 in pagemap if this page is mapped only once.
> It allows to detect exclusively used pages without exposing PFN:
>
> present file exclusive state
> 0 0 0 non-present
> 1 1 0 file page mapped somewhere else
> 1 1 1 file page mapped only here
> 1 0 0 anon non-CoWed page (shared with parent/child)
> 1 0 1 anon CoWed page (or never forked)
>
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> Link: lkml.kernel.org/r/CAEVpBa+_RyACkhODZrRvQLs80iy0sqpdrd0AaP_-tgnX3Y9yNQ@mail.gmail.com
>
> ---
>
> v2:
> * handle transparent huge pages
> * invert bit and rename shared -> exclusive (less confusing name)
> ---
> Documentation/vm/pagemap.txt | 3 ++-
> fs/proc/task_mmu.c | 10 ++++++++++
> tools/vm/page-types.c | 12 ++++++++++++
> 3 files changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/vm/pagemap.txt b/Documentation/vm/pagemap.txt
> index 6bfbc172cdb9..3cfbbb333ea1 100644
> --- a/Documentation/vm/pagemap.txt
> +++ b/Documentation/vm/pagemap.txt
> @@ -16,7 +16,8 @@ There are three components to pagemap:
> * Bits 0-4 swap type if swapped
> * Bits 5-54 swap offset if swapped
> * Bit 55 pte is soft-dirty (see Documentation/vm/soft-dirty.txt)
> - * Bits 56-60 zero
> + * Bit 56 page exlusively mapped
> + * Bits 57-60 zero
> * Bit 61 page is file-page or shared-anon
> * Bit 62 page swapped
> * Bit 63 page present
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 6dee68d013ff..29febec65de4 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -982,6 +982,7 @@ struct pagemapread {
> #define PM_STATUS2(v2, x) (__PM_PSHIFT(v2 ? x : PAGE_SHIFT))
>
> #define __PM_SOFT_DIRTY (1LL)
> +#define __PM_MMAP_EXCLUSIVE (2LL)
> #define PM_PRESENT PM_STATUS(4LL)
> #define PM_SWAP PM_STATUS(2LL)
> #define PM_FILE PM_STATUS(1LL)
> @@ -1074,6 +1075,8 @@ static void pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
>
> if (page && !PageAnon(page))
> flags |= PM_FILE;
> + if (page && page_mapcount(page) == 1)
> + flags2 |= __PM_MMAP_EXCLUSIVE;
> if ((vma->vm_flags & VM_SOFTDIRTY))
> flags2 |= __PM_SOFT_DIRTY;
>
> @@ -1119,6 +1122,13 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
> else
> pmd_flags2 = 0;
>
> + if (pmd_present(*pmd)) {
> + struct page *page = pmd_page(*pmd);
> +
> + if (page_mapcount(page) == 1)
> + pmd_flags2 |= __PM_MMAP_EXCLUSIVE;
> + }
> +
> for (; addr != end; addr += PAGE_SIZE) {
> unsigned long offset;
> pagemap_entry_t pme;
> diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
> index 8bdf16b8ba60..3a9f193526ee 100644
> --- a/tools/vm/page-types.c
> +++ b/tools/vm/page-types.c
> @@ -70,9 +70,12 @@
> #define PM_PFRAME(x) ((x) & PM_PFRAME_MASK)
>
> #define __PM_SOFT_DIRTY (1LL)
> +#define __PM_MMAP_EXCLUSIVE (2LL)
> #define PM_PRESENT PM_STATUS(4LL)
> #define PM_SWAP PM_STATUS(2LL)
> +#define PM_FILE PM_STATUS(1LL)
> #define PM_SOFT_DIRTY __PM_PSHIFT(__PM_SOFT_DIRTY)
> +#define PM_MMAP_EXCLUSIVE __PM_PSHIFT(__PM_MMAP_EXCLUSIVE)
>
>
> /*
> @@ -100,6 +103,8 @@
> #define KPF_SLOB_FREE 49
> #define KPF_SLUB_FROZEN 50
> #define KPF_SLUB_DEBUG 51
> +#define KPF_FILE 62
> +#define KPF_MMAP_EXCLUSIVE 63
>
> #define KPF_ALL_BITS ((uint64_t)~0ULL)
> #define KPF_HACKERS_BITS (0xffffULL << 32)
> @@ -149,6 +154,9 @@ static const char * const page_flag_names[] = {
> [KPF_SLOB_FREE] = "P:slob_free",
> [KPF_SLUB_FROZEN] = "A:slub_frozen",
> [KPF_SLUB_DEBUG] = "E:slub_debug",
> +
> + [KPF_FILE] = "F:file",
> + [KPF_MMAP_EXCLUSIVE] = "1:mmap_exclusive",
> };
>
>
> @@ -452,6 +460,10 @@ static uint64_t expand_overloaded_flags(uint64_t flags, uint64_t pme)
>
> if (pme & PM_SOFT_DIRTY)
> flags |= BIT(SOFTDIRTY);
> + if (pme & PM_FILE)
> + flags |= BIT(FILE);
> + if (pme & PM_MMAP_EXCLUSIVE)
> + flags |= BIT(MMAP_EXCLUSIVE);
>
> return flags;
> }
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: CONFIG_ISOLATION=y (was: [PATCH 0/6] support "dataplane" mode for nohz_full)
From: Peter Zijlstra @ 2015-05-12 11:48 UTC (permalink / raw)
To: Ingo Molnar
Cc: Chris Metcalf, Steven Rostedt, Frederic Weisbecker, Andrew Morton,
paulmck-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8, Gilad Ben Yossef,
Rik van Riel, Tejun Heo, Thomas Gleixner, Christoph Lameter,
Srivatsa S. Bhat, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150512091032.GA10138-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
On Tue, May 12, 2015 at 11:10:32AM +0200, Ingo Molnar wrote:
>
> So I'd vote for Frederic's CONFIG_ISOLATION=y, mostly because this is
> a high level kernel feature, so it won't conflict with isolation
> concepts in lower level subsystems such as IOMMU isolation - and other
> higher level features like scheduler isolation are basically another
> partial implementation we want to merge with all this...
>
But why do we need a CONFIG flag for something that has no content?
That is, I do not see anything much; except the 'I want to stay in
userspace and kill me otherwise' flag, and I'm not sure that warrants a
CONFIG flag like this.
Other than that, its all a combination of NOHZ_FULL and cpusets/isolcpus
and whatnot.
^ permalink raw reply
* Re: [PATCH RFC] vfs: add a O_NOMTIME flag
From: Austin S Hemmelgarn @ 2015-05-12 11:45 UTC (permalink / raw)
To: Kevin Easton, Theodore Ts'o
Cc: Sage Weil, Trond Myklebust, Dave Chinner, Zach Brown,
Alexander Viro, Linux FS-devel Mailing List,
Linux Kernel Mailing List, Linux API Mailing List
In-Reply-To: <20150512050821.GA9404-Qr0l8DEfScZEV+tojptmR0B+6BGkLq7r@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 1985 bytes --]
On 2015-05-12 01:08, Kevin Easton wrote:
> On Mon, May 11, 2015 at 07:10:21PM -0400, Theodore Ts'o wrote:
>> On Mon, May 11, 2015 at 09:24:09AM -0700, Sage Weil wrote:
>>>> Let me re-ask the question that I asked last week (and was apparently
>>>> ignored). Why not trying to use the lazytime feature instead of
>>>> pointing a head straight at the application's --- and system
>>>> administrators' --- heads?
>>>
>>> Sorry Ted, I thought I responded already.
>>>
>>> The goal is to avoid inode writeout entirely when we can, and
>>> as I understand it lazytime will still force writeout before the inode
>>> is dropped from the cache. In systems like Ceph in particular, the
>>> IOs can be spread across lots of files, so simply deferring writeout
>>> doesn't always help.
>>
>> Sure, but it would reduce the writeout by orders of magnitude. I can
>> understand if you want to reduce it further, but it might be good
>> enough for your purposes.
>>
>> I considered doing the equivalent of O_NOMTIME for our purposes at
>> $WORK, and our use case is actually not that different from Ceph's
>> (i.e., using a local disk file system to support a cluster file
>> system), and lazytime was (a) something I figured was something I
>> could upstream in good conscience, and (b) was more than good enough
>> for us.
>
> A safer alternative might be a chattr file attribute that if set, the
> mtime is not updated on writes, and stat() on the file always shows the
> mtime as "right now". At least that way, the file won't accidentally
> get left out of backups that rely on the mtime.
>
> (If the file attribute is unset, you immediately update the mtime then
> too, and from then on the file is back to normal).
>
I like this even better than the flag suggestion, it provides better
control, means that you don't need to update applications to get the
benefits, and prevents backup software from breaking (although backups
would be bigger).
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]
^ permalink raw reply
* Re: [PATCH v2 2/3] pagemap: hide physical addresses from non-privileged users
From: Mark Williamson @ 2015-05-12 11:22 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Naoya Horiguchi, kernel list,
Andrew Morton, Pavel Emelyanov, Linux API, Andy Lutomirski,
Vlastimil Babka, Pavel Machek, Mark Seaborn, Kirill A. Shutemov,
Linus Torvalds, Daniel James, Finn Grimwood
In-Reply-To: <20150512094305.24768.51807.stgit@buzz>
Hi Konstantin,
Comments inline...
On Tue, May 12, 2015 at 10:43 AM, Konstantin Khlebnikov
<khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org> wrote:
> This patch makes pagemap readable for normal users back but hides physical
> addresses from them. For some use cases PFN isn't required at all: flags
> give information about presence, page type (anon/file/swap), soft-dirty mark,
> and hint about page mapcount state: exclusive(mapcount = 1) or (mapcount > 1).
>
> Signed-off-by: Konstantin Khlebnikov <khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org>
> Fixes: ab676b7d6fbf ("pagemap: do not leak physical addresses to non-privileged userspace")
> Link: lkml.kernel.org/r/1425935472-17949-1-git-send-email-kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org
> ---
> fs/proc/task_mmu.c | 36 ++++++++++++++++++++++--------------
> 1 file changed, 22 insertions(+), 14 deletions(-)
>
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 29febec65de4..0b7a8ffec95f 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -962,6 +962,7 @@ struct pagemapread {
> int pos, len; /* units: PM_ENTRY_BYTES, not bytes */
> pagemap_entry_t *buffer;
> bool v2;
> + bool show_pfn;
> };
>
> #define PAGEMAP_WALK_SIZE (PMD_SIZE)
> @@ -1046,12 +1047,13 @@ out:
> static void pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
> struct vm_area_struct *vma, unsigned long addr, pte_t pte)
> {
> - u64 frame, flags;
> + u64 frame = 0, flags;
> struct page *page = NULL;
> int flags2 = 0;
>
> if (pte_present(pte)) {
> - frame = pte_pfn(pte);
> + if (pm->show_pfn)
> + frame = pte_pfn(pte);
> flags = PM_PRESENT;
> page = vm_normal_page(vma, addr, pte);
> if (pte_soft_dirty(pte))
> @@ -1087,15 +1089,19 @@ static void pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
> static void thp_pmd_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
> pmd_t pmd, int offset, int pmd_flags2)
> {
> + u64 frame = 0;
> +
> /*
> * Currently pmd for thp is always present because thp can not be
> * swapped-out, migrated, or HWPOISONed (split in such cases instead.)
> * This if-check is just to prepare for future implementation.
> */
> - if (pmd_present(pmd))
> - *pme = make_pme(PM_PFRAME(pmd_pfn(pmd) + offset)
> - | PM_STATUS2(pm->v2, pmd_flags2) | PM_PRESENT);
> - else
> + if (pmd_present(pmd)) {
> + if (pm->show_pfn)
> + frame = pmd_pfn(pmd) + offset;
> + *pme = make_pme(PM_PFRAME(frame) | PM_PRESENT |
> + PM_STATUS2(pm->v2, pmd_flags2));
> + } else
> *pme = make_pme(PM_NOT_PRESENT(pm->v2) | PM_STATUS2(pm->v2, pmd_flags2));
> }
> #else
> @@ -1171,11 +1177,14 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
> static void huge_pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
> pte_t pte, int offset, int flags2)
> {
> - if (pte_present(pte))
> - *pme = make_pme(PM_PFRAME(pte_pfn(pte) + offset) |
> - PM_STATUS2(pm->v2, flags2) |
> - PM_PRESENT);
> - else
> + u64 frame = 0;
> +
> + if (pte_present(pte)) {
> + if (pm->show_pfn)
> + frame = pte_pfn(pte) + offset;
> + *pme = make_pme(PM_PFRAME(frame) | PM_PRESENT |
> + PM_STATUS2(pm->v2, flags2));
> + } else
> *pme = make_pme(PM_NOT_PRESENT(pm->v2) |
> PM_STATUS2(pm->v2, flags2));
> }
> @@ -1260,6 +1269,8 @@ static ssize_t pagemap_read(struct file *file, char __user *buf,
> if (!count)
> goto out_task;
>
> + /* do not disclose physical addresses: attack vector */
> + pm.show_pfn = capable(CAP_SYS_ADMIN);
If I understood correctly, Linus recommended to me that we use the
open-time capabilities of the file descriptor rather than the current
capability state (to mitigate against an attacker passing an FD to a
setuid process, I think).
FWIW, I knocked up a quick internal patch (less comprehensive than
yours!) and used file_ns_capable() successfully, i.e:
pm.show_pfn = file_ns_capable(file, &init_user_ns, CAP_SYS_ADMIN);
It looked promising to be but I've not done the checking to verify
that this is strictly correct; the capabilities stuff is not an area
of the kernel I'm familiar with.
> pm.v2 = soft_dirty_cleared;
> pm.len = (PAGEMAP_WALK_SIZE >> PAGE_SHIFT);
> pm.buffer = kmalloc(pm.len * PM_ENTRY_BYTES, GFP_TEMPORARY);
> @@ -1335,9 +1346,6 @@ out:
>
> static int pagemap_open(struct inode *inode, struct file *file)
> {
> - /* do not disclose physical addresses: attack vector */
> - if (!capable(CAP_SYS_ADMIN))
> - return -EPERM;
> pr_warn_once("Bits 55-60 of /proc/PID/pagemap entries are about "
> "to stop being page-shift some time soon. See the "
> "linux/Documentation/vm/pagemap.txt for details.\n");
>
No other comments on this, looks like it would help us.
Thanks,
Mark
^ permalink raw reply
* Re: [PATCH RFC 0/3] pagemap: make useable for non-privilege users
From: Mark Williamson @ 2015-05-12 11:13 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: linux-mm, Naoya Horiguchi, kernel list, Andrew Morton,
Pavel Emelyanov, Linux API, Andy Lutomirski, Vlastimil Babka,
Pavel Machek, Mark Seaborn, Kirill A. Shutemov, Linus Torvalds,
Daniel James, Finn Grimwood
In-Reply-To: <20150512090156.24768.2521.stgit@buzz>
Hi Konstantin,
Thanks very much for continuing to look at this! It's very much
appreciated. I've been investigating from our end but got caught up
in some gnarly details of our pagemap-consuming code.
I like the approach and it seems like the information you're exposing
will be useful for our application. I'll test the patch and see if it
works for us as-is.
Will follow up with any comments on the individual patches.
Thanks,
Mark
On Tue, May 12, 2015 at 10:43 AM, Konstantin Khlebnikov
<khlebnikov@yandex-team.ru> wrote:
> This patchset tries to make pagemap useable again in the safe way.
> First patch adds bit 'map-exlusive' which is set if page is mapped only here.
> Second patch restores access for non-privileged users but hides pfn if task
> has no capability CAP_SYS_ADMIN. Third patch removes page-shift bits and
> completes migration to the new pagemap format (flags soft-dirty and
> mmap-exlusive are available only in the new format).
>
> ---
>
> Konstantin Khlebnikov (3):
> pagemap: add mmap-exclusive bit for marking pages mapped only here
> pagemap: hide physical addresses from non-privileged users
> pagemap: switch to the new format and do some cleanup
>
>
> Documentation/vm/pagemap.txt | 3 -
> fs/proc/task_mmu.c | 178 +++++++++++++++++-------------------------
> tools/vm/page-types.c | 35 ++++----
> 3 files changed, 91 insertions(+), 125 deletions(-)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 3/3] pagemap: switch to the new format and do some cleanup
From: Kirill A. Shutemov @ 2015-05-12 10:54 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: linux-mm, Naoya Horiguchi, linux-kernel, Andrew Morton,
Mark Williamson, Pavel Emelyanov, linux-api, Andy Lutomirski,
Vlastimil Babka, Pavel Machek, Mark Seaborn, Linus Torvalds,
Daniel James, Finn Grimwood
In-Reply-To: <20150512094306.24768.51325.stgit@buzz>
On Tue, May 12, 2015 at 12:43:06PM +0300, Konstantin Khlebnikov wrote:
> This patch removes page-shift bits (scheduled to remove since 3.11) and
> completes migration to the new bit layout. Also it cleans messy macro.
>
> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> ---
> fs/proc/task_mmu.c | 152 ++++++++++++++++---------------------------------
> tools/vm/page-types.c | 29 +++------
> 2 files changed, 58 insertions(+), 123 deletions(-)
>
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 0b7a8ffec95f..66bc7207ce90 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -710,23 +710,6 @@ const struct file_operations proc_tid_smaps_operations = {
> .release = proc_map_release,
> };
>
> -/*
> - * We do not want to have constant page-shift bits sitting in
> - * pagemap entries and are about to reuse them some time soon.
> - *
> - * Here's the "migration strategy":
> - * 1. when the system boots these bits remain what they are,
> - * but a warning about future change is printed in log;
> - * 2. once anyone clears soft-dirty bits via clear_refs file,
> - * these flag is set to denote, that user is aware of the
> - * new API and those page-shift bits change their meaning.
> - * The respective warning is printed in dmesg;
> - * 3. In a couple of releases we will remove all the mentions
> - * of page-shift in pagemap entries.
> - */
Wouldn't it be better to just have v2=1 by default for couple releases to
see if anything breaks? This way we can revert easily if regression reported.
I guess someone could miss this change coming if he didn't touch clear_refs.
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v6 0/8] vhost: support for cross endian guests
From: Michael S. Tsirkin @ 2015-05-12 10:52 UTC (permalink / raw)
To: Greg Kurz
Cc: Thomas Huth, kvm-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Cornelia Huck
In-Reply-To: <20150512124426.4e487cb6-GiB8zCg7hOfDOqzlkpFKJg@public.gmane.org>
On Tue, May 12, 2015 at 12:44:26PM +0200, Greg Kurz wrote:
> On Fri, 24 Apr 2015 15:31:54 +0200
> "Michael S. Tsirkin" <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>
> > On Fri, Apr 24, 2015 at 02:24:15PM +0200, Greg Kurz wrote:
> > > Only cosmetic and documentation changes since v5.
> > >
> > > ---
> >
> > Looks sane to me. I plan to review and apply next week.
> >
>
> Hi Michael,
>
> I realize you just got back and have tons of things to do... Do
> you still plan to apply shortly ? Would you also have time to
> comment the QEMU part ?
>
> Thanks.
>
> --
> Greg
Yes, sorry about the delay - I also got virtio upstream landed on my lap.
I'll do my best to prioritize.
> > > Greg Kurz (8):
> > > virtio: introduce virtio_is_little_endian() helper
> > > tun: add tun_is_little_endian() helper
> > > macvtap: introduce macvtap_is_little_endian() helper
> > > vringh: introduce vringh_is_little_endian() helper
> > > vhost: introduce vhost_is_little_endian() helper
> > > virtio: add explicit big-endian support to memory accessors
> > > vhost: cross-endian support for legacy devices
> > > macvtap/tun: cross-endian support for little-endian hosts
> > >
> > >
> > > drivers/net/Kconfig | 14 ++++++
> > > drivers/net/macvtap.c | 66 +++++++++++++++++++++++++++++-
> > > drivers/net/tun.c | 68 ++++++++++++++++++++++++++++++
> > > drivers/vhost/Kconfig | 15 +++++++
> > > drivers/vhost/vhost.c | 85 ++++++++++++++++++++++++++++++++++++++
> > > drivers/vhost/vhost.h | 25 ++++++++---
> > > include/linux/virtio_byteorder.h | 24 ++++++-----
> > > include/linux/virtio_config.h | 18 +++++---
> > > include/linux/vringh.h | 18 +++++---
> > > include/uapi/linux/if_tun.h | 6 +++
> > > include/uapi/linux/vhost.h | 14 ++++++
> > > 11 files changed, 320 insertions(+), 33 deletions(-)
> > >
> > > --
> > > Greg
> >
^ permalink raw reply
* Re: [PATCH 0/6] support "dataplane" mode for nohz_full
From: Peter Zijlstra @ 2015-05-12 10:46 UTC (permalink / raw)
To: Steven Rostedt
Cc: Ingo Molnar, Andrew Morton, Chris Metcalf, Gilad Ben Yossef,
Ingo Molnar, Rik van Riel, Tejun Heo, Frederic Weisbecker,
Thomas Gleixner, Paul E. McKenney, Christoph Lameter,
Srivatsa S. Bhat, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150511085759.71deeb64-f9ZlEuEWxVcJvu8Pb33WZ0EMvNT87kid@public.gmane.org>
On Mon, May 11, 2015 at 08:57:59AM -0400, Steven Rostedt wrote:
>
> Please lets get NO_HZ_FULL up to par. That should be the main focus.
>
ACK, much of this dataplane stuff is (useful) hacks working around the
fact that nohz_full just isn't complete.
^ permalink raw reply
* Re: [PATCH v6 0/8] vhost: support for cross endian guests
From: Greg Kurz @ 2015-05-12 10:44 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Thomas Huth, kvm-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Cornelia Huck
In-Reply-To: <20150424153137-mutt-send-email-mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Fri, 24 Apr 2015 15:31:54 +0200
"Michael S. Tsirkin" <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Fri, Apr 24, 2015 at 02:24:15PM +0200, Greg Kurz wrote:
> > Only cosmetic and documentation changes since v5.
> >
> > ---
>
> Looks sane to me. I plan to review and apply next week.
>
Hi Michael,
I realize you just got back and have tons of things to do... Do
you still plan to apply shortly ? Would you also have time to
comment the QEMU part ?
Thanks.
--
Greg
> > Greg Kurz (8):
> > virtio: introduce virtio_is_little_endian() helper
> > tun: add tun_is_little_endian() helper
> > macvtap: introduce macvtap_is_little_endian() helper
> > vringh: introduce vringh_is_little_endian() helper
> > vhost: introduce vhost_is_little_endian() helper
> > virtio: add explicit big-endian support to memory accessors
> > vhost: cross-endian support for legacy devices
> > macvtap/tun: cross-endian support for little-endian hosts
> >
> >
> > drivers/net/Kconfig | 14 ++++++
> > drivers/net/macvtap.c | 66 +++++++++++++++++++++++++++++-
> > drivers/net/tun.c | 68 ++++++++++++++++++++++++++++++
> > drivers/vhost/Kconfig | 15 +++++++
> > drivers/vhost/vhost.c | 85 ++++++++++++++++++++++++++++++++++++++
> > drivers/vhost/vhost.h | 25 ++++++++---
> > include/linux/virtio_byteorder.h | 24 ++++++-----
> > include/linux/virtio_config.h | 18 +++++---
> > include/linux/vringh.h | 18 +++++---
> > include/uapi/linux/if_tun.h | 6 +++
> > include/uapi/linux/vhost.h | 14 ++++++
> > 11 files changed, 320 insertions(+), 33 deletions(-)
> >
> > --
> > Greg
>
^ permalink raw reply
* Re: [PATCH v2 1/3] pagemap: add mmap-exclusive bit for marking pages mapped only here
From: Kirill A. Shutemov @ 2015-05-12 10:40 UTC (permalink / raw)
To: Konstantin Khlebnikov
Cc: linux-mm, Naoya Horiguchi, linux-kernel, Andrew Morton,
Mark Williamson, Pavel Emelyanov, linux-api, Andy Lutomirski,
Vlastimil Babka, Pavel Machek, Mark Seaborn, Linus Torvalds,
Daniel James, Finn Grimwood
In-Reply-To: <20150512094303.24768.10282.stgit@buzz>
On Tue, May 12, 2015 at 12:43:03PM +0300, Konstantin Khlebnikov wrote:
> This patch sets bit 56 in pagemap if this page is mapped only once.
> It allows to detect exclusively used pages without exposing PFN:
>
> present file exclusive state
> 0 0 0 non-present
> 1 1 0 file page mapped somewhere else
> 1 1 1 file page mapped only here
> 1 0 0 anon non-CoWed page (shared with parent/child)
> 1 0 1 anon CoWed page (or never forked)
Probably, worth noting that file-private pages are anon in this context.
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 4/6] nohz: support PR_DATAPLANE_QUIESCE
From: Peter Zijlstra @ 2015-05-12 10:38 UTC (permalink / raw)
To: Ingo Molnar
Cc: Chris Metcalf, Gilad Ben Yossef, Steven Rostedt, Ingo Molnar,
Andrew Morton, Rik van Riel, Tejun Heo, Thomas Gleixner,
Frederic Weisbecker, Paul E. McKenney, Christoph Lameter,
Srivatsa S. Bhat, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150512095030.GD11477-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
On Tue, May 12, 2015 at 11:50:30AM +0200, Ingo Molnar wrote:
>
> * Peter Zijlstra <peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> wrote:
>
> > On Fri, May 08, 2015 at 01:58:45PM -0400, Chris Metcalf wrote:
> > > This prctl() flag for PR_SET_DATAPLANE sets a mode that requires the
> > > kernel to quiesce any pending timer interrupts prior to returning
> > > to userspace. When running with this mode set, sys calls (and page
> > > faults, etc.) can be inordinately slow. However, user applications
> > > that want to guarantee that no unexpected interrupts will occur
> > > (even if they call into the kernel) can set this flag to guarantee
> > > that semantics.
> >
> > Currently people hot-unplug and hot-plug the CPU to do this.
> > Obviously that's a wee bit horrible :-)
> >
> > Not sure if a prctl like this is any better though. This is a CPU
> > properly not a process one.
>
> So if then a prctl() (or other system call) could be a shortcut to:
>
> - move the task to an isolated CPU
> - make sure there _is_ such an isolated domain available
>
> I.e. have some programmatic, kernel provided way for an application to
> be sure it's running in the right environment. Relying on random
> administration flags here and there won't cut it.
No, we already have sched_setaffinity() and we should not duplicate its
ability to move tasks about.
What this is about is 'clearing' CPU state, its nothing to do with
tasks.
Ideally we'd never have to clear the state because it should be
impossible to get into this predicament in the first place.
The typical example here is a periodic timer that found its way onto the
cpu and stays there. We're actually working on allowing such self arming
timers to migrate, so once we have that sorted this could be fixed
proper I think.
Not sure if there's more pollution that people worry about.
The hotplug hack worked because unplug force migrates the timers away.
^ permalink raw reply
* Re: [PATCH 1/5] selftests: Add futex functional tests
From: Cyril Hrubis @ 2015-05-12 10:02 UTC (permalink / raw)
To: Darren Hart
Cc: Shuah Khan, linux-api-u79uwXL29TY76Z2rM5mHXA,
Linux Kernel Mailing List, John Stultz, Ingo Molnar,
Peter Zijlstra, Thomas Gleixner, Davidlohr Bueso, KOSAKI Motohiro
In-Reply-To: <D1767EA1.CC3DA%dvhart-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Hi!
> I'm happy to do that, but I would like to make sure I'm doing the right
> thing.
The right thing here is to add -pthread to CFLAGS which sets both flags
for preprocessor and linker (see man gcc).
--
Cyril Hrubis
chrubis-AlSwsSmVLrQ@public.gmane.org
^ permalink raw reply
* Re: [PATCH 4/6] nohz: support PR_DATAPLANE_QUIESCE
From: Ingo Molnar @ 2015-05-12 9:50 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Chris Metcalf, Gilad Ben Yossef, Steven Rostedt, Ingo Molnar,
Andrew Morton, Rik van Riel, Tejun Heo, Thomas Gleixner,
Frederic Weisbecker, Paul E. McKenney, Christoph Lameter,
Srivatsa S. Bhat, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150512093349.GH21418-ndre7Fmf5hadTX5a5knrm8zTDFooKrT+cvkQGrU6aU0@public.gmane.org>
* Peter Zijlstra <peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> wrote:
> On Fri, May 08, 2015 at 01:58:45PM -0400, Chris Metcalf wrote:
> > This prctl() flag for PR_SET_DATAPLANE sets a mode that requires the
> > kernel to quiesce any pending timer interrupts prior to returning
> > to userspace. When running with this mode set, sys calls (and page
> > faults, etc.) can be inordinately slow. However, user applications
> > that want to guarantee that no unexpected interrupts will occur
> > (even if they call into the kernel) can set this flag to guarantee
> > that semantics.
>
> Currently people hot-unplug and hot-plug the CPU to do this.
> Obviously that's a wee bit horrible :-)
>
> Not sure if a prctl like this is any better though. This is a CPU
> properly not a process one.
So if then a prctl() (or other system call) could be a shortcut to:
- move the task to an isolated CPU
- make sure there _is_ such an isolated domain available
I.e. have some programmatic, kernel provided way for an application to
be sure it's running in the right environment. Relying on random
administration flags here and there won't cut it.
Thanks,
Ingo
^ permalink raw reply
* [PATCH v2 3/3] pagemap: switch to the new format and do some cleanup
From: Konstantin Khlebnikov @ 2015-05-12 9:43 UTC (permalink / raw)
To: linux-mm, Naoya Horiguchi, linux-kernel, Andrew Morton
Cc: Mark Williamson, Pavel Emelyanov, linux-api, Andy Lutomirski,
Vlastimil Babka, Pavel Machek, Mark Seaborn, Kirill A. Shutemov,
Linus Torvalds, Daniel James, Finn Grimwood
In-Reply-To: <20150512090156.24768.2521.stgit@buzz>
This patch removes page-shift bits (scheduled to remove since 3.11) and
completes migration to the new bit layout. Also it cleans messy macro.
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
fs/proc/task_mmu.c | 152 ++++++++++++++++---------------------------------
tools/vm/page-types.c | 29 +++------
2 files changed, 58 insertions(+), 123 deletions(-)
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 0b7a8ffec95f..66bc7207ce90 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -710,23 +710,6 @@ const struct file_operations proc_tid_smaps_operations = {
.release = proc_map_release,
};
-/*
- * We do not want to have constant page-shift bits sitting in
- * pagemap entries and are about to reuse them some time soon.
- *
- * Here's the "migration strategy":
- * 1. when the system boots these bits remain what they are,
- * but a warning about future change is printed in log;
- * 2. once anyone clears soft-dirty bits via clear_refs file,
- * these flag is set to denote, that user is aware of the
- * new API and those page-shift bits change their meaning.
- * The respective warning is printed in dmesg;
- * 3. In a couple of releases we will remove all the mentions
- * of page-shift in pagemap entries.
- */
-
-static bool soft_dirty_cleared __read_mostly;
-
enum clear_refs_types {
CLEAR_REFS_ALL = 1,
CLEAR_REFS_ANON,
@@ -887,13 +870,6 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf,
if (type < CLEAR_REFS_ALL || type >= CLEAR_REFS_LAST)
return -EINVAL;
- if (type == CLEAR_REFS_SOFT_DIRTY) {
- soft_dirty_cleared = true;
- pr_warn_once("The pagemap bits 55-60 has changed their meaning!"
- " See the linux/Documentation/vm/pagemap.txt for "
- "details.\n");
- }
-
task = get_proc_task(file_inode(file));
if (!task)
return -ESRCH;
@@ -961,38 +937,26 @@ typedef struct {
struct pagemapread {
int pos, len; /* units: PM_ENTRY_BYTES, not bytes */
pagemap_entry_t *buffer;
- bool v2;
bool show_pfn;
};
#define PAGEMAP_WALK_SIZE (PMD_SIZE)
#define PAGEMAP_WALK_MASK (PMD_MASK)
-#define PM_ENTRY_BYTES sizeof(pagemap_entry_t)
-#define PM_STATUS_BITS 3
-#define PM_STATUS_OFFSET (64 - PM_STATUS_BITS)
-#define PM_STATUS_MASK (((1LL << PM_STATUS_BITS) - 1) << PM_STATUS_OFFSET)
-#define PM_STATUS(nr) (((nr) << PM_STATUS_OFFSET) & PM_STATUS_MASK)
-#define PM_PSHIFT_BITS 6
-#define PM_PSHIFT_OFFSET (PM_STATUS_OFFSET - PM_PSHIFT_BITS)
-#define PM_PSHIFT_MASK (((1LL << PM_PSHIFT_BITS) - 1) << PM_PSHIFT_OFFSET)
-#define __PM_PSHIFT(x) (((u64) (x) << PM_PSHIFT_OFFSET) & PM_PSHIFT_MASK)
-#define PM_PFRAME_MASK ((1LL << PM_PSHIFT_OFFSET) - 1)
-#define PM_PFRAME(x) ((x) & PM_PFRAME_MASK)
-/* in "new" pagemap pshift bits are occupied with more status bits */
-#define PM_STATUS2(v2, x) (__PM_PSHIFT(v2 ? x : PAGE_SHIFT))
-
-#define __PM_SOFT_DIRTY (1LL)
-#define __PM_MMAP_EXCLUSIVE (2LL)
-#define PM_PRESENT PM_STATUS(4LL)
-#define PM_SWAP PM_STATUS(2LL)
-#define PM_FILE PM_STATUS(1LL)
-#define PM_NOT_PRESENT(v2) PM_STATUS2(v2, 0)
+#define PM_ENTRY_BYTES sizeof(pagemap_entry_t)
+#define PM_PFEAME_BITS 54
+#define PM_PFRAME_MASK GENMASK_ULL(PM_PFEAME_BITS - 1, 0)
+#define PM_SOFT_DIRTY BIT_ULL(55)
+#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
+#define PM_FILE BIT_ULL(61)
+#define PM_SWAP BIT_ULL(62)
+#define PM_PRESENT BIT_ULL(63)
+
#define PM_END_OF_BUFFER 1
-static inline pagemap_entry_t make_pme(u64 val)
+static inline pagemap_entry_t make_pme(u64 frame, u64 flags)
{
- return (pagemap_entry_t) { .pme = val };
+ return (pagemap_entry_t) { .pme = (frame & PM_PFRAME_MASK) | flags };
}
static int add_to_pagemap(unsigned long addr, pagemap_entry_t *pme,
@@ -1013,7 +977,7 @@ static int pagemap_pte_hole(unsigned long start, unsigned long end,
while (addr < end) {
struct vm_area_struct *vma = find_vma(walk->mm, addr);
- pagemap_entry_t pme = make_pme(PM_NOT_PRESENT(pm->v2));
+ pagemap_entry_t pme = make_pme(0, 0);
/* End of address space hole, which we mark as non-present. */
unsigned long hole_end;
@@ -1033,7 +997,7 @@ static int pagemap_pte_hole(unsigned long start, unsigned long end,
/* Addresses in the VMA. */
if (vma->vm_flags & VM_SOFTDIRTY)
- pme.pme |= PM_STATUS2(pm->v2, __PM_SOFT_DIRTY);
+ pme = make_pme(0, PM_SOFT_DIRTY);
for (; addr < min(end, vma->vm_end); addr += PAGE_SIZE) {
err = add_to_pagemap(addr, &pme, pm);
if (err)
@@ -1044,50 +1008,44 @@ out:
return err;
}
-static void pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
+static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm,
struct vm_area_struct *vma, unsigned long addr, pte_t pte)
{
- u64 frame = 0, flags;
+ u64 frame = 0, flags = 0;
struct page *page = NULL;
- int flags2 = 0;
if (pte_present(pte)) {
if (pm->show_pfn)
frame = pte_pfn(pte);
- flags = PM_PRESENT;
+ flags |= PM_PRESENT;
page = vm_normal_page(vma, addr, pte);
if (pte_soft_dirty(pte))
- flags2 |= __PM_SOFT_DIRTY;
+ flags |= PM_SOFT_DIRTY;
} else if (is_swap_pte(pte)) {
swp_entry_t entry;
if (pte_swp_soft_dirty(pte))
- flags2 |= __PM_SOFT_DIRTY;
+ flags |= PM_SOFT_DIRTY;
entry = pte_to_swp_entry(pte);
frame = swp_type(entry) |
(swp_offset(entry) << MAX_SWAPFILES_SHIFT);
- flags = PM_SWAP;
+ flags |= PM_SWAP;
if (is_migration_entry(entry))
page = migration_entry_to_page(entry);
- } else {
- if (vma->vm_flags & VM_SOFTDIRTY)
- flags2 |= __PM_SOFT_DIRTY;
- *pme = make_pme(PM_NOT_PRESENT(pm->v2) | PM_STATUS2(pm->v2, flags2));
- return;
}
if (page && !PageAnon(page))
flags |= PM_FILE;
if (page && page_mapcount(page) == 1)
- flags2 |= __PM_MMAP_EXCLUSIVE;
- if ((vma->vm_flags & VM_SOFTDIRTY))
- flags2 |= __PM_SOFT_DIRTY;
+ flags |= PM_MMAP_EXCLUSIVE;
+ if (vma->vm_flags & VM_SOFTDIRTY)
+ flags |= PM_SOFT_DIRTY;
- *pme = make_pme(PM_PFRAME(frame) | PM_STATUS2(pm->v2, flags2) | flags);
+ return make_pme(frame, flags);
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-static void thp_pmd_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
- pmd_t pmd, int offset, int pmd_flags2)
+static pagemap_entry_t thp_pmd_to_pagemap_entry(struct pagemapread *pm,
+ pmd_t pmd, int offset, u64 flags)
{
u64 frame = 0;
@@ -1099,15 +1057,16 @@ static void thp_pmd_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *p
if (pmd_present(pmd)) {
if (pm->show_pfn)
frame = pmd_pfn(pmd) + offset;
- *pme = make_pme(PM_PFRAME(frame) | PM_PRESENT |
- PM_STATUS2(pm->v2, pmd_flags2));
- } else
- *pme = make_pme(PM_NOT_PRESENT(pm->v2) | PM_STATUS2(pm->v2, pmd_flags2));
+ flags |= PM_PRESENT;
+ }
+
+ return make_pme(frame, flags);
}
#else
-static inline void thp_pmd_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
- pmd_t pmd, int offset, int pmd_flags2)
+static pagemap_entry_t thp_pmd_to_pagemap_entry(struct pagemapread *pm,
+ pmd_t pmd, int offset, u64 flags)
{
+ return make_pme(0, 0);
}
#endif
@@ -1121,18 +1080,16 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
int err = 0;
if (pmd_trans_huge_lock(pmd, vma, &ptl) == 1) {
- int pmd_flags2;
+ u64 flags = 0;
if ((vma->vm_flags & VM_SOFTDIRTY) || pmd_soft_dirty(*pmd))
- pmd_flags2 = __PM_SOFT_DIRTY;
- else
- pmd_flags2 = 0;
+ flags |= PM_SOFT_DIRTY;
if (pmd_present(*pmd)) {
struct page *page = pmd_page(*pmd);
if (page_mapcount(page) == 1)
- pmd_flags2 |= __PM_MMAP_EXCLUSIVE;
+ flags |= PM_MMAP_EXCLUSIVE;
}
for (; addr != end; addr += PAGE_SIZE) {
@@ -1141,7 +1098,7 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
offset = (addr & ~PAGEMAP_WALK_MASK) >>
PAGE_SHIFT;
- thp_pmd_to_pagemap_entry(&pme, pm, *pmd, offset, pmd_flags2);
+ pme = thp_pmd_to_pagemap_entry(pm, *pmd, offset, flags);
err = add_to_pagemap(addr, &pme, pm);
if (err)
break;
@@ -1161,7 +1118,7 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
for (; addr < end; pte++, addr += PAGE_SIZE) {
pagemap_entry_t pme;
- pte_to_pagemap_entry(&pme, pm, vma, addr, *pte);
+ pme = pte_to_pagemap_entry(pm, vma, addr, *pte);
err = add_to_pagemap(addr, &pme, pm);
if (err)
break;
@@ -1174,19 +1131,18 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
}
#ifdef CONFIG_HUGETLB_PAGE
-static void huge_pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
- pte_t pte, int offset, int flags2)
+static pagemap_entry_t huge_pte_to_pagemap_entry(struct pagemapread *pm,
+ pte_t pte, int offset, u64 flags)
{
u64 frame = 0;
if (pte_present(pte)) {
if (pm->show_pfn)
frame = pte_pfn(pte) + offset;
- *pme = make_pme(PM_PFRAME(frame) | PM_PRESENT |
- PM_STATUS2(pm->v2, flags2));
- } else
- *pme = make_pme(PM_NOT_PRESENT(pm->v2) |
- PM_STATUS2(pm->v2, flags2));
+ flags |= PM_PRESENT;
+ }
+
+ return make_pme(frame, flags);
}
/* This function walks within one hugetlb entry in the single call */
@@ -1197,17 +1153,15 @@ static int pagemap_hugetlb_range(pte_t *pte, unsigned long hmask,
struct pagemapread *pm = walk->private;
struct vm_area_struct *vma = walk->vma;
int err = 0;
- int flags2;
+ u64 flags = 0;
pagemap_entry_t pme;
if (vma->vm_flags & VM_SOFTDIRTY)
- flags2 = __PM_SOFT_DIRTY;
- else
- flags2 = 0;
+ flags |= PM_SOFT_DIRTY;
for (; addr != end; addr += PAGE_SIZE) {
int offset = (addr & ~hmask) >> PAGE_SHIFT;
- huge_pte_to_pagemap_entry(&pme, pm, *pte, offset, flags2);
+ pme = huge_pte_to_pagemap_entry(pm, *pte, offset, flags);
err = add_to_pagemap(addr, &pme, pm);
if (err)
return err;
@@ -1228,7 +1182,9 @@ static int pagemap_hugetlb_range(pte_t *pte, unsigned long hmask,
* Bits 0-54 page frame number (PFN) if present
* Bits 0-4 swap type if swapped
* Bits 5-54 swap offset if swapped
- * Bits 55-60 page shift (page size = 1<<page shift)
+ * Bit 55 pte is soft-dirty (see Documentation/vm/soft-dirty.txt)
+ * Bit 56 page exclusively mapped
+ * Bits 57-60 zero
* Bit 61 page is file-page or shared-anon
* Bit 62 page swapped
* Bit 63 page present
@@ -1271,7 +1227,6 @@ static ssize_t pagemap_read(struct file *file, char __user *buf,
/* do not disclose physical addresses: attack vector */
pm.show_pfn = capable(CAP_SYS_ADMIN);
- pm.v2 = soft_dirty_cleared;
pm.len = (PAGEMAP_WALK_SIZE >> PAGE_SHIFT);
pm.buffer = kmalloc(pm.len * PM_ENTRY_BYTES, GFP_TEMPORARY);
ret = -ENOMEM;
@@ -1344,18 +1299,9 @@ out:
return ret;
}
-static int pagemap_open(struct inode *inode, struct file *file)
-{
- pr_warn_once("Bits 55-60 of /proc/PID/pagemap entries are about "
- "to stop being page-shift some time soon. See the "
- "linux/Documentation/vm/pagemap.txt for details.\n");
- return 0;
-}
-
const struct file_operations proc_pagemap_operations = {
.llseek = mem_lseek, /* borrow this */
.read = pagemap_read,
- .open = pagemap_open,
};
#endif /* CONFIG_PROC_PAGE_MONITOR */
diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
index 3a9f193526ee..1fa872e58238 100644
--- a/tools/vm/page-types.c
+++ b/tools/vm/page-types.c
@@ -57,26 +57,15 @@
* pagemap kernel ABI bits
*/
-#define PM_ENTRY_BYTES sizeof(uint64_t)
-#define PM_STATUS_BITS 3
-#define PM_STATUS_OFFSET (64 - PM_STATUS_BITS)
-#define PM_STATUS_MASK (((1LL << PM_STATUS_BITS) - 1) << PM_STATUS_OFFSET)
-#define PM_STATUS(nr) (((nr) << PM_STATUS_OFFSET) & PM_STATUS_MASK)
-#define PM_PSHIFT_BITS 6
-#define PM_PSHIFT_OFFSET (PM_STATUS_OFFSET - PM_PSHIFT_BITS)
-#define PM_PSHIFT_MASK (((1LL << PM_PSHIFT_BITS) - 1) << PM_PSHIFT_OFFSET)
-#define __PM_PSHIFT(x) (((uint64_t) (x) << PM_PSHIFT_OFFSET) & PM_PSHIFT_MASK)
-#define PM_PFRAME_MASK ((1LL << PM_PSHIFT_OFFSET) - 1)
-#define PM_PFRAME(x) ((x) & PM_PFRAME_MASK)
-
-#define __PM_SOFT_DIRTY (1LL)
-#define __PM_MMAP_EXCLUSIVE (2LL)
-#define PM_PRESENT PM_STATUS(4LL)
-#define PM_SWAP PM_STATUS(2LL)
-#define PM_FILE PM_STATUS(1LL)
-#define PM_SOFT_DIRTY __PM_PSHIFT(__PM_SOFT_DIRTY)
-#define PM_MMAP_EXCLUSIVE __PM_PSHIFT(__PM_MMAP_EXCLUSIVE)
-
+#define PM_ENTRY_BYTES 8
+#define PM_PFEAME_BITS 54
+#define PM_PFRAME_MASK ((1LL << PM_PFEAME_BITS) - 1)
+#define PM_PFRAME(x) ((x) & PM_PFRAME_MASK)
+#define PM_SOFT_DIRTY (1ULL << 55)
+#define PM_MMAP_EXCLUSIVE (1ULL << 56)
+#define PM_FILE (1ULL << 61)
+#define PM_SWAP (1ULL << 62)
+#define PM_PRESENT (1ULL << 63)
/*
* kernel page flags
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 2/3] pagemap: hide physical addresses from non-privileged users
From: Konstantin Khlebnikov @ 2015-05-12 9:43 UTC (permalink / raw)
To: linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Naoya Horiguchi,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Andrew Morton
Cc: Mark Williamson, Pavel Emelyanov,
linux-api-u79uwXL29TY76Z2rM5mHXA, Andy Lutomirski,
Vlastimil Babka, Pavel Machek, Mark Seaborn, Kirill A. Shutemov,
Linus Torvalds, Daniel James, Finn Grimwood
In-Reply-To: <20150512090156.24768.2521.stgit@buzz>
This patch makes pagemap readable for normal users back but hides physical
addresses from them. For some use cases PFN isn't required at all: flags
give information about presence, page type (anon/file/swap), soft-dirty mark,
and hint about page mapcount state: exclusive(mapcount = 1) or (mapcount > 1).
Signed-off-by: Konstantin Khlebnikov <khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org>
Fixes: ab676b7d6fbf ("pagemap: do not leak physical addresses to non-privileged userspace")
Link: lkml.kernel.org/r/1425935472-17949-1-git-send-email-kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org
---
fs/proc/task_mmu.c | 36 ++++++++++++++++++++++--------------
1 file changed, 22 insertions(+), 14 deletions(-)
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 29febec65de4..0b7a8ffec95f 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -962,6 +962,7 @@ struct pagemapread {
int pos, len; /* units: PM_ENTRY_BYTES, not bytes */
pagemap_entry_t *buffer;
bool v2;
+ bool show_pfn;
};
#define PAGEMAP_WALK_SIZE (PMD_SIZE)
@@ -1046,12 +1047,13 @@ out:
static void pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
struct vm_area_struct *vma, unsigned long addr, pte_t pte)
{
- u64 frame, flags;
+ u64 frame = 0, flags;
struct page *page = NULL;
int flags2 = 0;
if (pte_present(pte)) {
- frame = pte_pfn(pte);
+ if (pm->show_pfn)
+ frame = pte_pfn(pte);
flags = PM_PRESENT;
page = vm_normal_page(vma, addr, pte);
if (pte_soft_dirty(pte))
@@ -1087,15 +1089,19 @@ static void pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
static void thp_pmd_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
pmd_t pmd, int offset, int pmd_flags2)
{
+ u64 frame = 0;
+
/*
* Currently pmd for thp is always present because thp can not be
* swapped-out, migrated, or HWPOISONed (split in such cases instead.)
* This if-check is just to prepare for future implementation.
*/
- if (pmd_present(pmd))
- *pme = make_pme(PM_PFRAME(pmd_pfn(pmd) + offset)
- | PM_STATUS2(pm->v2, pmd_flags2) | PM_PRESENT);
- else
+ if (pmd_present(pmd)) {
+ if (pm->show_pfn)
+ frame = pmd_pfn(pmd) + offset;
+ *pme = make_pme(PM_PFRAME(frame) | PM_PRESENT |
+ PM_STATUS2(pm->v2, pmd_flags2));
+ } else
*pme = make_pme(PM_NOT_PRESENT(pm->v2) | PM_STATUS2(pm->v2, pmd_flags2));
}
#else
@@ -1171,11 +1177,14 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
static void huge_pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
pte_t pte, int offset, int flags2)
{
- if (pte_present(pte))
- *pme = make_pme(PM_PFRAME(pte_pfn(pte) + offset) |
- PM_STATUS2(pm->v2, flags2) |
- PM_PRESENT);
- else
+ u64 frame = 0;
+
+ if (pte_present(pte)) {
+ if (pm->show_pfn)
+ frame = pte_pfn(pte) + offset;
+ *pme = make_pme(PM_PFRAME(frame) | PM_PRESENT |
+ PM_STATUS2(pm->v2, flags2));
+ } else
*pme = make_pme(PM_NOT_PRESENT(pm->v2) |
PM_STATUS2(pm->v2, flags2));
}
@@ -1260,6 +1269,8 @@ static ssize_t pagemap_read(struct file *file, char __user *buf,
if (!count)
goto out_task;
+ /* do not disclose physical addresses: attack vector */
+ pm.show_pfn = capable(CAP_SYS_ADMIN);
pm.v2 = soft_dirty_cleared;
pm.len = (PAGEMAP_WALK_SIZE >> PAGE_SHIFT);
pm.buffer = kmalloc(pm.len * PM_ENTRY_BYTES, GFP_TEMPORARY);
@@ -1335,9 +1346,6 @@ out:
static int pagemap_open(struct inode *inode, struct file *file)
{
- /* do not disclose physical addresses: attack vector */
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
pr_warn_once("Bits 55-60 of /proc/PID/pagemap entries are about "
"to stop being page-shift some time soon. See the "
"linux/Documentation/vm/pagemap.txt for details.\n");
^ permalink raw reply related
* [PATCH v2 1/3] pagemap: add mmap-exclusive bit for marking pages mapped only here
From: Konstantin Khlebnikov @ 2015-05-12 9:43 UTC (permalink / raw)
To: linux-mm, Naoya Horiguchi, linux-kernel, Andrew Morton
Cc: Mark Williamson, Pavel Emelyanov, linux-api, Andy Lutomirski,
Vlastimil Babka, Pavel Machek, Mark Seaborn, Kirill A. Shutemov,
Linus Torvalds, Daniel James, Finn Grimwood
In-Reply-To: <20150512090156.24768.2521.stgit@buzz>
This patch sets bit 56 in pagemap if this page is mapped only once.
It allows to detect exclusively used pages without exposing PFN:
present file exclusive state
0 0 0 non-present
1 1 0 file page mapped somewhere else
1 1 1 file page mapped only here
1 0 0 anon non-CoWed page (shared with parent/child)
1 0 1 anon CoWed page (or never forked)
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Link: lkml.kernel.org/r/CAEVpBa+_RyACkhODZrRvQLs80iy0sqpdrd0AaP_-tgnX3Y9yNQ@mail.gmail.com
---
v2:
* handle transparent huge pages
* invert bit and rename shared -> exclusive (less confusing name)
---
Documentation/vm/pagemap.txt | 3 ++-
fs/proc/task_mmu.c | 10 ++++++++++
tools/vm/page-types.c | 12 ++++++++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/Documentation/vm/pagemap.txt b/Documentation/vm/pagemap.txt
index 6bfbc172cdb9..3cfbbb333ea1 100644
--- a/Documentation/vm/pagemap.txt
+++ b/Documentation/vm/pagemap.txt
@@ -16,7 +16,8 @@ There are three components to pagemap:
* Bits 0-4 swap type if swapped
* Bits 5-54 swap offset if swapped
* Bit 55 pte is soft-dirty (see Documentation/vm/soft-dirty.txt)
- * Bits 56-60 zero
+ * Bit 56 page exlusively mapped
+ * Bits 57-60 zero
* Bit 61 page is file-page or shared-anon
* Bit 62 page swapped
* Bit 63 page present
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 6dee68d013ff..29febec65de4 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -982,6 +982,7 @@ struct pagemapread {
#define PM_STATUS2(v2, x) (__PM_PSHIFT(v2 ? x : PAGE_SHIFT))
#define __PM_SOFT_DIRTY (1LL)
+#define __PM_MMAP_EXCLUSIVE (2LL)
#define PM_PRESENT PM_STATUS(4LL)
#define PM_SWAP PM_STATUS(2LL)
#define PM_FILE PM_STATUS(1LL)
@@ -1074,6 +1075,8 @@ static void pte_to_pagemap_entry(pagemap_entry_t *pme, struct pagemapread *pm,
if (page && !PageAnon(page))
flags |= PM_FILE;
+ if (page && page_mapcount(page) == 1)
+ flags2 |= __PM_MMAP_EXCLUSIVE;
if ((vma->vm_flags & VM_SOFTDIRTY))
flags2 |= __PM_SOFT_DIRTY;
@@ -1119,6 +1122,13 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
else
pmd_flags2 = 0;
+ if (pmd_present(*pmd)) {
+ struct page *page = pmd_page(*pmd);
+
+ if (page_mapcount(page) == 1)
+ pmd_flags2 |= __PM_MMAP_EXCLUSIVE;
+ }
+
for (; addr != end; addr += PAGE_SIZE) {
unsigned long offset;
pagemap_entry_t pme;
diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
index 8bdf16b8ba60..3a9f193526ee 100644
--- a/tools/vm/page-types.c
+++ b/tools/vm/page-types.c
@@ -70,9 +70,12 @@
#define PM_PFRAME(x) ((x) & PM_PFRAME_MASK)
#define __PM_SOFT_DIRTY (1LL)
+#define __PM_MMAP_EXCLUSIVE (2LL)
#define PM_PRESENT PM_STATUS(4LL)
#define PM_SWAP PM_STATUS(2LL)
+#define PM_FILE PM_STATUS(1LL)
#define PM_SOFT_DIRTY __PM_PSHIFT(__PM_SOFT_DIRTY)
+#define PM_MMAP_EXCLUSIVE __PM_PSHIFT(__PM_MMAP_EXCLUSIVE)
/*
@@ -100,6 +103,8 @@
#define KPF_SLOB_FREE 49
#define KPF_SLUB_FROZEN 50
#define KPF_SLUB_DEBUG 51
+#define KPF_FILE 62
+#define KPF_MMAP_EXCLUSIVE 63
#define KPF_ALL_BITS ((uint64_t)~0ULL)
#define KPF_HACKERS_BITS (0xffffULL << 32)
@@ -149,6 +154,9 @@ static const char * const page_flag_names[] = {
[KPF_SLOB_FREE] = "P:slob_free",
[KPF_SLUB_FROZEN] = "A:slub_frozen",
[KPF_SLUB_DEBUG] = "E:slub_debug",
+
+ [KPF_FILE] = "F:file",
+ [KPF_MMAP_EXCLUSIVE] = "1:mmap_exclusive",
};
@@ -452,6 +460,10 @@ static uint64_t expand_overloaded_flags(uint64_t flags, uint64_t pme)
if (pme & PM_SOFT_DIRTY)
flags |= BIT(SOFTDIRTY);
+ if (pme & PM_FILE)
+ flags |= BIT(FILE);
+ if (pme & PM_MMAP_EXCLUSIVE)
+ flags |= BIT(MMAP_EXCLUSIVE);
return flags;
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH RFC 0/3] pagemap: make useable for non-privilege users
From: Konstantin Khlebnikov @ 2015-05-12 9:43 UTC (permalink / raw)
To: linux-mm, Naoya Horiguchi, linux-kernel, Andrew Morton
Cc: Mark Williamson, Pavel Emelyanov, linux-api, Andy Lutomirski,
Vlastimil Babka, Pavel Machek, Mark Seaborn, Kirill A. Shutemov,
Linus Torvalds, Daniel James, Finn Grimwood
This patchset tries to make pagemap useable again in the safe way.
First patch adds bit 'map-exlusive' which is set if page is mapped only here.
Second patch restores access for non-privileged users but hides pfn if task
has no capability CAP_SYS_ADMIN. Third patch removes page-shift bits and
completes migration to the new pagemap format (flags soft-dirty and
mmap-exlusive are available only in the new format).
---
Konstantin Khlebnikov (3):
pagemap: add mmap-exclusive bit for marking pages mapped only here
pagemap: hide physical addresses from non-privileged users
pagemap: switch to the new format and do some cleanup
Documentation/vm/pagemap.txt | 3 -
fs/proc/task_mmu.c | 178 +++++++++++++++++-------------------------
tools/vm/page-types.c | 35 ++++----
3 files changed, 91 insertions(+), 125 deletions(-)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v3 3/3] proc: add kpageidle file
From: Vladimir Davydov @ 2015-05-12 9:41 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, Johannes Weiner, Michal Hocko, Greg Thelen,
Michel Lespinasse, David Rientjes, Pavel Emelyanov,
Cyrill Gorcunov, Jonathan Corbet, linux-api, linux-doc, linux-mm,
cgroups, linux-kernel, Rik van Riel, Hugh Dickins,
Christoph Lameter, Paul E. McKenney, Peter Zijlstra
In-Reply-To: <20150510103429.GA17628@esperanza>
On Sun, May 10, 2015 at 01:34:29PM +0300, Vladimir Davydov wrote:
> On Sun, May 10, 2015 at 12:12:38AM +0900, Minchan Kim wrote:
> > Yeb, I might be paranoid but my point is it might work now on most of
> > arch but it seem to be buggy/fragile/subtle because we couldn't prove
> > all arch/compiler don't make any trouble. So, intead of adding more
> > logics based on fragile, please use right lock model. If lock becomes
> > big trouble by overhead, let's fix it(for instance, use WRITE_ONCE for
> > update-side and READ_ONCE for read-side) if I don't miss something.
>
> IMO, locking would be an overkill. READ_ONCE is OK, because it has no
> performance implications, but I would prefer to be convinced that it is
> 100% necessary before adding it just in case.
Finally, I'm convinced we do need synchronization here :-) Sorry for
being so stubborn and thank you for your patience.
After examining page_referenced() with the knowledge that the compiler
may be completely unreliable and split page->mapping read/writes as it
wants, I've drawn the conclusion that it is safer to take
page_zone->lru_lock for checking if the page is on an LRU list, just as
you proposed initially, because otherwise we need to insert those
READ/WRITE_ONCE in every nook and cranny, which would look confusing
provided we only needed them for this idle page tracking feature, which
might even be not compiled in.
I'll fix it and resend.
Thanks,
Vladimir
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 5/6] nohz: support PR_DATAPLANE_STRICT mode
From: Peter Zijlstra @ 2015-05-12 9:38 UTC (permalink / raw)
To: Chris Metcalf
Cc: Gilad Ben Yossef, Steven Rostedt, Ingo Molnar, Andrew Morton,
Rik van Riel, Tejun Heo, Thomas Gleixner, Frederic Weisbecker,
Paul E. McKenney, Christoph Lameter, Srivatsa S. Bhat, linux-doc,
linux-api, linux-kernel
In-Reply-To: <1431107927-13998-6-git-send-email-cmetcalf@ezchip.com>
On Fri, May 08, 2015 at 01:58:46PM -0400, Chris Metcalf wrote:
> +++ b/kernel/time/tick-sched.c
> @@ -436,6 +436,20 @@ static void dataplane_quiesce(void)
> (jiffies - start));
> dump_stack();
> }
> +
> + /*
> + * Kill the process if it violates STRICT mode. Note that this
> + * code also results in killing the task if a kernel bug causes an
> + * irq to be delivered to this core.
> + */
> + if ((task->dataplane_flags & (PR_DATAPLANE_STRICT|PR_DATAPLANE_PRCTL))
> + == PR_DATAPLANE_STRICT) {
> + pr_warn("Dataplane STRICT mode violated; process killed.\n");
> + dump_stack();
> + task->dataplane_flags &= ~PR_DATAPLANE_QUIESCE;
> + local_irq_enable();
> + do_group_exit(SIGKILL);
> + }
> }
So while I'm all for hard fails like this, can we not provide a wee bit
more information in the siginfo ? And maybe use a slightly less fatal
signal, such that userspace can actually catch it and dump state in
debug modes?
^ permalink raw reply
* Re: [PATCH 4/6] nohz: support PR_DATAPLANE_QUIESCE
From: Peter Zijlstra @ 2015-05-12 9:33 UTC (permalink / raw)
To: Chris Metcalf
Cc: Gilad Ben Yossef, Steven Rostedt, Ingo Molnar, Andrew Morton,
Rik van Riel, Tejun Heo, Thomas Gleixner, Frederic Weisbecker,
Paul E. McKenney, Christoph Lameter, Srivatsa S. Bhat,
linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1431107927-13998-5-git-send-email-cmetcalf-d5a29ZRxExrQT0dZR+AlfA@public.gmane.org>
On Fri, May 08, 2015 at 01:58:45PM -0400, Chris Metcalf wrote:
> This prctl() flag for PR_SET_DATAPLANE sets a mode that requires the
> kernel to quiesce any pending timer interrupts prior to returning
> to userspace. When running with this mode set, sys calls (and page
> faults, etc.) can be inordinately slow. However, user applications
> that want to guarantee that no unexpected interrupts will occur
> (even if they call into the kernel) can set this flag to guarantee
> that semantics.
Currently people hot-unplug and hot-plug the CPU to do this. Obviously
that's a wee bit horrible :-)
Not sure if a prctl like this is any better though. This is a CPU
properly not a process one.
ISTR people talking about 'quiesce' sysfs file, along side the hotplug
stuff, I can't quite remember.
^ permalink raw reply
* CONFIG_ISOLATION=y (was: [PATCH 0/6] support "dataplane" mode for nohz_full)
From: Ingo Molnar @ 2015-05-12 9:10 UTC (permalink / raw)
To: Chris Metcalf
Cc: Steven Rostedt, Frederic Weisbecker, Andrew Morton, paulmck,
Gilad Ben Yossef, Peter Zijlstra, Rik van Riel, Tejun Heo,
Thomas Gleixner, Christoph Lameter, Srivatsa S. Bhat, linux-doc,
linux-api, linux-kernel
In-Reply-To: <5550F077.6030906@ezchip.com>
* Chris Metcalf <cmetcalf@ezchip.com> wrote:
> - ISOLATION (Frederic). I like this but it conflicts with other uses
> of "isolation" in the kernel: cgroup isolation, lru page isolation,
> iommu isolation, scheduler isolation (at least it's a superset of
> that one), etc. Also, we're not exactly isolating a task - often
> a "dataplane" app consists of a bunch of interacting threads in
> userspace, so not exactly isolated. So perhaps it's too confusing.
So I'd vote for Frederic's CONFIG_ISOLATION=y, mostly because this is
a high level kernel feature, so it won't conflict with isolation
concepts in lower level subsystems such as IOMMU isolation - and other
higher level features like scheduler isolation are basically another
partial implementation we want to merge with all this...
nohz, RCU tricks, watchdog defaults, isolcpus and various other
measures to keep these CPUs and workloads as isolated as possible
are (or should become) components of this high level concept.
Ideally CONFIG_ISOLATION=y would be a kernel feature that has almost
zero overhead on normal workloads and on non-isolated CPUs, so that
Linux distributions can enable it.
Enabling CONFIG_ISOLATION=y should be the only 'kernel config' step
needed: just like cpusets, the configuration of isolated CPUs should
be a completely boot option free excercise that can be dynamically
done and undone by the administrator via an intuitive interface.
Thanks,
Ingo
^ permalink raw reply
* Re: [PATCH 2/2] Define syscall number for fchmodat4
From: Michael Kerrisk @ 2015-05-12 8:49 UTC (permalink / raw)
To: William Orr
Cc: Linux Kernel, Andrew Ayer, Al Viro, Linux-Fsdevel, Miklos Szeredi,
Linux API
In-Reply-To: <1431310042-52247-3-git-send-email-will-o9VNkMWGZsBWk0Htik3J/w@public.gmane.org>
[expanding CC]
On Mon, May 11, 2015 at 4:07 AM, William Orr <will-o9VNkMWGZsBWk0Htik3J/w@public.gmane.org> wrote:
> ---
> arch/x86/syscalls/syscall_32.tbl | 1 +
> arch/x86/syscalls/syscall_64.tbl | 1 +
> include/uapi/asm-generic/unistd.h | 4 +++-
> 3 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/syscalls/syscall_32.tbl b/arch/x86/syscalls/syscall_32.tbl
> index ef8187f..cc8ada8 100644
> --- a/arch/x86/syscalls/syscall_32.tbl
> +++ b/arch/x86/syscalls/syscall_32.tbl
> @@ -365,3 +365,4 @@
> 356 i386 memfd_create sys_memfd_create
> 357 i386 bpf sys_bpf
> 358 i386 execveat sys_execveat stub32_execveat
> +359 i386 fchmodat4 sys_fchmodat4
> diff --git a/arch/x86/syscalls/syscall_64.tbl b/arch/x86/syscalls/syscall_64.tbl
> index 9ef32d5..bbf8c6b 100644
> --- a/arch/x86/syscalls/syscall_64.tbl
> +++ b/arch/x86/syscalls/syscall_64.tbl
> @@ -329,6 +329,7 @@
> 320 common kexec_file_load sys_kexec_file_load
> 321 common bpf sys_bpf
> 322 64 execveat stub_execveat
> +323 common fchmodat4 sys_fchmodat4
>
> #
> # x32-specific system call numbers start at 512 to avoid cache impact
> diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
> index e016bd9..6e362a2 100644
> --- a/include/uapi/asm-generic/unistd.h
> +++ b/include/uapi/asm-generic/unistd.h
> @@ -709,9 +709,11 @@ __SYSCALL(__NR_memfd_create, sys_memfd_create)
> __SYSCALL(__NR_bpf, sys_bpf)
> #define __NR_execveat 281
> __SC_COMP(__NR_execveat, sys_execveat, compat_sys_execveat)
> +#define __NR_fchmodat4 282
> +__SYSCALL(__NR_fchmodat4, sys_fchmodat4);
>
> #undef __NR_syscalls
> -#define __NR_syscalls 282
> +#define __NR_syscalls 283
>
> /*
> * All syscalls below here should go away really,
> --
> 2.1.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Michael Kerrisk Linux man-pages maintainer;
http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface", http://blog.man7.org/
^ permalink raw reply
* Re: [PATCH 1/2] Implement fchmodat4 syscall
From: Michael Kerrisk @ 2015-05-12 8:48 UTC (permalink / raw)
To: William Orr
Cc: Linux Kernel, Al Viro, Miklos Szeredi, Linux API, Linux-Fsdevel,
Andrew Ayer
In-Reply-To: <1431310042-52247-2-git-send-email-will@worrbase.com>
[expanding CC]
On Mon, May 11, 2015 at 4:07 AM, William Orr <will@worrbase.com> wrote:
> Adds fchmodat4 which more closely matches POSIX by taking 4 arguments,
> including the flags argument. flags are the same as fchownat(2), implementing
> both AT_SYMLINK_NOFOLLOW and AT_EMPTY_PATH
>
> Based heavily off of Andrew Ayer's patch from 2012.
> ---
> fs/open.c | 19 +++++++++++++++++--
> include/linux/syscalls.h | 2 ++
> 2 files changed, 19 insertions(+), 2 deletions(-)
>
> diff --git a/fs/open.c b/fs/open.c
> index 98e5a52..00dd0f7 100644
> --- a/fs/open.c
> +++ b/fs/open.c
> @@ -504,6 +504,9 @@ static int chmod_common(struct path *path, umode_t mode)
> struct iattr newattrs;
> int error;
>
> + if (S_ISLNK(inode->i_mode))
> + return -EOPNOTSUPP;
> +
> error = mnt_want_write(path->mnt);
> if (error)
> return error;
> @@ -541,9 +544,21 @@ SYSCALL_DEFINE2(fchmod, unsigned int, fd, umode_t, mode)
>
> SYSCALL_DEFINE3(fchmodat, int, dfd, const char __user *, filename, umode_t, mode)
> {
> + return sys_fchmodat4(dfd, filename, mode, 0);
> +}
> +
> +SYSCALL_DEFINE4(fchmodat4, int, dfd, const char __user *, filename, umode_t, mode, int, flags)
> +{
> struct path path;
> int error;
> - unsigned int lookup_flags = LOOKUP_FOLLOW;
> + unsigned int lookup_flags;
> +
> + if (flags & ~(AT_SYMLINK_NOFOLLOW | AT_EMPTY_PATH))
> + return -EINVAL;
> +
> + lookup_flags = (flags & AT_SYMLINK_NOFOLLOW) ? 0 : LOOKUP_FOLLOW;
> + if (flags & AT_EMPTY_PATH)
> + lookup_flags |= LOOKUP_EMPTY;
> retry:
> error = user_path_at(dfd, filename, lookup_flags, &path);
> if (!error) {
> @@ -559,7 +574,7 @@ retry:
>
> SYSCALL_DEFINE2(chmod, const char __user *, filename, umode_t, mode)
> {
> - return sys_fchmodat(AT_FDCWD, filename, mode);
> + return sys_fchmodat4(AT_FDCWD, filename, mode, 0);
> }
>
> static int chown_common(struct path *path, uid_t user, gid_t group)
> diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
> index 76d1e38..d6e9602 100644
> --- a/include/linux/syscalls.h
> +++ b/include/linux/syscalls.h
> @@ -769,6 +769,8 @@ asmlinkage long sys_futimesat(int dfd, const char __user *filename,
> asmlinkage long sys_faccessat(int dfd, const char __user *filename, int mode);
> asmlinkage long sys_fchmodat(int dfd, const char __user * filename,
> umode_t mode);
> +asmlinkage long sys_fchmodat4(int dfd, const char __user * filename,
> + umode_t mode, int flags);
> asmlinkage long sys_fchownat(int dfd, const char __user *filename, uid_t user,
> gid_t group, int flag);
> asmlinkage long sys_openat(int dfd, const char __user *filename, int flags,
> --
> 2.1.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Michael Kerrisk Linux man-pages maintainer;
http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface", http://blog.man7.org/
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox