* Re: [PATCH v27 04/12] LRNG - add switchable DRNG support
From: Stephan Müller @ 2020-01-12 10:12 UTC (permalink / raw)
To: kbuild test robot
Cc: kbuild-all, Arnd Bergmann, Greg Kroah-Hartman, linux-crypto, LKML,
linux-api, Eric W. Biederman, Alexander E. Patrakov,
Ahmed S. Darwish, Theodore Y. Ts'o, Willy Tarreau,
Matthew Garrett, Vito Caputo, Andreas Dilger, Jan Kara,
Ray Strode, William Jon McCann, zhangjs, Andy Lutomirski,
Florian Weimer, Lennart Poettering, Ni
In-Reply-To: <202001111540.5y2emdGQ%lkp@intel.com>
Am Samstag, 11. Januar 2020, 08:09:50 CET schrieb kbuild test robot:
Hi,
> Hi "Stephan,
>
> Thank you for the patch! Perhaps something to improve:
>
> [auto build test WARNING on char-misc/char-misc-testing]
> [also build test WARNING on cryptodev/master crypto/master v5.5-rc5
> next-20200110] [if your patch is applied to the wrong git tree, please drop
> us a note to help improve the system. BTW, we also suggest to use '--base'
> option to specify the base tree in git format-patch, please see
> https://stackoverflow.com/a/37406982]
>
> url:
> https://github.com/0day-ci/linux/commits/Stephan-M-ller/dev-random-a-new-ap
> proach-with-full-SP800-90B/20200110-084934 base:
> https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git
> 68faa679b8be1a74e6663c21c3a9d25d32f1c079 reproduce:
> # apt-get install sparse
> # sparse version: v0.6.1-129-g341daf20-dirty
> make ARCH=x86_64 allmodconfig
> make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'
>
> If you fix the issue, kindly add following tag
> Reported-by: kbuild test robot <lkp@intel.com>
>
>
> sparse warnings: (new ones prefixed by >>)
>
> >> drivers/char/lrng/lrng_internal.h:239:39: sparse: sparse: context
> >> imbalance in 'lrng_drng_switch' - unexpected unlock
> vim +/lrng_drng_switch +239 drivers/char/lrng/lrng_internal.h
>
> 58c283819a1e87 Stephan Müller 2020-01-09 233
> 58c283819a1e87 Stephan Müller 2020-01-09 234 /* Unlock the DRNG */
> 58c283819a1e87 Stephan Müller 2020-01-09 235 static __always_inline void
> lrng_drng_unlock(struct lrng_drng *drng, 58c283819a1e87 Stephan Müller
> 2020-01-09 236 unsigned long *flags)
58c283819a1e87 Stephan
> Müller 2020-01-09 237 {
> 58c283819a1e87 Stephan Müller 2020-01-09 238 if
> (lrng_drng_is_atomic(drng)) 58c283819a1e87 Stephan Müller 2020-01-09 @239
> spin_unlock_irqrestore(&drng->spin_lock, *flags); 58c283819a1e87
Stephan
> Müller 2020-01-09 240 else
> 58c283819a1e87 Stephan Müller 2020-01-09 241 mutex_unlock(&drng-
>lock);
> 58c283819a1e87 Stephan Müller 2020-01-09 242 }
> 58c283819a1e87 Stephan Müller 2020-01-09 243
>
> :::::: The code at line 239 was first introduced by commit
> :::::: 58c283819a1e879bc2e30d05720285f9709f7f6d Linux Random Number
> :::::: Generator
> ::::::
> :::::: TO: Stephan Müller <smueller@chronox.de>
> :::::: CC: 0day robot <lkp@intel.com>
>
> ---
> 0-DAY kernel test infrastructure Open Source Technology
> Center https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org Intel
> Corporation
After analyzing the issue a bit more, it seems that I have to remove
"unlikely" from lrng_drng_lock which seems to cause additional grief
with sparse. Note, sparse will still report a lock context imbalance as it
used to since we indeed have two lock context as documented in
lrng_drng_switch.
Ciao
Stephan
^ permalink raw reply
* Investment opportunity
From: Peter Wong @ 2020-01-12 5:00 UTC (permalink / raw)
To: linux-api-u79uwXL29TY76Z2rM5mHXA
Greetings,
Please read the attached investment proposal and reply for more details.
Are you interested in loan?
Sincerely: Peter Wong
----------------------------------------------------
This email was sent by the shareware version of Postman Professional.
^ permalink raw reply
* Re: [PATCH 4/4] mm/madvise: allow KSM hints for remote API
From: SeongJae Park @ 2020-01-11 7:42 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, LKML, linux-mm, linux-api, oleksandr,
Suren Baghdasaryan, Tim Murray, Daniel Colascione, Sandeep Patil,
Sonny Rao, Brian Geffon, Michal Hocko, Johannes Weiner,
Shakeel Butt, John Dias
In-Reply-To: <20200110213433.94739-5-minchan@kernel.org>
On Fri, 10 Jan 2020 13:34:33 -0800 Minchan Kim <minchan@kernel.org> wrote:
> From: Oleksandr Natalenko <oleksandr@redhat.com>
>
> It all began with the fact that KSM works only on memory that is marked
> by madvise(). And the only way to get around that is to either:
>
> * use LD_PRELOAD; or
> * patch the kernel with something like UKSM or PKSM.
>
> (i skip ptrace can of worms here intentionally)
>
> To overcome this restriction, lets employ a new remote madvise API. This
> can be used by some small userspace helper daemon that will do auto-KSM
> job for us.
>
> I think of two major consumers of remote KSM hints:
>
> * hosts, that run containers, especially similar ones and especially in
> a trusted environment, sharing the same runtime like Node.js;
>
> * heavy applications, that can be run in multiple instances, not
> limited to opensource ones like Firefox, but also those that cannot be
> modified since they are binary-only and, maybe, statically linked.
>
> Speaking of statistics, more numbers can be found in the very first
> submission, that is related to this one [1]. For my current setup with
> two Firefox instances I get 100 to 200 MiB saved for the second instance
> depending on the amount of tabs.
>
> 1 FF instance with 15 tabs:
>
> $ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
> 410
>
> 2 FF instances, second one has 12 tabs (all the tabs are different):
>
> $ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
> 592
>
> At the very moment I do not have specific numbers for containerised
> workload, but those should be comparable in case the containers share
> similar/same runtime.
>
> [1] https://lore.kernel.org/patchwork/patch/1012142/
>
> Signed-off-by: Oleksandr Natalenko <oleksandr@redhat.com>
> Signed-off-by: Minchan Kim <minchan@kernel.org>
Reviewed-by: SeongJae Park <sjpark@amazon.de>
> ---
> mm/madvise.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/mm/madvise.c b/mm/madvise.c
> index eb42b2b7f49b..3aa9aec6bfd9 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -1000,6 +1000,8 @@ process_madvise_behavior_valid(int behavior)
> switch (behavior) {
> case MADV_COLD:
> case MADV_PAGEOUT:
> + case MADV_MERGEABLE:
> + case MADV_UNMERGEABLE:
> return true;
> default:
> return false;
> --
> 2.25.0.rc1.283.g88dfdc4193-goog
^ permalink raw reply
* Re: [PATCH 1/4] mm: factor out madvise's core functionality
From: SeongJae Park @ 2020-01-11 7:37 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, LKML, linux-mm, linux-api-u79uwXL29TY76Z2rM5mHXA,
oleksandr-H+wXaHxf7aLQT0dZR+AlfA, Suren Baghdasaryan, Tim Murray,
Daniel Colascione, Sandeep Patil, Sonny Rao, Brian Geffon,
Michal Hocko, Johannes Weiner, Shakeel Butt, John Dias
In-Reply-To: <20200110213433.94739-2-minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
On Fri, 10 Jan 2020 13:34:30 -0800 Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> This patch factor out madvise's core functionality so that upcoming
> patch can reuse it without duplication. It shouldn't change any behavior.
>
> Signed-off-by: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---
> mm/madvise.c | 194 +++++++++++++++++++++++++++++----------------------
> 1 file changed, 111 insertions(+), 83 deletions(-)
>
> diff --git a/mm/madvise.c b/mm/madvise.c
> index bcdb6a042787..0c901de531e4 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -35,6 +35,7 @@
> struct madvise_walk_private {
> struct mmu_gather *tlb;
> bool pageout;
> + struct task_struct *task;
> };
>
> /*
> @@ -306,12 +307,13 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
> bool pageout = private->pageout;
> struct mm_struct *mm = tlb->mm;
> struct vm_area_struct *vma = walk->vma;
> + struct task_struct *task = private->task;
> pte_t *orig_pte, *pte, ptent;
> spinlock_t *ptl;
> struct page *page = NULL;
> LIST_HEAD(page_list);
>
> - if (fatal_signal_pending(current))
> + if (fatal_signal_pending(task))
> return -EINTR;
>
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> @@ -469,12 +471,14 @@ static const struct mm_walk_ops cold_walk_ops = {
> };
>
> static void madvise_cold_page_range(struct mmu_gather *tlb,
> + struct task_struct *task,
> struct vm_area_struct *vma,
> unsigned long addr, unsigned long end)
> {
> struct madvise_walk_private walk_private = {
> .pageout = false,
> .tlb = tlb,
> + .task = task,
> };
>
> tlb_start_vma(tlb, vma);
> @@ -482,7 +486,7 @@ static void madvise_cold_page_range(struct mmu_gather *tlb,
> tlb_end_vma(tlb, vma);
> }
>
> -static long madvise_cold(struct vm_area_struct *vma,
> +static long madvise_cold(struct task_struct *task, struct vm_area_struct *vma,
> struct vm_area_struct **prev,
> unsigned long start_addr, unsigned long end_addr)
> {
> @@ -495,19 +499,21 @@ static long madvise_cold(struct vm_area_struct *vma,
>
> lru_add_drain();
> tlb_gather_mmu(&tlb, mm, start_addr, end_addr);
> - madvise_cold_page_range(&tlb, vma, start_addr, end_addr);
> + madvise_cold_page_range(&tlb, task, vma, start_addr, end_addr);
> tlb_finish_mmu(&tlb, start_addr, end_addr);
>
> return 0;
> }
>
> static void madvise_pageout_page_range(struct mmu_gather *tlb,
> + struct task_struct *task,
> struct vm_area_struct *vma,
> unsigned long addr, unsigned long end)
> {
> struct madvise_walk_private walk_private = {
> .pageout = true,
> .tlb = tlb,
> + .task = task,
> };
>
> tlb_start_vma(tlb, vma);
> @@ -531,9 +537,9 @@ static inline bool can_do_pageout(struct vm_area_struct *vma)
> inode_permission(file_inode(vma->vm_file), MAY_WRITE) == 0;
> }
>
> -static long madvise_pageout(struct vm_area_struct *vma,
> - struct vm_area_struct **prev,
> - unsigned long start_addr, unsigned long end_addr)
> +static long madvise_pageout(struct task_struct *task,
> + struct vm_area_struct *vma, struct vm_area_struct **prev,
> + unsigned long start_addr, unsigned long end_addr)
> {
> struct mm_struct *mm = vma->vm_mm;
> struct mmu_gather tlb;
> @@ -547,7 +553,7 @@ static long madvise_pageout(struct vm_area_struct *vma,
>
> lru_add_drain();
> tlb_gather_mmu(&tlb, mm, start_addr, end_addr);
> - madvise_pageout_page_range(&tlb, vma, start_addr, end_addr);
> + madvise_pageout_page_range(&tlb, task, vma, start_addr, end_addr);
> tlb_finish_mmu(&tlb, start_addr, end_addr);
>
> return 0;
> @@ -751,7 +757,8 @@ static long madvise_dontneed_single_vma(struct vm_area_struct *vma,
> return 0;
> }
>
> -static long madvise_dontneed_free(struct vm_area_struct *vma,
> +static long madvise_dontneed_free(struct mm_struct *mm,
> + struct vm_area_struct *vma,
> struct vm_area_struct **prev,
> unsigned long start, unsigned long end,
> int behavior)
> @@ -763,8 +770,8 @@ static long madvise_dontneed_free(struct vm_area_struct *vma,
> if (!userfaultfd_remove(vma, start, end)) {
> *prev = NULL; /* mmap_sem has been dropped, prev is stale */
>
> - down_read(¤t->mm->mmap_sem);
> - vma = find_vma(current->mm, start);
> + down_read(&mm->mmap_sem);
> + vma = find_vma(mm, start);
> if (!vma)
> return -ENOMEM;
> if (start < vma->vm_start) {
> @@ -811,7 +818,8 @@ static long madvise_dontneed_free(struct vm_area_struct *vma,
> * Application wants to free up the pages and associated backing store.
> * This is effectively punching a hole into the middle of a file.
> */
> -static long madvise_remove(struct vm_area_struct *vma,
> +static long madvise_remove(struct mm_struct *mm,
> + struct vm_area_struct *vma,
> struct vm_area_struct **prev,
> unsigned long start, unsigned long end)
> {
> @@ -845,13 +853,13 @@ static long madvise_remove(struct vm_area_struct *vma,
> get_file(f);
> if (userfaultfd_remove(vma, start, end)) {
> /* mmap_sem was not released by userfaultfd_remove() */
> - up_read(¤t->mm->mmap_sem);
> + up_read(&mm->mmap_sem);
> }
> error = vfs_fallocate(f,
> FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
> offset, end - start);
> fput(f);
> - down_read(¤t->mm->mmap_sem);
> + down_read(&mm->mmap_sem);
> return error;
> }
>
> @@ -925,21 +933,23 @@ static int madvise_inject_error(int behavior,
> #endif
>
> static long
> -madvise_vma(struct vm_area_struct *vma, struct vm_area_struct **prev,
> +madvise_vma(struct task_struct *task, struct mm_struct *mm,
> + struct vm_area_struct *vma, struct vm_area_struct **prev,
> unsigned long start, unsigned long end, int behavior)
> {
> switch (behavior) {
> case MADV_REMOVE:
> - return madvise_remove(vma, prev, start, end);
> + return madvise_remove(mm, vma, prev, start, end);
> case MADV_WILLNEED:
> return madvise_willneed(vma, prev, start, end);
> case MADV_COLD:
> - return madvise_cold(vma, prev, start, end);
> + return madvise_cold(task, vma, prev, start, end);
> case MADV_PAGEOUT:
> - return madvise_pageout(vma, prev, start, end);
> + return madvise_pageout(task, vma, prev, start, end);
> case MADV_FREE:
> case MADV_DONTNEED:
> - return madvise_dontneed_free(vma, prev, start, end, behavior);
> + return madvise_dontneed_free(mm, vma, prev, start,
> + end, behavior);
> default:
> return madvise_behavior(vma, prev, start, end, behavior);
> }
> @@ -984,67 +994,19 @@ madvise_behavior_valid(int behavior)
> }
>
> /*
> - * The madvise(2) system call.
> + * madvise_common - request behavior hint to address range of the target process
> *
> - * Applications can use madvise() to advise the kernel how it should
> - * handle paging I/O in this VM area. The idea is to help the kernel
> - * use appropriate read-ahead and caching techniques. The information
> - * provided is advisory only, and can be safely disregarded by the
> - * kernel without affecting the correct operation of the application.
> + * @task: task_struct got behavior hint, not giving the hint
> + * @mm: mm_struct got behavior hint, not giving the hint
> + * @start: base address of the hinted range
> + * @len_in: length of the hinted range
> + * @behavior: requested hint
> *
> - * behavior values:
> - * MADV_NORMAL - the default behavior is to read clusters. This
> - * results in some read-ahead and read-behind.
> - * MADV_RANDOM - the system should read the minimum amount of data
> - * on any access, since it is unlikely that the appli-
> - * cation will need more than what it asks for.
> - * MADV_SEQUENTIAL - pages in the given range will probably be accessed
> - * once, so they can be aggressively read ahead, and
> - * can be freed soon after they are accessed.
> - * MADV_WILLNEED - the application is notifying the system to read
> - * some pages ahead.
> - * MADV_DONTNEED - the application is finished with the given range,
> - * so the kernel can free resources associated with it.
> - * MADV_FREE - the application marks pages in the given range as lazy free,
> - * where actual purges are postponed until memory pressure happens.
> - * MADV_REMOVE - the application wants to free up the given range of
> - * pages and associated backing store.
> - * MADV_DONTFORK - omit this area from child's address space when forking:
> - * typically, to avoid COWing pages pinned by get_user_pages().
> - * MADV_DOFORK - cancel MADV_DONTFORK: no longer omit this area when forking.
> - * MADV_WIPEONFORK - present the child process with zero-filled memory in this
> - * range after a fork.
> - * MADV_KEEPONFORK - undo the effect of MADV_WIPEONFORK
> - * MADV_HWPOISON - trigger memory error handler as if the given memory range
> - * were corrupted by unrecoverable hardware memory failure.
> - * MADV_SOFT_OFFLINE - try to soft-offline the given range of memory.
> - * MADV_MERGEABLE - the application recommends that KSM try to merge pages in
> - * this area with pages of identical content from other such areas.
> - * MADV_UNMERGEABLE- cancel MADV_MERGEABLE: no longer merge pages with others.
> - * MADV_HUGEPAGE - the application wants to back the given range by transparent
> - * huge pages in the future. Existing pages might be coalesced and
> - * new pages might be allocated as THP.
> - * MADV_NOHUGEPAGE - mark the given range as not worth being backed by
> - * transparent huge pages so the existing pages will not be
> - * coalesced into THP and new pages will not be allocated as THP.
> - * MADV_DONTDUMP - the application wants to prevent pages in the given range
> - * from being included in its core dump.
> - * MADV_DODUMP - cancel MADV_DONTDUMP: no longer exclude from core dump.
> - *
> - * return values:
> - * zero - success
> - * -EINVAL - start + len < 0, start is not page-aligned,
> - * "behavior" is not a valid value, or application
> - * is attempting to release locked or shared pages,
> - * or the specified address range includes file, Huge TLB,
> - * MAP_SHARED or VMPFNMAP range.
> - * -ENOMEM - addresses in the specified range are not currently
> - * mapped, or are outside the AS of the process.
> - * -EIO - an I/O error occurred while paging in data.
> - * -EBADF - map exists, but area maps something that isn't a file.
> - * -EAGAIN - a kernel resource was temporarily unavailable.
> + * @task could be a zombie leader if it calls sys_exit so accessing mm_struct
> + * via task->mm is prohibited. Please use @mm instead of task->mm.
> */
> -SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
> +static int madvise_common(struct task_struct *task, struct mm_struct *mm,
> + unsigned long start, size_t len_in, int behavior)
> {
> unsigned long end, tmp;
> struct vm_area_struct *vma, *prev;
> @@ -1082,10 +1044,10 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
>
> write = madvise_need_mmap_write(behavior);
> if (write) {
> - if (down_write_killable(¤t->mm->mmap_sem))
> + if (down_write_killable(&mm->mmap_sem))
> return -EINTR;
> } else {
> - down_read(¤t->mm->mmap_sem);
> + down_read(&mm->mmap_sem);
> }
>
> /*
> @@ -1093,7 +1055,7 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
> * ranges, just ignore them, but return -ENOMEM at the end.
> * - different from the way of handling in mlock etc.
> */
> - vma = find_vma_prev(current->mm, start, &prev);
> + vma = find_vma_prev(mm, start, &prev);
> if (vma && start > vma->vm_start)
> prev = vma;
>
> @@ -1118,7 +1080,7 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
> tmp = end;
>
> /* Here vma->vm_start <= start < tmp <= (end|vma->vm_end). */
> - error = madvise_vma(vma, &prev, start, tmp, behavior);
> + error = madvise_vma(task, mm, vma, &prev, start, tmp, behavior);
> if (error)
> goto out;
> start = tmp;
> @@ -1130,14 +1092,80 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
> if (prev)
> vma = prev->vm_next;
> else /* madvise_remove dropped mmap_sem */
> - vma = find_vma(current->mm, start);
> + vma = find_vma(mm, start);
> }
> out:
> blk_finish_plug(&plug);
> if (write)
> - up_write(¤t->mm->mmap_sem);
> + up_write(&mm->mmap_sem);
> else
> - up_read(¤t->mm->mmap_sem);
> + up_read(&mm->mmap_sem);
>
> return error;
> }
> +
> +/*
> + * The madvise(2) system call.
> + *
> + * Applications can use madvise() to advise the kernel how it should
> + * handle paging I/O in this VM area. The idea is to help the kernel
> + * use appropriate read-ahead and caching techniques. The information
> + * provided is advisory only, and can be safely disregarded by the
> + * kernel without affecting the correct operation of the application.
> + *
> + * behavior values:
> + * MADV_NORMAL - the default behavior is to read clusters. This
> + * results in some read-ahead and read-behind.
> + * MADV_RANDOM - the system should read the minimum amount of data
> + * on any access, since it is unlikely that the appli-
> + * cation will need more than what it asks for.
> + * MADV_SEQUENTIAL - pages in the given range will probably be accessed
> + * once, so they can be aggressively read ahead, and
> + * can be freed soon after they are accessed.
> + * MADV_WILLNEED - the application is notifying the system to read
> + * some pages ahead.
> + * MADV_DONTNEED - the application is finished with the given range,
> + * so the kernel can free resources associated with it.
> + * MADV_FREE - the application marks pages in the given range as lazy free,
> + * where actual purges are postponed until memory pressure happens.
> + * MADV_REMOVE - the application wants to free up the given range of
> + * pages and associated backing store.
> + * MADV_DONTFORK - omit this area from child's address space when forking:
> + * typically, to avoid COWing pages pinned by get_user_pages().
> + * MADV_DOFORK - cancel MADV_DONTFORK: no longer omit this area when forking.
> + * MADV_WIPEONFORK - present the child process with zero-filled memory in this
> + * range after a fork.
> + * MADV_KEEPONFORK - undo the effect of MADV_WIPEONFORK
> + * MADV_HWPOISON - trigger memory error handler as if the given memory range
> + * were corrupted by unrecoverable hardware memory failure.
> + * MADV_SOFT_OFFLINE - try to soft-offline the given range of memory.
> + * MADV_MERGEABLE - the application recommends that KSM try to merge pages in
> + * this area with pages of identical content from other such areas.
> + * MADV_UNMERGEABLE- cancel MADV_MERGEABLE: no longer merge pages with others.
> + * MADV_HUGEPAGE - the application wants to back the given range by transparent
> + * huge pages in the future. Existing pages might be coalesced and
> + * new pages might be allocated as THP.
> + * MADV_NOHUGEPAGE - mark the given range as not worth being backed by
> + * transparent huge pages so the existing pages will not be
> + * coalesced into THP and new pages will not be allocated as THP.
> + * MADV_DONTDUMP - the application wants to prevent pages in the given range
> + * from being included in its core dump.
> + * MADV_DODUMP - cancel MADV_DONTDUMP: no longer exclude from core dump.
Just a trivial suggestion. How about adding brief descriptions for the
'MADV_COLD' and 'MADV_PAGEOUT' here, probably with another patch?
Thanks,
SeongJae Park
> + *
> + * return values:
> + * zero - success
> + * -EINVAL - start + len < 0, start is not page-aligned,
> + * "behavior" is not a valid value, or application
> + * is attempting to release locked or shared pages,
> + * or the specified address range includes file, Huge TLB,
> + * MAP_SHARED or VMPFNMAP range.
> + * -ENOMEM - addresses in the specified range are not currently
> + * mapped, or are outside the AS of the process.
> + * -EIO - an I/O error occurred while paging in data.
> + * -EBADF - map exists, but area maps something that isn't a file.
> + * -EAGAIN - a kernel resource was temporarily unavailable.
> + */
> +SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
> +{
> + return madvise_common(current, current->mm, start, len_in, behavior);
> +}
> --
> 2.25.0.rc1.283.g88dfdc4193-goog
^ permalink raw reply
* Re: [PATCH 2/4] mm: introduce external memory hinting API
From: SeongJae Park @ 2020-01-11 7:34 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, LKML, linux-mm, linux-api-u79uwXL29TY76Z2rM5mHXA,
oleksandr-H+wXaHxf7aLQT0dZR+AlfA, Suren Baghdasaryan, Tim Murray,
Daniel Colascione, Sandeep Patil, Sonny Rao, Brian Geffon,
Michal Hocko, Johannes Weiner, Shakeel Butt, John Dias
In-Reply-To: <20200110213433.94739-3-minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
On Fri, 10 Jan 2020 13:34:31 -0800 Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> There are usecases that System Management Software(SMS) want to give
> a memory hint to other processes because it's not known to the
> application. In the case of Android, ActivityManagerService daemon
> manges app's life cycle and that daemon must be able to initiate
> reclaim on its own without any app involvement.
>
> To solve the issue, this patch introduces new syscall process_madvise(2).
> It uses pidfd of an external processs to give the hint.
>
> int process_madvise(int pidfd, void *addr, size_t length, int advise,
> unsigned long flag);
>
> Since it could affect other process's address range, only privileged
> process(CAP_SYS_PTRACE) or something else(e.g., being the same UID)
> gives it the right to ptrace the process could use it successfully.
> The flag argument is reserved for future use if we need to extend the
> API.
>
> Supporting all hints madvise has/will supported/support to process_madvise
> is rather risky. Because we are not sure all hints make sense from external
> process and implementation for the hint may rely on the caller being
> in the current context so it could be error-prone. Thus, I just limited
> hints as MADV_[COLD|PAGEOUT] in this patch.
>
> If someone want to add other hints, we could hear hear the usecase and
> review it for each hint. It's more safe for maintainace rather than
> introducing a buggy syscall but hard to fix it later.
>
> Signed-off-by: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---
> arch/alpha/kernel/syscalls/syscall.tbl | 1 +
> arch/arm/tools/syscall.tbl | 1 +
> arch/arm64/include/asm/unistd.h | 2 +-
> arch/arm64/include/asm/unistd32.h | 2 +
> arch/ia64/kernel/syscalls/syscall.tbl | 1 +
> arch/m68k/kernel/syscalls/syscall.tbl | 1 +
> arch/microblaze/kernel/syscalls/syscall.tbl | 1 +
> arch/mips/kernel/syscalls/syscall_n32.tbl | 1 +
> arch/mips/kernel/syscalls/syscall_n64.tbl | 1 +
> arch/parisc/kernel/syscalls/syscall.tbl | 1 +
> arch/powerpc/kernel/syscalls/syscall.tbl | 1 +
> arch/s390/kernel/syscalls/syscall.tbl | 1 +
> arch/sh/kernel/syscalls/syscall.tbl | 1 +
> arch/sparc/kernel/syscalls/syscall.tbl | 1 +
> arch/x86/entry/syscalls/syscall_32.tbl | 1 +
> arch/x86/entry/syscalls/syscall_64.tbl | 1 +
> arch/xtensa/kernel/syscalls/syscall.tbl | 1 +
> include/linux/syscalls.h | 2 +
> include/uapi/asm-generic/unistd.h | 5 +-
> kernel/sys_ni.c | 1 +
> mm/madvise.c | 64 +++++++++++++++++++++
> 21 files changed, 89 insertions(+), 2 deletions(-)
>
> diff --git a/arch/alpha/kernel/syscalls/syscall.tbl b/arch/alpha/kernel/syscalls/syscall.tbl
> index e56950f23b49..776c61803315 100644
> --- a/arch/alpha/kernel/syscalls/syscall.tbl
> +++ b/arch/alpha/kernel/syscalls/syscall.tbl
> @@ -477,3 +477,4 @@
> # 545 reserved for clone3
> 546 common watch_devices sys_watch_devices
> 547 common openat2 sys_openat2
> +548 common process_madvise sys_process_madvise
> diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl
> index 7fb2f4d59210..a43381542276 100644
> --- a/arch/arm/tools/syscall.tbl
> +++ b/arch/arm/tools/syscall.tbl
> @@ -451,3 +451,4 @@
> 435 common clone3 sys_clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +438 common process_madvise sys_process_madvise
> diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
> index 8aa00ccb0b96..b722e47377a5 100644
> --- a/arch/arm64/include/asm/unistd.h
> +++ b/arch/arm64/include/asm/unistd.h
> @@ -38,7 +38,7 @@
> #define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE + 5)
> #define __ARM_NR_COMPAT_END (__ARM_NR_COMPAT_BASE + 0x800)
>
> -#define __NR_compat_syscalls 438
> +#define __NR_compat_syscalls 439
> #endif
>
> #define __ARCH_WANT_SYS_CLONE
> diff --git a/arch/arm64/include/asm/unistd32.h b/arch/arm64/include/asm/unistd32.h
> index 31f0ce25719e..5c82557d408f 100644
> --- a/arch/arm64/include/asm/unistd32.h
> +++ b/arch/arm64/include/asm/unistd32.h
> @@ -883,6 +883,8 @@ __SYSCALL(__NR_clone3, sys_clone3)
> __SYSCALL(__NR_watch_devices, sys_watch_devices)
> #define __NR_openat2 437
> __SYSCALL(__NR_openat2, sys_openat2)
> +#define __NR_openat2 438
Shouldn't this be '#define __NR_process_madvise 438'?
> +__SYSCALL(__NR_process_madvise, process_madvise)
>
> /*
> * Please add new compat syscalls above this comment and update
> diff --git a/arch/ia64/kernel/syscalls/syscall.tbl b/arch/ia64/kernel/syscalls/syscall.tbl
> index b9aa59931905..c156abc9a298 100644
> --- a/arch/ia64/kernel/syscalls/syscall.tbl
> +++ b/arch/ia64/kernel/syscalls/syscall.tbl
> @@ -358,3 +358,4 @@
> # 435 reserved for clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +438 common process_madvise sys_process_madvise
> diff --git a/arch/m68k/kernel/syscalls/syscall.tbl b/arch/m68k/kernel/syscalls/syscall.tbl
> index 868c1ef89d35..5b6034b6650f 100644
> --- a/arch/m68k/kernel/syscalls/syscall.tbl
> +++ b/arch/m68k/kernel/syscalls/syscall.tbl
> @@ -437,3 +437,4 @@
> # 435 reserved for clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +438 common process_madvise sys_process_madvise
> diff --git a/arch/microblaze/kernel/syscalls/syscall.tbl b/arch/microblaze/kernel/syscalls/syscall.tbl
> index 544b4cef18b3..4bef584af09c 100644
> --- a/arch/microblaze/kernel/syscalls/syscall.tbl
> +++ b/arch/microblaze/kernel/syscalls/syscall.tbl
> @@ -443,3 +443,4 @@
> 435 common clone3 sys_clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +438 common process_madvise sys_process_madvise
> diff --git a/arch/mips/kernel/syscalls/syscall_n32.tbl b/arch/mips/kernel/syscalls/syscall_n32.tbl
> index 05e8aee5dae7..94fbd0fcccce 100644
> --- a/arch/mips/kernel/syscalls/syscall_n32.tbl
> +++ b/arch/mips/kernel/syscalls/syscall_n32.tbl
> @@ -376,3 +376,4 @@
> 435 n32 clone3 __sys_clone3
> 436 n32 watch_devices sys_watch_devices
> 437 n32 openat2 sys_openat2
> +437 n32 process_madivse sys_process_madvise
Shouldn't the number for the 'process_madvise' be '438' instead of '437'?
> diff --git a/arch/mips/kernel/syscalls/syscall_n64.tbl b/arch/mips/kernel/syscalls/syscall_n64.tbl
> index 24d6c01328fb..4e6982c429d5 100644
> --- a/arch/mips/kernel/syscalls/syscall_n64.tbl
> +++ b/arch/mips/kernel/syscalls/syscall_n64.tbl
> @@ -352,3 +352,4 @@
> 435 n64 clone3 __sys_clone3
> 436 n64 watch_devices sys_watch_devices
> 437 n64 openat2 sys_openat2
> +437 n64 process_madvise sys_process_madvise
438? Same for below 5 changes.
> diff --git a/arch/parisc/kernel/syscalls/syscall.tbl b/arch/parisc/kernel/syscalls/syscall.tbl
> index 4b5f77a4e1a2..3aa990caf9dc 100644
> --- a/arch/parisc/kernel/syscalls/syscall.tbl
> +++ b/arch/parisc/kernel/syscalls/syscall.tbl
> @@ -435,3 +435,4 @@
> 435 common clone3 sys_clone3_wrapper
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +437 common process_madvise sys_process_madvise
> diff --git a/arch/powerpc/kernel/syscalls/syscall.tbl b/arch/powerpc/kernel/syscalls/syscall.tbl
> index 9716dc85a517..30e727a23f33 100644
> --- a/arch/powerpc/kernel/syscalls/syscall.tbl
> +++ b/arch/powerpc/kernel/syscalls/syscall.tbl
> @@ -519,3 +519,4 @@
> 435 nospu clone3 ppc_clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +437 common process_madvise sys_process_madvise
> diff --git a/arch/s390/kernel/syscalls/syscall.tbl b/arch/s390/kernel/syscalls/syscall.tbl
> index 7da330f8b03e..75722e5ff496 100644
> --- a/arch/s390/kernel/syscalls/syscall.tbl
> +++ b/arch/s390/kernel/syscalls/syscall.tbl
> @@ -440,3 +440,4 @@
> 435 common clone3 sys_clone3 sys_clone3
> 436 common watch_devices sys_watch_devices sys_watch_devices
> 437 common openat2 sys_openat2 sys_openat2
> +437 common process_madvise sys_process_madvise sys_process_madvise
> diff --git a/arch/sh/kernel/syscalls/syscall.tbl b/arch/sh/kernel/syscalls/syscall.tbl
> index bb7e68e25337..7d7bc7befad3 100644
> --- a/arch/sh/kernel/syscalls/syscall.tbl
> +++ b/arch/sh/kernel/syscalls/syscall.tbl
> @@ -440,3 +440,4 @@
> # 435 reserved for clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +437 common process_madvise sys_process_madvise
> diff --git a/arch/sparc/kernel/syscalls/syscall.tbl b/arch/sparc/kernel/syscalls/syscall.tbl
> index 646a1fad7218..581d331ff62f 100644
> --- a/arch/sparc/kernel/syscalls/syscall.tbl
> +++ b/arch/sparc/kernel/syscalls/syscall.tbl
> @@ -483,3 +483,4 @@
> # 435 reserved for clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +437 common process_madvise sys_process_madvise
> diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl
> index 57c53acee290..76a2c266fe7e 100644
> --- a/arch/x86/entry/syscalls/syscall_32.tbl
> +++ b/arch/x86/entry/syscalls/syscall_32.tbl
> @@ -442,3 +442,4 @@
> 435 i386 clone3 sys_clone3 __ia32_sys_clone3
> 436 i386 watch_devices sys_watch_devices __ia32_sys_watch_devices
> 437 i386 openat2 sys_openat2 __ia32_sys_openat2
> +438 i386 process_madvise sys_process_madvise __ia32_sys_process_madvise
> diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl
> index 1dd8d21f6500..b697cd8620cb 100644
> --- a/arch/x86/entry/syscalls/syscall_64.tbl
> +++ b/arch/x86/entry/syscalls/syscall_64.tbl
> @@ -359,6 +359,7 @@
> 435 common clone3 __x64_sys_clone3/ptregs
> 436 common watch_devices __x64_sys_watch_devices
> 437 common openat2 __x64_sys_openat2
> +438 common process_madvise __x64_sys_process_madvise
>
> #
> # x32-specific system call numbers start at 512 to avoid cache impact
> diff --git a/arch/xtensa/kernel/syscalls/syscall.tbl b/arch/xtensa/kernel/syscalls/syscall.tbl
> index 0f48ab7bd75b..2e9813ecfd7d 100644
> --- a/arch/xtensa/kernel/syscalls/syscall.tbl
> +++ b/arch/xtensa/kernel/syscalls/syscall.tbl
> @@ -408,3 +408,4 @@
> 435 common clone3 sys_clone3
> 436 common watch_devices sys_watch_devices
> 437 common openat2 sys_openat2
> +438 common process_madvise sys_process_madvise
> diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
> index 433c8c85636e..1b58a11ff49f 100644
> --- a/include/linux/syscalls.h
> +++ b/include/linux/syscalls.h
> @@ -877,6 +877,8 @@ asmlinkage long sys_munlockall(void);
> asmlinkage long sys_mincore(unsigned long start, size_t len,
> unsigned char __user * vec);
> asmlinkage long sys_madvise(unsigned long start, size_t len, int behavior);
> +asmlinkage long sys_process_madvise(int pidfd, unsigned long start,
> + size_t len, int behavior, unsigned long flags);
> asmlinkage long sys_remap_file_pages(unsigned long start, unsigned long size,
> unsigned long prot, unsigned long pgoff,
> unsigned long flags);
> diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
> index 33f3856a9c3c..4bcd8d366f38 100644
> --- a/include/uapi/asm-generic/unistd.h
> +++ b/include/uapi/asm-generic/unistd.h
> @@ -856,8 +856,11 @@ __SYSCALL(__NR_watch_devices, sys_watch_devices)
> #define __NR_openat2 437
> __SYSCALL(__NR_openat2, sys_openat2)
>
> +#define __NR_openat2 438
Shouldn't this be '#define __NR_process_madvise 438'?
Thanks,
SeongJae Park
> +__SYSCALL(__NR_process_madvise, sys_process_madvise)
> +
> #undef __NR_syscalls
> -#define __NR_syscalls 438
> +#define __NR_syscalls 439
>
> /*
> * 32 bit systems traditionally used different
> diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
> index 0e9b275260f8..10ce5eac8b4b 100644
> --- a/kernel/sys_ni.c
> +++ b/kernel/sys_ni.c
> @@ -281,6 +281,7 @@ COND_SYSCALL(mlockall);
> COND_SYSCALL(munlockall);
> COND_SYSCALL(mincore);
> COND_SYSCALL(madvise);
> +COND_SYSCALL(process_madvise);
> COND_SYSCALL(remap_file_pages);
> COND_SYSCALL(mbind);
> COND_SYSCALL_COMPAT(mbind);
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 0c901de531e4..e15dfb4df7bf 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -17,6 +17,7 @@
> #include <linux/falloc.h>
> #include <linux/fadvise.h>
> #include <linux/sched.h>
> +#include <linux/sched/mm.h>
> #include <linux/ksm.h>
> #include <linux/fs.h>
> #include <linux/file.h>
> @@ -993,6 +994,18 @@ madvise_behavior_valid(int behavior)
> }
> }
>
> +static bool
> +process_madvise_behavior_valid(int behavior)
> +{
> + switch (behavior) {
> + case MADV_COLD:
> + case MADV_PAGEOUT:
> + return true;
> + default:
> + return false;
> + }
> +}
> +
> /*
> * madvise_common - request behavior hint to address range of the target process
> *
> @@ -1169,3 +1182,54 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
> {
> return madvise_common(current, current->mm, start, len_in, behavior);
> }
> +
> +SYSCALL_DEFINE5(process_madvise, int, pidfd, unsigned long, start,
> + size_t, len_in, int, behavior, unsigned long, flags)
> +{
> + int ret;
> + struct fd f;
> + struct pid *pid;
> + struct task_struct *task;
> + struct mm_struct *mm;
> +
> + if (flags != 0)
> + return -EINVAL;
> +
> + if (!process_madvise_behavior_valid(behavior))
> + return -EINVAL;
> +
> + f = fdget(pidfd);
> + if (!f.file)
> + return -EBADF;
> +
> + pid = pidfd_pid(f.file);
> + if (IS_ERR(pid)) {
> + ret = PTR_ERR(pid);
> + goto err;
> + }
> +
> + rcu_read_lock();
> + task = pid_task(pid, PIDTYPE_PID);
> + if (!task) {
> + rcu_read_unlock();
> + ret = -ESRCH;
> + goto err;
> + }
> +
> + get_task_struct(task);
> + rcu_read_unlock();
> +
> + mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS);
> + if (IS_ERR_OR_NULL(mm)) {
> + ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH;
> + goto release_task;
> + }
> +
> + ret = madvise_common(task, mm, start, len_in, behavior);
> + mmput(mm);
> +release_task:
> + put_task_struct(task);
> +err:
> + fdput(f);
> + return ret;
> +}
> --
> 2.25.0.rc1.283.g88dfdc4193-goog
>
^ permalink raw reply
* Re: [PATCH v27 04/12] LRNG - add switchable DRNG support
From: kbuild test robot @ 2020-01-11 7:09 UTC (permalink / raw)
To: Stephan Müller
Cc: kbuild-all, Arnd Bergmann, Greg Kroah-Hartman, linux-crypto, LKML,
linux-api, Eric W. Biederman, Alexander E. Patrakov,
Ahmed S. Darwish, Theodore Y. Ts'o, Willy Tarreau,
Matthew Garrett, Vito Caputo, Andreas Dilger, Jan Kara,
Ray Strode, William Jon McCann, zhangjs, Andy Lutomirski,
Florian Weimer, Lennart Poettering, Ni
In-Reply-To: <5087131.2PHHu6SUIE@positron.chronox.de>
Hi "Stephan,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on char-misc/char-misc-testing]
[also build test WARNING on cryptodev/master crypto/master v5.5-rc5 next-20200110]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Stephan-M-ller/dev-random-a-new-approach-with-full-SP800-90B/20200110-084934
base: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 68faa679b8be1a74e6663c21c3a9d25d32f1c079
reproduce:
# apt-get install sparse
# sparse version: v0.6.1-129-g341daf20-dirty
make ARCH=x86_64 allmodconfig
make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'
If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>
sparse warnings: (new ones prefixed by >>)
>> drivers/char/lrng/lrng_internal.h:239:39: sparse: sparse: context imbalance in 'lrng_drng_switch' - unexpected unlock
vim +/lrng_drng_switch +239 drivers/char/lrng/lrng_internal.h
58c283819a1e87 Stephan Müller 2020-01-09 233
58c283819a1e87 Stephan Müller 2020-01-09 234 /* Unlock the DRNG */
58c283819a1e87 Stephan Müller 2020-01-09 235 static __always_inline void lrng_drng_unlock(struct lrng_drng *drng,
58c283819a1e87 Stephan Müller 2020-01-09 236 unsigned long *flags)
58c283819a1e87 Stephan Müller 2020-01-09 237 {
58c283819a1e87 Stephan Müller 2020-01-09 238 if (lrng_drng_is_atomic(drng))
58c283819a1e87 Stephan Müller 2020-01-09 @239 spin_unlock_irqrestore(&drng->spin_lock, *flags);
58c283819a1e87 Stephan Müller 2020-01-09 240 else
58c283819a1e87 Stephan Müller 2020-01-09 241 mutex_unlock(&drng->lock);
58c283819a1e87 Stephan Müller 2020-01-09 242 }
58c283819a1e87 Stephan Müller 2020-01-09 243
:::::: The code at line 239 was first introduced by commit
:::::: 58c283819a1e879bc2e30d05720285f9709f7f6d Linux Random Number Generator
:::::: TO: Stephan Müller <smueller@chronox.de>
:::::: CC: 0day robot <lkp@intel.com>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org Intel Corporation
^ permalink raw reply
* Re: [PATCH RFC 0/1] mount: universally disallow mounting over symlinks
From: Al Viro @ 2020-01-10 23:19 UTC (permalink / raw)
To: Aleksa Sarai
Cc: David Howells, Eric Biederman, Linus Torvalds, stable,
Christian Brauner, Serge Hallyn, dev, containers, linux-api,
linux-fsdevel, linux-kernel, Ian Kent
In-Reply-To: <20200103014901.GC8904@ZenIV.linux.org.uk>
On Fri, Jan 03, 2020 at 01:49:01AM +0000, Al Viro wrote:
> On Thu, Jan 02, 2020 at 02:59:20PM +1100, Aleksa Sarai wrote:
> > On 2020-01-01, Al Viro <viro@zeniv.linux.org.uk> wrote:
> > > On Thu, Jan 02, 2020 at 01:44:07AM +1100, Aleksa Sarai wrote:
> > >
> > > > Thanks, this fixes the issue for me (and also fixes another reproducer I
> > > > found -- mounting a symlink on top of itself then trying to umount it).
> > > >
> > > > Reported-by: Aleksa Sarai <cyphar@cyphar.com>
> > > > Tested-by: Aleksa Sarai <cyphar@cyphar.com>
> > >
> > > Pushed into #fixes.
> >
> > Thanks. One other thing I noticed is that umount applies to the
> > underlying symlink rather than the mountpoint on top. So, for example
> > (using the same scripts I posted in the thread):
> >
> > # ln -s /tmp/foo link
> > # ./mount_to_symlink /etc/passwd link
> > # umount -l link # will attempt to unmount "/tmp/foo"
> >
> > Is that intentional?
>
> It's a mess, again in mountpoint_last(). FWIW, at some point I proposed
> to have nd_jump_link() to fail with -ELOOP if the target was a symlink;
> Linus asked for reasons deeper than my dislike of the semantics, I looked
> around and hadn't spotted anything. And there hadn't been at the time,
> but when four months later umount_lookup_last() went in I failed to look
> for that source of potential problems in it ;-/
FWIW, since Ian appears to agree that we want ->d_manage() on the mount
crossing at the end of umount(2) lookup, here's a much simpler solution -
kill mountpoint_last() and switch to using lookup_last(). As a side
benefit, LOOKUP_NO_REVAL also goes away. It's possible to trim the
things even more (path_mountpoint() is very similar to path_lookupat()
at that point, and it's not hard to make the differences conditional on
something like LOOKUP_UMOUNT); I would rather do that part in the
cleanups series - the one below is easier to backport.
Aleksa, Ian - could you see if the patch below works for you?
commit e56b43b971a7c08762fceab330a52b7245041dbc
Author: Al Viro <viro@zeniv.linux.org.uk>
Date: Fri Jan 10 17:17:19 2020 -0500
reimplement path_mountpoint() with less magic
... and get rid of a bunch of bugs in it. Background:
the reason for path_mountpoint() is that umount() really doesn't
want attempts to revalidate the root of what it's trying to umount.
The thing we want to avoid actually happen from complete_walk();
solution was to do something parallel to normal path_lookupat()
and it both went overboard and got the boilerplate subtly
(and not so subtly) wrong.
A better solution is to do pretty much what the normal path_lookupat()
does, but instead of complete_walk() do unlazy_walk(). All it takes
to avoid that ->d_weak_revalidate() call... mountpoint_last() goes
away, along with everything it got wrong, and so does the magic around
LOOKUP_NO_REVAL.
Another source of bugs is that when we traverse mounts at the final
location (and we need to do that - umount . expects to get whatever's
overmounting ., if any, out of the lookup) we really ought to take
care of ->d_manage() - as it is, manual umount of autofs automount
in progress can lead to unpleasant surprises for the daemon. Easily
solved by using handle_lookup_down() instead of follow_mount().
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
diff --git a/fs/namei.c b/fs/namei.c
index d6c91d1e88cb..1793661c3342 100644
--- a/fs/namei.c
+++ b/fs/namei.c
@@ -1649,17 +1649,15 @@ static struct dentry *__lookup_slow(const struct qstr *name,
if (IS_ERR(dentry))
return dentry;
if (unlikely(!d_in_lookup(dentry))) {
- if (!(flags & LOOKUP_NO_REVAL)) {
- int error = d_revalidate(dentry, flags);
- if (unlikely(error <= 0)) {
- if (!error) {
- d_invalidate(dentry);
- dput(dentry);
- goto again;
- }
+ int error = d_revalidate(dentry, flags);
+ if (unlikely(error <= 0)) {
+ if (!error) {
+ d_invalidate(dentry);
dput(dentry);
- dentry = ERR_PTR(error);
+ goto again;
}
+ dput(dentry);
+ dentry = ERR_PTR(error);
}
} else {
old = inode->i_op->lookup(inode, dentry, flags);
@@ -2618,72 +2616,6 @@ int user_path_at_empty(int dfd, const char __user *name, unsigned flags,
EXPORT_SYMBOL(user_path_at_empty);
/**
- * mountpoint_last - look up last component for umount
- * @nd: pathwalk nameidata - currently pointing at parent directory of "last"
- *
- * This is a special lookup_last function just for umount. In this case, we
- * need to resolve the path without doing any revalidation.
- *
- * The nameidata should be the result of doing a LOOKUP_PARENT pathwalk. Since
- * mountpoints are always pinned in the dcache, their ancestors are too. Thus,
- * in almost all cases, this lookup will be served out of the dcache. The only
- * cases where it won't are if nd->last refers to a symlink or the path is
- * bogus and it doesn't exist.
- *
- * Returns:
- * -error: if there was an error during lookup. This includes -ENOENT if the
- * lookup found a negative dentry.
- *
- * 0: if we successfully resolved nd->last and found it to not to be a
- * symlink that needs to be followed.
- *
- * 1: if we successfully resolved nd->last and found it to be a symlink
- * that needs to be followed.
- */
-static int
-mountpoint_last(struct nameidata *nd)
-{
- int error = 0;
- struct dentry *dir = nd->path.dentry;
- struct path path;
-
- /* If we're in rcuwalk, drop out of it to handle last component */
- if (nd->flags & LOOKUP_RCU) {
- if (unlazy_walk(nd))
- return -ECHILD;
- }
-
- nd->flags &= ~LOOKUP_PARENT;
-
- if (unlikely(nd->last_type != LAST_NORM)) {
- error = handle_dots(nd, nd->last_type);
- if (error)
- return error;
- path.dentry = dget(nd->path.dentry);
- } else {
- path.dentry = d_lookup(dir, &nd->last);
- if (!path.dentry) {
- /*
- * No cached dentry. Mounted dentries are pinned in the
- * cache, so that means that this dentry is probably
- * a symlink or the path doesn't actually point
- * to a mounted dentry.
- */
- path.dentry = lookup_slow(&nd->last, dir,
- nd->flags | LOOKUP_NO_REVAL);
- if (IS_ERR(path.dentry))
- return PTR_ERR(path.dentry);
- }
- }
- if (d_flags_negative(smp_load_acquire(&path.dentry->d_flags))) {
- dput(path.dentry);
- return -ENOENT;
- }
- path.mnt = nd->path.mnt;
- return step_into(nd, &path, 0, d_backing_inode(path.dentry), 0);
-}
-
-/**
* path_mountpoint - look up a path to be umounted
* @nd: lookup context
* @flags: lookup flags
@@ -2699,14 +2631,17 @@ path_mountpoint(struct nameidata *nd, unsigned flags, struct path *path)
int err;
while (!(err = link_path_walk(s, nd)) &&
- (err = mountpoint_last(nd)) > 0) {
+ (err = lookup_last(nd)) > 0) {
s = trailing_symlink(nd);
}
+ if (!err)
+ err = unlazy_walk(nd);
+ if (!err)
+ err = handle_lookup_down(nd);
if (!err) {
*path = nd->path;
nd->path.mnt = NULL;
nd->path.dentry = NULL;
- follow_mount(path);
}
terminate_walk(nd);
return err;
diff --git a/fs/nfs/nfstrace.h b/fs/nfs/nfstrace.h
index f64a33d2a1d1..2a82dcce5fc1 100644
--- a/fs/nfs/nfstrace.h
+++ b/fs/nfs/nfstrace.h
@@ -206,7 +206,6 @@ TRACE_DEFINE_ENUM(LOOKUP_AUTOMOUNT);
TRACE_DEFINE_ENUM(LOOKUP_PARENT);
TRACE_DEFINE_ENUM(LOOKUP_REVAL);
TRACE_DEFINE_ENUM(LOOKUP_RCU);
-TRACE_DEFINE_ENUM(LOOKUP_NO_REVAL);
TRACE_DEFINE_ENUM(LOOKUP_OPEN);
TRACE_DEFINE_ENUM(LOOKUP_CREATE);
TRACE_DEFINE_ENUM(LOOKUP_EXCL);
@@ -224,7 +223,6 @@ TRACE_DEFINE_ENUM(LOOKUP_DOWN);
{ LOOKUP_PARENT, "PARENT" }, \
{ LOOKUP_REVAL, "REVAL" }, \
{ LOOKUP_RCU, "RCU" }, \
- { LOOKUP_NO_REVAL, "NO_REVAL" }, \
{ LOOKUP_OPEN, "OPEN" }, \
{ LOOKUP_CREATE, "CREATE" }, \
{ LOOKUP_EXCL, "EXCL" }, \
diff --git a/include/linux/namei.h b/include/linux/namei.h
index 7fe7b87a3ded..07bfb0874033 100644
--- a/include/linux/namei.h
+++ b/include/linux/namei.h
@@ -34,7 +34,6 @@ enum {LAST_NORM, LAST_ROOT, LAST_DOT, LAST_DOTDOT, LAST_BIND};
/* internal use only */
#define LOOKUP_PARENT 0x0010
-#define LOOKUP_NO_REVAL 0x0080
#define LOOKUP_JUMPED 0x1000
#define LOOKUP_ROOT 0x2000
#define LOOKUP_ROOT_GRABBED 0x0008
^ permalink raw reply related
* [PATCH 4/4] mm/madvise: allow KSM hints for remote API
From: Minchan Kim @ 2020-01-10 21:34 UTC (permalink / raw)
To: Andrew Morton
Cc: LKML, linux-mm, linux-api-u79uwXL29TY76Z2rM5mHXA,
oleksandr-H+wXaHxf7aLQT0dZR+AlfA, Suren Baghdasaryan, Tim Murray,
Daniel Colascione, Sandeep Patil, Sonny Rao, Brian Geffon,
Michal Hocko, Johannes Weiner, Shakeel Butt, John Dias,
Minchan Kim
In-Reply-To: <20200110213433.94739-1-minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
From: Oleksandr Natalenko <oleksandr-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
It all began with the fact that KSM works only on memory that is marked
by madvise(). And the only way to get around that is to either:
* use LD_PRELOAD; or
* patch the kernel with something like UKSM or PKSM.
(i skip ptrace can of worms here intentionally)
To overcome this restriction, lets employ a new remote madvise API. This
can be used by some small userspace helper daemon that will do auto-KSM
job for us.
I think of two major consumers of remote KSM hints:
* hosts, that run containers, especially similar ones and especially in
a trusted environment, sharing the same runtime like Node.js;
* heavy applications, that can be run in multiple instances, not
limited to opensource ones like Firefox, but also those that cannot be
modified since they are binary-only and, maybe, statically linked.
Speaking of statistics, more numbers can be found in the very first
submission, that is related to this one [1]. For my current setup with
two Firefox instances I get 100 to 200 MiB saved for the second instance
depending on the amount of tabs.
1 FF instance with 15 tabs:
$ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
410
2 FF instances, second one has 12 tabs (all the tabs are different):
$ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
592
At the very moment I do not have specific numbers for containerised
workload, but those should be comparable in case the containers share
similar/same runtime.
[1] https://lore.kernel.org/patchwork/patch/1012142/
Signed-off-by: Oleksandr Natalenko <oleksandr-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
mm/madvise.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/mm/madvise.c b/mm/madvise.c
index eb42b2b7f49b..3aa9aec6bfd9 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -1000,6 +1000,8 @@ process_madvise_behavior_valid(int behavior)
switch (behavior) {
case MADV_COLD:
case MADV_PAGEOUT:
+ case MADV_MERGEABLE:
+ case MADV_UNMERGEABLE:
return true;
default:
return false;
--
2.25.0.rc1.283.g88dfdc4193-goog
^ permalink raw reply related
* [PATCH 3/4] mm/madvise: employ mmget_still_valid for write lock
From: Minchan Kim @ 2020-01-10 21:34 UTC (permalink / raw)
To: Andrew Morton
Cc: LKML, linux-mm, linux-api, oleksandr, Suren Baghdasaryan,
Tim Murray, Daniel Colascione, Sandeep Patil, Sonny Rao,
Brian Geffon, Michal Hocko, Johannes Weiner, Shakeel Butt,
John Dias, Minchan Kim
In-Reply-To: <20200110213433.94739-1-minchan@kernel.org>
From: Oleksandr Natalenko <oleksandr@redhat.com>
Do the very same trick as we already do since 04f5866e41fb. KSM hints
will require locking mmap_sem for write since they modify vm_flags, so
for remote KSM hinting this additional check is needed.
Signed-off-by: Oleksandr Natalenko <oleksandr@redhat.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
mm/madvise.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/mm/madvise.c b/mm/madvise.c
index e15dfb4df7bf..eb42b2b7f49b 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -1059,6 +1059,8 @@ static int madvise_common(struct task_struct *task, struct mm_struct *mm,
if (write) {
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
+ if (current->mm != mm && !mmget_still_valid(mm))
+ goto skip_mm;
} else {
down_read(&mm->mmap_sem);
}
@@ -1109,6 +1111,7 @@ static int madvise_common(struct task_struct *task, struct mm_struct *mm,
}
out:
blk_finish_plug(&plug);
+skip_mm:
if (write)
up_write(&mm->mmap_sem);
else
--
2.25.0.rc1.283.g88dfdc4193-goog
^ permalink raw reply related
* [PATCH 2/4] mm: introduce external memory hinting API
From: Minchan Kim @ 2020-01-10 21:34 UTC (permalink / raw)
To: Andrew Morton
Cc: LKML, linux-mm, linux-api, oleksandr, Suren Baghdasaryan,
Tim Murray, Daniel Colascione, Sandeep Patil, Sonny Rao,
Brian Geffon, Michal Hocko, Johannes Weiner, Shakeel Butt,
John Dias, Minchan Kim
In-Reply-To: <20200110213433.94739-1-minchan@kernel.org>
There are usecases that System Management Software(SMS) want to give
a memory hint to other processes because it's not known to the
application. In the case of Android, ActivityManagerService daemon
manges app's life cycle and that daemon must be able to initiate
reclaim on its own without any app involvement.
To solve the issue, this patch introduces new syscall process_madvise(2).
It uses pidfd of an external processs to give the hint.
int process_madvise(int pidfd, void *addr, size_t length, int advise,
unsigned long flag);
Since it could affect other process's address range, only privileged
process(CAP_SYS_PTRACE) or something else(e.g., being the same UID)
gives it the right to ptrace the process could use it successfully.
The flag argument is reserved for future use if we need to extend the
API.
Supporting all hints madvise has/will supported/support to process_madvise
is rather risky. Because we are not sure all hints make sense from external
process and implementation for the hint may rely on the caller being
in the current context so it could be error-prone. Thus, I just limited
hints as MADV_[COLD|PAGEOUT] in this patch.
If someone want to add other hints, we could hear hear the usecase and
review it for each hint. It's more safe for maintainace rather than
introducing a buggy syscall but hard to fix it later.
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
arch/alpha/kernel/syscalls/syscall.tbl | 1 +
arch/arm/tools/syscall.tbl | 1 +
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 2 +
arch/ia64/kernel/syscalls/syscall.tbl | 1 +
arch/m68k/kernel/syscalls/syscall.tbl | 1 +
arch/microblaze/kernel/syscalls/syscall.tbl | 1 +
arch/mips/kernel/syscalls/syscall_n32.tbl | 1 +
arch/mips/kernel/syscalls/syscall_n64.tbl | 1 +
arch/parisc/kernel/syscalls/syscall.tbl | 1 +
arch/powerpc/kernel/syscalls/syscall.tbl | 1 +
arch/s390/kernel/syscalls/syscall.tbl | 1 +
arch/sh/kernel/syscalls/syscall.tbl | 1 +
arch/sparc/kernel/syscalls/syscall.tbl | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/xtensa/kernel/syscalls/syscall.tbl | 1 +
include/linux/syscalls.h | 2 +
include/uapi/asm-generic/unistd.h | 5 +-
kernel/sys_ni.c | 1 +
mm/madvise.c | 64 +++++++++++++++++++++
21 files changed, 89 insertions(+), 2 deletions(-)
diff --git a/arch/alpha/kernel/syscalls/syscall.tbl b/arch/alpha/kernel/syscalls/syscall.tbl
index e56950f23b49..776c61803315 100644
--- a/arch/alpha/kernel/syscalls/syscall.tbl
+++ b/arch/alpha/kernel/syscalls/syscall.tbl
@@ -477,3 +477,4 @@
# 545 reserved for clone3
546 common watch_devices sys_watch_devices
547 common openat2 sys_openat2
+548 common process_madvise sys_process_madvise
diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl
index 7fb2f4d59210..a43381542276 100644
--- a/arch/arm/tools/syscall.tbl
+++ b/arch/arm/tools/syscall.tbl
@@ -451,3 +451,4 @@
435 common clone3 sys_clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+438 common process_madvise sys_process_madvise
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index 8aa00ccb0b96..b722e47377a5 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -38,7 +38,7 @@
#define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE + 5)
#define __ARM_NR_COMPAT_END (__ARM_NR_COMPAT_BASE + 0x800)
-#define __NR_compat_syscalls 438
+#define __NR_compat_syscalls 439
#endif
#define __ARCH_WANT_SYS_CLONE
diff --git a/arch/arm64/include/asm/unistd32.h b/arch/arm64/include/asm/unistd32.h
index 31f0ce25719e..5c82557d408f 100644
--- a/arch/arm64/include/asm/unistd32.h
+++ b/arch/arm64/include/asm/unistd32.h
@@ -883,6 +883,8 @@ __SYSCALL(__NR_clone3, sys_clone3)
__SYSCALL(__NR_watch_devices, sys_watch_devices)
#define __NR_openat2 437
__SYSCALL(__NR_openat2, sys_openat2)
+#define __NR_openat2 438
+__SYSCALL(__NR_process_madvise, process_madvise)
/*
* Please add new compat syscalls above this comment and update
diff --git a/arch/ia64/kernel/syscalls/syscall.tbl b/arch/ia64/kernel/syscalls/syscall.tbl
index b9aa59931905..c156abc9a298 100644
--- a/arch/ia64/kernel/syscalls/syscall.tbl
+++ b/arch/ia64/kernel/syscalls/syscall.tbl
@@ -358,3 +358,4 @@
# 435 reserved for clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+438 common process_madvise sys_process_madvise
diff --git a/arch/m68k/kernel/syscalls/syscall.tbl b/arch/m68k/kernel/syscalls/syscall.tbl
index 868c1ef89d35..5b6034b6650f 100644
--- a/arch/m68k/kernel/syscalls/syscall.tbl
+++ b/arch/m68k/kernel/syscalls/syscall.tbl
@@ -437,3 +437,4 @@
# 435 reserved for clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+438 common process_madvise sys_process_madvise
diff --git a/arch/microblaze/kernel/syscalls/syscall.tbl b/arch/microblaze/kernel/syscalls/syscall.tbl
index 544b4cef18b3..4bef584af09c 100644
--- a/arch/microblaze/kernel/syscalls/syscall.tbl
+++ b/arch/microblaze/kernel/syscalls/syscall.tbl
@@ -443,3 +443,4 @@
435 common clone3 sys_clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+438 common process_madvise sys_process_madvise
diff --git a/arch/mips/kernel/syscalls/syscall_n32.tbl b/arch/mips/kernel/syscalls/syscall_n32.tbl
index 05e8aee5dae7..94fbd0fcccce 100644
--- a/arch/mips/kernel/syscalls/syscall_n32.tbl
+++ b/arch/mips/kernel/syscalls/syscall_n32.tbl
@@ -376,3 +376,4 @@
435 n32 clone3 __sys_clone3
436 n32 watch_devices sys_watch_devices
437 n32 openat2 sys_openat2
+437 n32 process_madivse sys_process_madvise
diff --git a/arch/mips/kernel/syscalls/syscall_n64.tbl b/arch/mips/kernel/syscalls/syscall_n64.tbl
index 24d6c01328fb..4e6982c429d5 100644
--- a/arch/mips/kernel/syscalls/syscall_n64.tbl
+++ b/arch/mips/kernel/syscalls/syscall_n64.tbl
@@ -352,3 +352,4 @@
435 n64 clone3 __sys_clone3
436 n64 watch_devices sys_watch_devices
437 n64 openat2 sys_openat2
+437 n64 process_madvise sys_process_madvise
diff --git a/arch/parisc/kernel/syscalls/syscall.tbl b/arch/parisc/kernel/syscalls/syscall.tbl
index 4b5f77a4e1a2..3aa990caf9dc 100644
--- a/arch/parisc/kernel/syscalls/syscall.tbl
+++ b/arch/parisc/kernel/syscalls/syscall.tbl
@@ -435,3 +435,4 @@
435 common clone3 sys_clone3_wrapper
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+437 common process_madvise sys_process_madvise
diff --git a/arch/powerpc/kernel/syscalls/syscall.tbl b/arch/powerpc/kernel/syscalls/syscall.tbl
index 9716dc85a517..30e727a23f33 100644
--- a/arch/powerpc/kernel/syscalls/syscall.tbl
+++ b/arch/powerpc/kernel/syscalls/syscall.tbl
@@ -519,3 +519,4 @@
435 nospu clone3 ppc_clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+437 common process_madvise sys_process_madvise
diff --git a/arch/s390/kernel/syscalls/syscall.tbl b/arch/s390/kernel/syscalls/syscall.tbl
index 7da330f8b03e..75722e5ff496 100644
--- a/arch/s390/kernel/syscalls/syscall.tbl
+++ b/arch/s390/kernel/syscalls/syscall.tbl
@@ -440,3 +440,4 @@
435 common clone3 sys_clone3 sys_clone3
436 common watch_devices sys_watch_devices sys_watch_devices
437 common openat2 sys_openat2 sys_openat2
+437 common process_madvise sys_process_madvise sys_process_madvise
diff --git a/arch/sh/kernel/syscalls/syscall.tbl b/arch/sh/kernel/syscalls/syscall.tbl
index bb7e68e25337..7d7bc7befad3 100644
--- a/arch/sh/kernel/syscalls/syscall.tbl
+++ b/arch/sh/kernel/syscalls/syscall.tbl
@@ -440,3 +440,4 @@
# 435 reserved for clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+437 common process_madvise sys_process_madvise
diff --git a/arch/sparc/kernel/syscalls/syscall.tbl b/arch/sparc/kernel/syscalls/syscall.tbl
index 646a1fad7218..581d331ff62f 100644
--- a/arch/sparc/kernel/syscalls/syscall.tbl
+++ b/arch/sparc/kernel/syscalls/syscall.tbl
@@ -483,3 +483,4 @@
# 435 reserved for clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+437 common process_madvise sys_process_madvise
diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl
index 57c53acee290..76a2c266fe7e 100644
--- a/arch/x86/entry/syscalls/syscall_32.tbl
+++ b/arch/x86/entry/syscalls/syscall_32.tbl
@@ -442,3 +442,4 @@
435 i386 clone3 sys_clone3 __ia32_sys_clone3
436 i386 watch_devices sys_watch_devices __ia32_sys_watch_devices
437 i386 openat2 sys_openat2 __ia32_sys_openat2
+438 i386 process_madvise sys_process_madvise __ia32_sys_process_madvise
diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl
index 1dd8d21f6500..b697cd8620cb 100644
--- a/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/arch/x86/entry/syscalls/syscall_64.tbl
@@ -359,6 +359,7 @@
435 common clone3 __x64_sys_clone3/ptregs
436 common watch_devices __x64_sys_watch_devices
437 common openat2 __x64_sys_openat2
+438 common process_madvise __x64_sys_process_madvise
#
# x32-specific system call numbers start at 512 to avoid cache impact
diff --git a/arch/xtensa/kernel/syscalls/syscall.tbl b/arch/xtensa/kernel/syscalls/syscall.tbl
index 0f48ab7bd75b..2e9813ecfd7d 100644
--- a/arch/xtensa/kernel/syscalls/syscall.tbl
+++ b/arch/xtensa/kernel/syscalls/syscall.tbl
@@ -408,3 +408,4 @@
435 common clone3 sys_clone3
436 common watch_devices sys_watch_devices
437 common openat2 sys_openat2
+438 common process_madvise sys_process_madvise
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 433c8c85636e..1b58a11ff49f 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -877,6 +877,8 @@ asmlinkage long sys_munlockall(void);
asmlinkage long sys_mincore(unsigned long start, size_t len,
unsigned char __user * vec);
asmlinkage long sys_madvise(unsigned long start, size_t len, int behavior);
+asmlinkage long sys_process_madvise(int pidfd, unsigned long start,
+ size_t len, int behavior, unsigned long flags);
asmlinkage long sys_remap_file_pages(unsigned long start, unsigned long size,
unsigned long prot, unsigned long pgoff,
unsigned long flags);
diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
index 33f3856a9c3c..4bcd8d366f38 100644
--- a/include/uapi/asm-generic/unistd.h
+++ b/include/uapi/asm-generic/unistd.h
@@ -856,8 +856,11 @@ __SYSCALL(__NR_watch_devices, sys_watch_devices)
#define __NR_openat2 437
__SYSCALL(__NR_openat2, sys_openat2)
+#define __NR_openat2 438
+__SYSCALL(__NR_process_madvise, sys_process_madvise)
+
#undef __NR_syscalls
-#define __NR_syscalls 438
+#define __NR_syscalls 439
/*
* 32 bit systems traditionally used different
diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
index 0e9b275260f8..10ce5eac8b4b 100644
--- a/kernel/sys_ni.c
+++ b/kernel/sys_ni.c
@@ -281,6 +281,7 @@ COND_SYSCALL(mlockall);
COND_SYSCALL(munlockall);
COND_SYSCALL(mincore);
COND_SYSCALL(madvise);
+COND_SYSCALL(process_madvise);
COND_SYSCALL(remap_file_pages);
COND_SYSCALL(mbind);
COND_SYSCALL_COMPAT(mbind);
diff --git a/mm/madvise.c b/mm/madvise.c
index 0c901de531e4..e15dfb4df7bf 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -17,6 +17,7 @@
#include <linux/falloc.h>
#include <linux/fadvise.h>
#include <linux/sched.h>
+#include <linux/sched/mm.h>
#include <linux/ksm.h>
#include <linux/fs.h>
#include <linux/file.h>
@@ -993,6 +994,18 @@ madvise_behavior_valid(int behavior)
}
}
+static bool
+process_madvise_behavior_valid(int behavior)
+{
+ switch (behavior) {
+ case MADV_COLD:
+ case MADV_PAGEOUT:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* madvise_common - request behavior hint to address range of the target process
*
@@ -1169,3 +1182,54 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
{
return madvise_common(current, current->mm, start, len_in, behavior);
}
+
+SYSCALL_DEFINE5(process_madvise, int, pidfd, unsigned long, start,
+ size_t, len_in, int, behavior, unsigned long, flags)
+{
+ int ret;
+ struct fd f;
+ struct pid *pid;
+ struct task_struct *task;
+ struct mm_struct *mm;
+
+ if (flags != 0)
+ return -EINVAL;
+
+ if (!process_madvise_behavior_valid(behavior))
+ return -EINVAL;
+
+ f = fdget(pidfd);
+ if (!f.file)
+ return -EBADF;
+
+ pid = pidfd_pid(f.file);
+ if (IS_ERR(pid)) {
+ ret = PTR_ERR(pid);
+ goto err;
+ }
+
+ rcu_read_lock();
+ task = pid_task(pid, PIDTYPE_PID);
+ if (!task) {
+ rcu_read_unlock();
+ ret = -ESRCH;
+ goto err;
+ }
+
+ get_task_struct(task);
+ rcu_read_unlock();
+
+ mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS);
+ if (IS_ERR_OR_NULL(mm)) {
+ ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH;
+ goto release_task;
+ }
+
+ ret = madvise_common(task, mm, start, len_in, behavior);
+ mmput(mm);
+release_task:
+ put_task_struct(task);
+err:
+ fdput(f);
+ return ret;
+}
--
2.25.0.rc1.283.g88dfdc4193-goog
^ permalink raw reply related
* [PATCH 1/4] mm: factor out madvise's core functionality
From: Minchan Kim @ 2020-01-10 21:34 UTC (permalink / raw)
To: Andrew Morton
Cc: LKML, linux-mm, linux-api, oleksandr, Suren Baghdasaryan,
Tim Murray, Daniel Colascione, Sandeep Patil, Sonny Rao,
Brian Geffon, Michal Hocko, Johannes Weiner, Shakeel Butt,
John Dias, Minchan Kim
In-Reply-To: <20200110213433.94739-1-minchan@kernel.org>
This patch factor out madvise's core functionality so that upcoming
patch can reuse it without duplication. It shouldn't change any behavior.
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
mm/madvise.c | 194 +++++++++++++++++++++++++++++----------------------
1 file changed, 111 insertions(+), 83 deletions(-)
diff --git a/mm/madvise.c b/mm/madvise.c
index bcdb6a042787..0c901de531e4 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -35,6 +35,7 @@
struct madvise_walk_private {
struct mmu_gather *tlb;
bool pageout;
+ struct task_struct *task;
};
/*
@@ -306,12 +307,13 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
bool pageout = private->pageout;
struct mm_struct *mm = tlb->mm;
struct vm_area_struct *vma = walk->vma;
+ struct task_struct *task = private->task;
pte_t *orig_pte, *pte, ptent;
spinlock_t *ptl;
struct page *page = NULL;
LIST_HEAD(page_list);
- if (fatal_signal_pending(current))
+ if (fatal_signal_pending(task))
return -EINTR;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -469,12 +471,14 @@ static const struct mm_walk_ops cold_walk_ops = {
};
static void madvise_cold_page_range(struct mmu_gather *tlb,
+ struct task_struct *task,
struct vm_area_struct *vma,
unsigned long addr, unsigned long end)
{
struct madvise_walk_private walk_private = {
.pageout = false,
.tlb = tlb,
+ .task = task,
};
tlb_start_vma(tlb, vma);
@@ -482,7 +486,7 @@ static void madvise_cold_page_range(struct mmu_gather *tlb,
tlb_end_vma(tlb, vma);
}
-static long madvise_cold(struct vm_area_struct *vma,
+static long madvise_cold(struct task_struct *task, struct vm_area_struct *vma,
struct vm_area_struct **prev,
unsigned long start_addr, unsigned long end_addr)
{
@@ -495,19 +499,21 @@ static long madvise_cold(struct vm_area_struct *vma,
lru_add_drain();
tlb_gather_mmu(&tlb, mm, start_addr, end_addr);
- madvise_cold_page_range(&tlb, vma, start_addr, end_addr);
+ madvise_cold_page_range(&tlb, task, vma, start_addr, end_addr);
tlb_finish_mmu(&tlb, start_addr, end_addr);
return 0;
}
static void madvise_pageout_page_range(struct mmu_gather *tlb,
+ struct task_struct *task,
struct vm_area_struct *vma,
unsigned long addr, unsigned long end)
{
struct madvise_walk_private walk_private = {
.pageout = true,
.tlb = tlb,
+ .task = task,
};
tlb_start_vma(tlb, vma);
@@ -531,9 +537,9 @@ static inline bool can_do_pageout(struct vm_area_struct *vma)
inode_permission(file_inode(vma->vm_file), MAY_WRITE) == 0;
}
-static long madvise_pageout(struct vm_area_struct *vma,
- struct vm_area_struct **prev,
- unsigned long start_addr, unsigned long end_addr)
+static long madvise_pageout(struct task_struct *task,
+ struct vm_area_struct *vma, struct vm_area_struct **prev,
+ unsigned long start_addr, unsigned long end_addr)
{
struct mm_struct *mm = vma->vm_mm;
struct mmu_gather tlb;
@@ -547,7 +553,7 @@ static long madvise_pageout(struct vm_area_struct *vma,
lru_add_drain();
tlb_gather_mmu(&tlb, mm, start_addr, end_addr);
- madvise_pageout_page_range(&tlb, vma, start_addr, end_addr);
+ madvise_pageout_page_range(&tlb, task, vma, start_addr, end_addr);
tlb_finish_mmu(&tlb, start_addr, end_addr);
return 0;
@@ -751,7 +757,8 @@ static long madvise_dontneed_single_vma(struct vm_area_struct *vma,
return 0;
}
-static long madvise_dontneed_free(struct vm_area_struct *vma,
+static long madvise_dontneed_free(struct mm_struct *mm,
+ struct vm_area_struct *vma,
struct vm_area_struct **prev,
unsigned long start, unsigned long end,
int behavior)
@@ -763,8 +770,8 @@ static long madvise_dontneed_free(struct vm_area_struct *vma,
if (!userfaultfd_remove(vma, start, end)) {
*prev = NULL; /* mmap_sem has been dropped, prev is stale */
- down_read(¤t->mm->mmap_sem);
- vma = find_vma(current->mm, start);
+ down_read(&mm->mmap_sem);
+ vma = find_vma(mm, start);
if (!vma)
return -ENOMEM;
if (start < vma->vm_start) {
@@ -811,7 +818,8 @@ static long madvise_dontneed_free(struct vm_area_struct *vma,
* Application wants to free up the pages and associated backing store.
* This is effectively punching a hole into the middle of a file.
*/
-static long madvise_remove(struct vm_area_struct *vma,
+static long madvise_remove(struct mm_struct *mm,
+ struct vm_area_struct *vma,
struct vm_area_struct **prev,
unsigned long start, unsigned long end)
{
@@ -845,13 +853,13 @@ static long madvise_remove(struct vm_area_struct *vma,
get_file(f);
if (userfaultfd_remove(vma, start, end)) {
/* mmap_sem was not released by userfaultfd_remove() */
- up_read(¤t->mm->mmap_sem);
+ up_read(&mm->mmap_sem);
}
error = vfs_fallocate(f,
FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
offset, end - start);
fput(f);
- down_read(¤t->mm->mmap_sem);
+ down_read(&mm->mmap_sem);
return error;
}
@@ -925,21 +933,23 @@ static int madvise_inject_error(int behavior,
#endif
static long
-madvise_vma(struct vm_area_struct *vma, struct vm_area_struct **prev,
+madvise_vma(struct task_struct *task, struct mm_struct *mm,
+ struct vm_area_struct *vma, struct vm_area_struct **prev,
unsigned long start, unsigned long end, int behavior)
{
switch (behavior) {
case MADV_REMOVE:
- return madvise_remove(vma, prev, start, end);
+ return madvise_remove(mm, vma, prev, start, end);
case MADV_WILLNEED:
return madvise_willneed(vma, prev, start, end);
case MADV_COLD:
- return madvise_cold(vma, prev, start, end);
+ return madvise_cold(task, vma, prev, start, end);
case MADV_PAGEOUT:
- return madvise_pageout(vma, prev, start, end);
+ return madvise_pageout(task, vma, prev, start, end);
case MADV_FREE:
case MADV_DONTNEED:
- return madvise_dontneed_free(vma, prev, start, end, behavior);
+ return madvise_dontneed_free(mm, vma, prev, start,
+ end, behavior);
default:
return madvise_behavior(vma, prev, start, end, behavior);
}
@@ -984,67 +994,19 @@ madvise_behavior_valid(int behavior)
}
/*
- * The madvise(2) system call.
+ * madvise_common - request behavior hint to address range of the target process
*
- * Applications can use madvise() to advise the kernel how it should
- * handle paging I/O in this VM area. The idea is to help the kernel
- * use appropriate read-ahead and caching techniques. The information
- * provided is advisory only, and can be safely disregarded by the
- * kernel without affecting the correct operation of the application.
+ * @task: task_struct got behavior hint, not giving the hint
+ * @mm: mm_struct got behavior hint, not giving the hint
+ * @start: base address of the hinted range
+ * @len_in: length of the hinted range
+ * @behavior: requested hint
*
- * behavior values:
- * MADV_NORMAL - the default behavior is to read clusters. This
- * results in some read-ahead and read-behind.
- * MADV_RANDOM - the system should read the minimum amount of data
- * on any access, since it is unlikely that the appli-
- * cation will need more than what it asks for.
- * MADV_SEQUENTIAL - pages in the given range will probably be accessed
- * once, so they can be aggressively read ahead, and
- * can be freed soon after they are accessed.
- * MADV_WILLNEED - the application is notifying the system to read
- * some pages ahead.
- * MADV_DONTNEED - the application is finished with the given range,
- * so the kernel can free resources associated with it.
- * MADV_FREE - the application marks pages in the given range as lazy free,
- * where actual purges are postponed until memory pressure happens.
- * MADV_REMOVE - the application wants to free up the given range of
- * pages and associated backing store.
- * MADV_DONTFORK - omit this area from child's address space when forking:
- * typically, to avoid COWing pages pinned by get_user_pages().
- * MADV_DOFORK - cancel MADV_DONTFORK: no longer omit this area when forking.
- * MADV_WIPEONFORK - present the child process with zero-filled memory in this
- * range after a fork.
- * MADV_KEEPONFORK - undo the effect of MADV_WIPEONFORK
- * MADV_HWPOISON - trigger memory error handler as if the given memory range
- * were corrupted by unrecoverable hardware memory failure.
- * MADV_SOFT_OFFLINE - try to soft-offline the given range of memory.
- * MADV_MERGEABLE - the application recommends that KSM try to merge pages in
- * this area with pages of identical content from other such areas.
- * MADV_UNMERGEABLE- cancel MADV_MERGEABLE: no longer merge pages with others.
- * MADV_HUGEPAGE - the application wants to back the given range by transparent
- * huge pages in the future. Existing pages might be coalesced and
- * new pages might be allocated as THP.
- * MADV_NOHUGEPAGE - mark the given range as not worth being backed by
- * transparent huge pages so the existing pages will not be
- * coalesced into THP and new pages will not be allocated as THP.
- * MADV_DONTDUMP - the application wants to prevent pages in the given range
- * from being included in its core dump.
- * MADV_DODUMP - cancel MADV_DONTDUMP: no longer exclude from core dump.
- *
- * return values:
- * zero - success
- * -EINVAL - start + len < 0, start is not page-aligned,
- * "behavior" is not a valid value, or application
- * is attempting to release locked or shared pages,
- * or the specified address range includes file, Huge TLB,
- * MAP_SHARED or VMPFNMAP range.
- * -ENOMEM - addresses in the specified range are not currently
- * mapped, or are outside the AS of the process.
- * -EIO - an I/O error occurred while paging in data.
- * -EBADF - map exists, but area maps something that isn't a file.
- * -EAGAIN - a kernel resource was temporarily unavailable.
+ * @task could be a zombie leader if it calls sys_exit so accessing mm_struct
+ * via task->mm is prohibited. Please use @mm instead of task->mm.
*/
-SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
+static int madvise_common(struct task_struct *task, struct mm_struct *mm,
+ unsigned long start, size_t len_in, int behavior)
{
unsigned long end, tmp;
struct vm_area_struct *vma, *prev;
@@ -1082,10 +1044,10 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
write = madvise_need_mmap_write(behavior);
if (write) {
- if (down_write_killable(¤t->mm->mmap_sem))
+ if (down_write_killable(&mm->mmap_sem))
return -EINTR;
} else {
- down_read(¤t->mm->mmap_sem);
+ down_read(&mm->mmap_sem);
}
/*
@@ -1093,7 +1055,7 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
* ranges, just ignore them, but return -ENOMEM at the end.
* - different from the way of handling in mlock etc.
*/
- vma = find_vma_prev(current->mm, start, &prev);
+ vma = find_vma_prev(mm, start, &prev);
if (vma && start > vma->vm_start)
prev = vma;
@@ -1118,7 +1080,7 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
tmp = end;
/* Here vma->vm_start <= start < tmp <= (end|vma->vm_end). */
- error = madvise_vma(vma, &prev, start, tmp, behavior);
+ error = madvise_vma(task, mm, vma, &prev, start, tmp, behavior);
if (error)
goto out;
start = tmp;
@@ -1130,14 +1092,80 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
if (prev)
vma = prev->vm_next;
else /* madvise_remove dropped mmap_sem */
- vma = find_vma(current->mm, start);
+ vma = find_vma(mm, start);
}
out:
blk_finish_plug(&plug);
if (write)
- up_write(¤t->mm->mmap_sem);
+ up_write(&mm->mmap_sem);
else
- up_read(¤t->mm->mmap_sem);
+ up_read(&mm->mmap_sem);
return error;
}
+
+/*
+ * The madvise(2) system call.
+ *
+ * Applications can use madvise() to advise the kernel how it should
+ * handle paging I/O in this VM area. The idea is to help the kernel
+ * use appropriate read-ahead and caching techniques. The information
+ * provided is advisory only, and can be safely disregarded by the
+ * kernel without affecting the correct operation of the application.
+ *
+ * behavior values:
+ * MADV_NORMAL - the default behavior is to read clusters. This
+ * results in some read-ahead and read-behind.
+ * MADV_RANDOM - the system should read the minimum amount of data
+ * on any access, since it is unlikely that the appli-
+ * cation will need more than what it asks for.
+ * MADV_SEQUENTIAL - pages in the given range will probably be accessed
+ * once, so they can be aggressively read ahead, and
+ * can be freed soon after they are accessed.
+ * MADV_WILLNEED - the application is notifying the system to read
+ * some pages ahead.
+ * MADV_DONTNEED - the application is finished with the given range,
+ * so the kernel can free resources associated with it.
+ * MADV_FREE - the application marks pages in the given range as lazy free,
+ * where actual purges are postponed until memory pressure happens.
+ * MADV_REMOVE - the application wants to free up the given range of
+ * pages and associated backing store.
+ * MADV_DONTFORK - omit this area from child's address space when forking:
+ * typically, to avoid COWing pages pinned by get_user_pages().
+ * MADV_DOFORK - cancel MADV_DONTFORK: no longer omit this area when forking.
+ * MADV_WIPEONFORK - present the child process with zero-filled memory in this
+ * range after a fork.
+ * MADV_KEEPONFORK - undo the effect of MADV_WIPEONFORK
+ * MADV_HWPOISON - trigger memory error handler as if the given memory range
+ * were corrupted by unrecoverable hardware memory failure.
+ * MADV_SOFT_OFFLINE - try to soft-offline the given range of memory.
+ * MADV_MERGEABLE - the application recommends that KSM try to merge pages in
+ * this area with pages of identical content from other such areas.
+ * MADV_UNMERGEABLE- cancel MADV_MERGEABLE: no longer merge pages with others.
+ * MADV_HUGEPAGE - the application wants to back the given range by transparent
+ * huge pages in the future. Existing pages might be coalesced and
+ * new pages might be allocated as THP.
+ * MADV_NOHUGEPAGE - mark the given range as not worth being backed by
+ * transparent huge pages so the existing pages will not be
+ * coalesced into THP and new pages will not be allocated as THP.
+ * MADV_DONTDUMP - the application wants to prevent pages in the given range
+ * from being included in its core dump.
+ * MADV_DODUMP - cancel MADV_DONTDUMP: no longer exclude from core dump.
+ *
+ * return values:
+ * zero - success
+ * -EINVAL - start + len < 0, start is not page-aligned,
+ * "behavior" is not a valid value, or application
+ * is attempting to release locked or shared pages,
+ * or the specified address range includes file, Huge TLB,
+ * MAP_SHARED or VMPFNMAP range.
+ * -ENOMEM - addresses in the specified range are not currently
+ * mapped, or are outside the AS of the process.
+ * -EIO - an I/O error occurred while paging in data.
+ * -EBADF - map exists, but area maps something that isn't a file.
+ * -EAGAIN - a kernel resource was temporarily unavailable.
+ */
+SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
+{
+ return madvise_common(current, current->mm, start, len_in, behavior);
+}
--
2.25.0.rc1.283.g88dfdc4193-goog
^ permalink raw reply related
* [PATCH 0/4] introduce memory hinting API for external process
From: Minchan Kim @ 2020-01-10 21:34 UTC (permalink / raw)
To: Andrew Morton
Cc: LKML, linux-mm, linux-api, oleksandr, Suren Baghdasaryan,
Tim Murray, Daniel Colascione, Sandeep Patil, Sonny Rao,
Brian Geffon, Michal Hocko, Johannes Weiner, Shakeel Butt,
John Dias, Minchan Kim
Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that,
application could give hints to kernel what memory range are preferred to be
reclaimed. However, in some platform(e.g., Android), the information
required to make the hinting decision is not known to the app.
Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService),
and that daemon must be able to initiate reclaim on its own without any app
involvement.
To solve the concern, this patch introduces new syscall - process_madvise(2).
Bascially, it's same with madvise(2) syscall but it has some differences.
1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT|MERGEABLE|UNMEREABLE} at this moment.
Other hints in madvise will be opened when there are explicit requests from
community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address
space.
Minchan Kim (2):
mm: factor out madvise's core functionality
mm: introduce external memory hinting API
Oleksandr Natalenko (2):
mm/madvise: employ mmget_still_valid for write lock
mm/madvise: allow KSM hints for remote API
arch/alpha/kernel/syscalls/syscall.tbl | 1 +
arch/arm/tools/syscall.tbl | 1 +
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 2 +
arch/ia64/kernel/syscalls/syscall.tbl | 1 +
arch/m68k/kernel/syscalls/syscall.tbl | 1 +
arch/microblaze/kernel/syscalls/syscall.tbl | 1 +
arch/mips/kernel/syscalls/syscall_n32.tbl | 1 +
arch/mips/kernel/syscalls/syscall_n64.tbl | 1 +
arch/parisc/kernel/syscalls/syscall.tbl | 1 +
arch/powerpc/kernel/syscalls/syscall.tbl | 1 +
arch/s390/kernel/syscalls/syscall.tbl | 1 +
arch/sh/kernel/syscalls/syscall.tbl | 1 +
arch/sparc/kernel/syscalls/syscall.tbl | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/xtensa/kernel/syscalls/syscall.tbl | 1 +
include/linux/syscalls.h | 2 +
include/uapi/asm-generic/unistd.h | 5 +-
kernel/sys_ni.c | 1 +
mm/madvise.c | 263 ++++++++++++++------
21 files changed, 205 insertions(+), 85 deletions(-)
--
2.25.0.rc1.283.g88dfdc4193-goog
^ permalink raw reply
* Re: [PATCH RFC 0/1] mount: universally disallow mounting over symlinks
From: Aleksa Sarai @ 2020-01-10 21:07 UTC (permalink / raw)
To: Linus Torvalds
Cc: Al Viro, David Howells, Eric Biederman, stable, Christian Brauner,
Serge Hallyn, dev, Linux Containers, Linux API, linux-fsdevel,
Linux Kernel Mailing List, Ian Kent
In-Reply-To: <CAHk-=wgQ3yOBuK8mxpnntD8cfX-+10ba81f86BYg8MhvwpvOMg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2739 bytes --]
On 2020-01-07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> On Tue, Jan 7, 2020 at 7:13 PM Al Viro <viro@zeniv.linux.org.uk> wrote:
> > Another interesting question is whether we want O_PATH open
> > to trigger automounts.
>
> It does sound like they shouldn't, but as you say:
>
> > The thing is, we do *NOT* trigger them
> > (or traverse mountpoints) at the starting point of lookups.
> > I believe it's a mistake (and mine, at that), but I doubt that
> > there's anything that can be done about it at that point.
> > It's a user-visible behaviour [..]
>
> Hmm. I wonder how set in stone that is. We may have two decades of
> history of not doing it at start point of lookups, but we do *not*
> have two decades of history of O_PATH.
>
> So what I think we agree would be sane behavior would be for O_PATH
> opens to not trigger automounts (unless there's a slash at the end,
> whatever), but _do_ add the mount-point traversal to the beginning of
> lookups.
>
> But only do it for the actual O_PATH fd case, not the cwd/root/non-O_PATH case.
>
> That way we maintain original behavior: if somebody overmounts your
> cwd, you still see the pre-mount directory on lookups, because your
> cwd is "under" the mount.
>
> But if you open a file with O_PATH, and somebody does a mount
> _afterwards_, the openat() will see that later mount and/or do the
> automount.
>
> Don't you think that would be the more sane/obvious semantics of how
> O_PATH should work?
If I'm understanding this proposal correctly, this would be a problem
for the libpathrs use-case -- if this is done then there's no way to
avoid a TOCTOU with someone mounting and the userspace program checking
whether something is a mountpoint (unless you have Linux >5.6 and
RESOLVE_NO_XDEV). Today, you can (in theory) do it with MNT_EXPIRE:
1. Open the candidate directory.
2. umount2(MNT_EXPIRE) the fd.
* -EINVAL means it wasn't a mountpoint when we got the fd, and the
fd is a stable handle to the underlying directory.
* -EAGAIN or -EBUSY means that it was a mountpoint or became a
mountpoint after the fd was opened (we don't care about that, but
fail-safe is better here).
3. Use the fd from (1) for all operations.
Don't get me wrong, I want to fix this issue *properly* by adding some
new kernel features that allow us to avoid worrying about
mounts-over-magiclinks -- but on old kernels (which libpathrs cares
about) I would be worried about changes like this being backported
resulting in it being not possible to implement the hardening I
mentioned up-thread.
--
Aleksa Sarai
Senior Software Engineer (Containers)
SUSE Linux GmbH
<https://www.cyphar.com/>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply
* Re: [PATCHv8 02/34] lib/vdso: make do_hres and do_coarse as __always_inline
From: Vincenzo Frascino @ 2020-01-10 12:18 UTC (permalink / raw)
To: Thomas Gleixner, Dmitry Safonov,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Cc: Dmitry Safonov, Andrei Vagin, Adrian Reber, Andrei Vagin,
Andy Lutomirski, Arnd Bergmann, Christian Brauner,
Cyrill Gorcunov, Eric W. Biederman, H. Peter Anvin, Ingo Molnar,
Jann Horn, Jeff Dike, Oleg Nesterov, Pavel Emelyanov, Shuah Khan,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
criu-GEFAQzZX7r8dnm+yROfE0A, linux-api-u79uwXL29TY76Z2rM5mHXA,
x86-DgEjT+Ai2ygdnm+yROfE0A
In-Reply-To: <875zhja59q.fsf-ecDvlHI5BZPZikZi3RtOZ1XZhhPuCNm+@public.gmane.org>
On 1/10/20 12:02 PM, Thomas Gleixner wrote:
> Vincenzo Frascino <vincenzo.frascino-5wv7dgnIgG8@public.gmane.org> writes:
>> On 1/10/20 11:42 AM, Thomas Gleixner wrote:
>>>>> +» » /*·goto·allows·to·avoid·extra·inlining·of·do_hres.·*/
>>>>> +» » goto·out_hres;
>>>>
>>>> What is the performance impact of "goto out_hres"?
>>>
>>> On x86 it's invisible at least in my limited testing.
>>
>> On arm64 as well based on mine as well. Shall we keep the code more readable
>> here (without goto)?
>
> The delta patch below makes it readable again and also avoids the double
> inlining. Quick testing shows no difference.
>
I tested it on arm64 and it does not show any difference as well.
I vote for it :)
> Thanks,
>
> tglx
>
> 8<---------------
> --- a/lib/vdso/gettimeofday.c
> +++ b/lib/vdso/gettimeofday.c
> @@ -98,17 +98,15 @@ static __maybe_unused int
> msk = 1U << clock;
> if (likely(msk & VDSO_HRES)) {
> vd = &vd[CS_HRES_COARSE];
> -out_hres:
> - return do_hres(vd, clock, ts);
> } else if (msk & VDSO_COARSE) {
> do_coarse(&vd[CS_HRES_COARSE], clock, ts);
> return 0;
> } else if (msk & VDSO_RAW) {
> vd = &vd[CS_RAW];
> - /* This goto avoids extra inlining of do_hres. */
> - goto out_hres;
> + } else {
> + return -1;
> }
> - return -1;
> + return do_hres(vd, clock, ts);
> }
>
> static __maybe_unused int
>
--
Regards,
Vincenzo
^ permalink raw reply
* Re: [PATCHv8 02/34] lib/vdso: make do_hres and do_coarse as __always_inline
From: Thomas Gleixner @ 2020-01-10 12:02 UTC (permalink / raw)
To: Vincenzo Frascino, Dmitry Safonov, linux-kernel
Cc: Dmitry Safonov, Andrei Vagin, Adrian Reber, Andrei Vagin,
Andy Lutomirski, Arnd Bergmann, Christian Brauner,
Cyrill Gorcunov, Eric W. Biederman, H. Peter Anvin, Ingo Molnar,
Jann Horn, Jeff Dike, Oleg Nesterov, Pavel Emelyanov, Shuah Khan,
containers, criu, linux-api, x86
In-Reply-To: <e74a63cb-5248-4473-81a7-d1b2f959ec7a@arm.com>
Vincenzo Frascino <vincenzo.frascino@arm.com> writes:
> On 1/10/20 11:42 AM, Thomas Gleixner wrote:
>>>> +» » /*·goto·allows·to·avoid·extra·inlining·of·do_hres.·*/
>>>> +» » goto·out_hres;
>>>
>>> What is the performance impact of "goto out_hres"?
>>
>> On x86 it's invisible at least in my limited testing.
>
> On arm64 as well based on mine as well. Shall we keep the code more readable
> here (without goto)?
The delta patch below makes it readable again and also avoids the double
inlining. Quick testing shows no difference.
Thanks,
tglx
8<---------------
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -98,17 +98,15 @@ static __maybe_unused int
msk = 1U << clock;
if (likely(msk & VDSO_HRES)) {
vd = &vd[CS_HRES_COARSE];
-out_hres:
- return do_hres(vd, clock, ts);
} else if (msk & VDSO_COARSE) {
do_coarse(&vd[CS_HRES_COARSE], clock, ts);
return 0;
} else if (msk & VDSO_RAW) {
vd = &vd[CS_RAW];
- /* This goto avoids extra inlining of do_hres. */
- goto out_hres;
+ } else {
+ return -1;
}
- return -1;
+ return do_hres(vd, clock, ts);
}
static __maybe_unused int
^ permalink raw reply
* Re: [PATCHv8 02/34] lib/vdso: make do_hres and do_coarse as __always_inline
From: Vincenzo Frascino @ 2020-01-10 11:47 UTC (permalink / raw)
To: Thomas Gleixner, Dmitry Safonov,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Cc: Dmitry Safonov, Andrei Vagin, Adrian Reber, Andrei Vagin,
Andy Lutomirski, Arnd Bergmann, Christian Brauner,
Cyrill Gorcunov, Eric W. Biederman, H. Peter Anvin, Ingo Molnar,
Jann Horn, Jeff Dike, Oleg Nesterov, Pavel Emelyanov, Shuah Khan,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
criu-GEFAQzZX7r8dnm+yROfE0A, linux-api-u79uwXL29TY76Z2rM5mHXA,
x86-DgEjT+Ai2ygdnm+yROfE0A
In-Reply-To: <878smfa66i.fsf-ecDvlHI5BZPZikZi3RtOZ1XZhhPuCNm+@public.gmane.org>
On 1/10/20 11:42 AM, Thomas Gleixner wrote:
> Vincenzo Frascino <vincenzo.frascino-5wv7dgnIgG8@public.gmane.org> writes:
>> On 11/12/19 1:26 AM, Dmitry Safonov wrote:
>>> +» » vd·=·&vd[CS_HRES_COARSE];
>>> +out_hres:
>>> +» » return·do_hres(vd,·clock,·ts);
>>> » }·else·if·(msk·&·VDSO_COARSE)·{
>>> » » do_coarse(&vd[CS_HRES_COARSE],·clock,·ts);
>>> » » return·0;
>>> » }·else·if·(msk·&·VDSO_RAW)·{
>>> -» » return·do_hres(&vd[CS_RAW],·clock,·ts);
>>> +» » vd·=·&vd[CS_RAW];
>>> +» » /*·goto·allows·to·avoid·extra·inlining·of·do_hres.·*/
>>> +» » goto·out_hres;
>>
>> What is the performance impact of "goto out_hres"?
>
> On x86 it's invisible at least in my limited testing.
On arm64 as well based on mine as well. Shall we keep the code more readable
here (without goto)?
>
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply
* Re: [PATCHv8 02/34] lib/vdso: make do_hres and do_coarse as __always_inline
From: Thomas Gleixner @ 2020-01-10 11:42 UTC (permalink / raw)
To: Vincenzo Frascino, Dmitry Safonov, linux-kernel
Cc: Dmitry Safonov, Andrei Vagin, Adrian Reber, Andrei Vagin,
Andy Lutomirski, Arnd Bergmann, Christian Brauner,
Cyrill Gorcunov, Eric W. Biederman, H. Peter Anvin, Ingo Molnar,
Jann Horn, Jeff Dike, Oleg Nesterov, Pavel Emelyanov, Shuah Khan,
containers, criu, linux-api, x86
In-Reply-To: <ed2e65ae-75b0-ed79-0a95-90be6b82e6be@arm.com>
Vincenzo Frascino <vincenzo.frascino@arm.com> writes:
> On 11/12/19 1:26 AM, Dmitry Safonov wrote:
>> +» » vd·=·&vd[CS_HRES_COARSE];
>> +out_hres:
>> +» » return·do_hres(vd,·clock,·ts);
>> » }·else·if·(msk·&·VDSO_COARSE)·{
>> » » do_coarse(&vd[CS_HRES_COARSE],·clock,·ts);
>> » » return·0;
>> » }·else·if·(msk·&·VDSO_RAW)·{
>> -» » return·do_hres(&vd[CS_RAW],·clock,·ts);
>> +» » vd·=·&vd[CS_RAW];
>> +» » /*·goto·allows·to·avoid·extra·inlining·of·do_hres.·*/
>> +» » goto·out_hres;
>
> What is the performance impact of "goto out_hres"?
On x86 it's invisible at least in my limited testing.
Thanks,
tglx
^ permalink raw reply
* Re: [PATCH v9 0/4] Add pidfd_getfd syscall
From: Christian Brauner @ 2020-01-10 11:40 UTC (permalink / raw)
To: Sargun Dhillon
Cc: linux-kernel, containers, linux-api, linux-fsdevel, tycho, jannh,
cyphar, oleg, luto, viro, gpascutto, ealvarez, fweimer, jld, arnd
In-Reply-To: <20200107205449.5dcp7o3hplg7r3fw@wittgenstein>
On Tue, Jan 07, 2020 at 09:54:49PM +0100, Christian Brauner wrote:
> On Tue, Jan 07, 2020 at 09:59:23AM -0800, Sargun Dhillon wrote:
> > This patchset introduces a mechanism (pidfd_getfd syscall) to get file
> > descriptors from other processes via pidfd. Although this can be achieved
> > using SCM_RIGHTS, and parasitic code injection, this offers a more
> > straightforward mechanism, with less overhead and complexity. The process
> > under manipulation's fd still remains valid, and unmodified by the
> > copy operation.
> >
> > It introduces a flags field. The flags field is reserved a the moment,
> > but the intent is to extend it with the following capabilities:
> > * Close the remote FD when copying it
> > * Drop the cgroup data if it's a fd pointing a socket when copying it
> >
> > The syscall numbers were chosen to be one greater than openat2.
> >
> > Summary of history:
> > This initially started as a ptrace command. It did not require the process
> > to be stopped, and felt like kind of an awkward fit for ptrace. After that,
> > it moved to an ioctl on the pidfd. Given the core functionality, it made
> > sense to make it a syscall which did not require the process to be stopped.
> >
> > Previous versions:
> > V8: https://lore.kernel.org/lkml/20200103162928.5271-1-sargun@sargun.me/
> > V7: https://lore.kernel.org/lkml/20191226180227.GA29389@ircssh-2.c.rugged-nimbus-611.internal/
> > V6: https://lore.kernel.org/lkml/20191223210823.GA25083@ircssh-2.c.rugged-nimbus-611.internal/
> > V5: https://lore.kernel.org/lkml/20191220232746.GA20215@ircssh-2.c.rugged-nimbus-611.internal/
> > V4: https://lore.kernel.org/lkml/20191218235310.GA17259@ircssh-2.c.rugged-nimbus-611.internal/
> > V3: https://lore.kernel.org/lkml/20191217005842.GA14379@ircssh-2.c.rugged-nimbus-611.internal/
> > V2: https://lore.kernel.org/lkml/20191209070446.GA32336@ircssh-2.c.rugged-nimbus-611.internal/
> > RFC V1: https://lore.kernel.org/lkml/20191205234450.GA26369@ircssh-2.c.rugged-nimbus-611.internal/
>
> I don't see anything wrong with this series anymore:
>
> Acked-by: Christian Brauner <christian.brauner@ubuntu.com>
>
> Other Acked-bys/Reviewed-bys and reviews of course strongly encouraged!
> Christian
Fyi, I'm waiting a few days on a reply from Al.
Depending on his input the intent rn is to move this into my for-next
early next week.
Christian
^ permalink raw reply
* Re: [PATCHv8 00/34] kernel: Introduce Time Namespace
From: Vincenzo Frascino @ 2020-01-10 9:52 UTC (permalink / raw)
To: Thomas Gleixner, Dmitry Safonov, Andy Lutomirski, Ingo Molnar
Cc: Dmitry Safonov, Andrei Vagin, open list, Adrian Reber,
Andrei Vagin, Arnd Bergmann, Christian Brauner, Cyrill Gorcunov,
Eric W. Biederman, H. Peter Anvin, Jann Horn, Jeff Dike,
Oleg Nesterov, Pavel Emelyanov, Shuah Khan, Linux Containers,
crml, Linux API, X86 ML
In-Reply-To: <87y2ug9w1h.fsf@nanos.tec.linutronix.de>
Hi Thomas,
On 1/9/20 9:09 PM, Thomas Gleixner wrote:
> Dmitry Safonov <0x7f454c46@gmail.com> writes:
>
>> Gentle ping, in case you have time to look at this.
>
> I'm looking at it and so far I'm quite happy.
>
> Andy, Vincenco any opinions?
>
I started looking at them after the holidays, in general I am happy with what I
have seen till now.
I would like to complete some testing especially on the platforms that are not
touched by this patchset to make sure that there are no side effects on the
unified vDSOs and then I think I am ok with the series.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply
* Re: [PATCHv8 02/34] lib/vdso: make do_hres and do_coarse as __always_inline
From: Vincenzo Frascino @ 2020-01-10 9:45 UTC (permalink / raw)
To: Dmitry Safonov, linux-kernel-u79uwXL29TY76Z2rM5mHXA
Cc: Dmitry Safonov, Andrei Vagin, Adrian Reber, Andrei Vagin,
Andy Lutomirski, Arnd Bergmann, Christian Brauner,
Cyrill Gorcunov, Eric W. Biederman, H. Peter Anvin, Ingo Molnar,
Jann Horn, Jeff Dike, Oleg Nesterov, Pavel Emelyanov, Shuah Khan,
Thomas Gleixner,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
criu-GEFAQzZX7r8dnm+yROfE0A, linux-api-u79uwXL29TY76Z2rM5mHXA,
x86-DgEjT+Ai2ygdnm+yROfE0A
In-Reply-To: <20191112012724.250792-3-dima-nzgTgzXrdUbQT0dZR+AlfA@public.gmane.org>
On 11/12/19 1:26 AM, Dmitry Safonov wrote:
> +» » vd·=·&vd[CS_HRES_COARSE];
> +out_hres:
> +» » return·do_hres(vd,·clock,·ts);
> » }·else·if·(msk·&·VDSO_COARSE)·{
> » » do_coarse(&vd[CS_HRES_COARSE],·clock,·ts);
> » » return·0;
> » }·else·if·(msk·&·VDSO_RAW)·{
> -» » return·do_hres(&vd[CS_RAW],·clock,·ts);
> +» » vd·=·&vd[CS_RAW];
> +» » /*·goto·allows·to·avoid·extra·inlining·of·do_hres.·*/
> +» » goto·out_hres;
What is the performance impact of "goto out_hres"?
--
Regards,
Vincenzo
^ permalink raw reply
* Re: [PATCH v27 10/12] LRNG - add SP800-90B compliant health tests
From: Stephan Mueller @ 2020-01-10 8:27 UTC (permalink / raw)
To: Randy Dunlap
Cc: Arnd Bergmann, Greg Kroah-Hartman,
linux-crypto-u79uwXL29TY76Z2rM5mHXA, LKML,
linux-api-u79uwXL29TY76Z2rM5mHXA, Eric W. Biederman,
Alexander E. Patrakov, Ahmed S. Darwish, Theodore Y. Ts'o,
Willy Tarreau, Matthew Garrett, Vito Caputo, Andreas Dilger,
Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange
In-Reply-To: <cd9e893a-ce63-4e9a-fc19-553b1c5f1cff-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Am Freitag, 10. Januar 2020, 01:20:44 CET schrieb Randy Dunlap:
Hi Randy,
I have removed the two stars in front of each comment.
I have also fixed the respective comments in lrng_interfaces.c,
lrng_archrandom.c, lrng_chacha20.c, lrng_drng.c, lrng_pool.c, lrng_switch.c,
lrng_sw_noise.c. In all other files there were no changes needed.
Thank you.
Ciao
Stephan
^ permalink raw reply
* Re: [PATCH v3 0/8] Rework random blocking
From: Stephan Mueller @ 2020-01-10 7:53 UTC (permalink / raw)
To: Kurt Roeckx
Cc: Theodore Y. Ts'o, Andy Lutomirski, Andy Lutomirski, LKML,
Linux API, Kees Cook, Jason A. Donenfeld, Ahmed S. Darwish,
Lennart Poettering, Eric W. Biederman, Alexander E. Patrakov,
Michael Kerrisk, Willy Tarreau, Matthew Garrett,
Ext4 Developers List, linux-man
In-Reply-To: <20200109230237.GA2992-burXGKnpAKGzQB+pC5nmwQ@public.gmane.org>
Am Freitag, 10. Januar 2020, 00:02:37 CET schrieb Kurt Roeckx:
Hi Kurt,
> On Thu, Jan 09, 2020 at 05:40:11PM -0500, Theodore Y. Ts'o wrote:
> > On Thu, Jan 09, 2020 at 11:02:30PM +0100, Kurt Roeckx wrote:
> > > One thing the NIST DRBGs have is prediction resistance, which is
> > > done by reseeding. If you chain DRBGs, you tell your parent DRBG
> > > that you want prediction resistance, so your parent will also
> > > reseed. There currently is no way to tell the kernel to reseed.
> >
> > It would be simple enough to add a new flag, perhaps GRND_RESEED, to
> > getrandom() which requests that the kernel reseed first. This would
> > require sufficient amounts of entropy in the input pool to do the
> > reseed; if there is not enough, the getrandom() call would block until
> > there was enough. If GRND_NONBLOCK is supplied, then getrandom()
> > would return EAGAIN if there wasn't sufficient entropy.
> >
> > Is this what you want?
>
> I think some people might want to see it, but I think you
> shouldn't add it.
Just for your information: I played with that already as seen in [1] which
does not require any kernel change.
The only issue that is currently there are the two races noted in [1]. These
races seem to be only addressable when the reseeding and the gathering of
random numbers are atomic. I was toying with the idea that the RNDRESEEDCRNG
allows the user to specify an output buffer which would be filled in an atomic
operation when the reseed is invoked. That buffer should only be at most in
size of the security strength of the DRNG.
[1] https://github.com/smuellerDD/lrng/blob/master/test/syscall_test.c#L101
>
> > > I don't think we want that. As far as I know, the only reason for
> > > using /dev/random is that /dev/urandom returns data before it
> > > has sufficient entropy.
> >
> > Is there any objections to just using getrandom(2)?
>
> It provides the interface we want, so no. But there are still
> people who don't have it for various reasons. OpenSSL actually
> does the system call itself if libc doesn't provider a wrapper for
> it.
>
>
> Kurt
Ciao
Stephan
^ permalink raw reply
* Re: [PATCH v27 12/12] LRNG - add power-on and runtime self-tests
From: Stephan Mueller @ 2020-01-10 7:48 UTC (permalink / raw)
To: Randy Dunlap
Cc: Arnd Bergmann, Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange
In-Reply-To: <2f225250-c1e1-c6e1-026c-9c63853ab7d1@infradead.org>
Am Freitag, 10. Januar 2020, 01:22:51 CET schrieb Randy Dunlap:
Hi Randy,
> Hi,
>
> On 1/9/20 12:35 AM, Stephan Müller wrote:
> > ---
> >
> > drivers/char/lrng/Kconfig | 25 ++
> > drivers/char/lrng/Makefile | 1 +
> > drivers/char/lrng/lrng_selftest.c | 418 ++++++++++++++++++++++++++++++
> > 3 files changed, 444 insertions(+)
> > create mode 100644 drivers/char/lrng/lrng_selftest.c
> >
> > diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
> > index 394066aa5a86..e9827e7df715 100644
> > --- a/drivers/char/lrng/Kconfig
> > +++ b/drivers/char/lrng/Kconfig
> > @@ -175,4 +175,29 @@ config LRNG_TESTING
> >
> > If unsure, say N.
> >
> > +config LRNG_SELFTEST
> > + bool "Enable power-on and on-demand self-tests"
> > + help
> > + The power-on self tests are executed during boot time
>
> self-tests
Fixed in all occurrences. Thank you.
>
> > + covering the ChaCha20 DRNG, the LFSR processing and the
> > + time stamp management of the LRNG.
> > +
> > + The on-demand self-tests are triggered by writing any
> > + value into the SysFS file selftest_status. At the same
> > + time, when reading this file, the test status is
> > + returned. A zero indicates that all tests were executed
> > + successfully.
> > +
> > + If unsure, say Y.
> > +
> > +if LRNG_SELFTEST
> > +
> > +config LRNG_SELFTEST_PANIC
> > + bool "Panic the kernel upon self-test failure"
> > + help
> > + If the option is enabled, the kernel is terminated if an
> > + LRNG power-on self-test failure is detected.
> > +
> > +endif # LRNG_SELFTEST
> > +
> >
> > endif # LRNG
Ciao
Stephan
^ permalink raw reply
* Re: [PATCH v27 09/12] LRNG - add Jitter RNG fast noise source
From: Stephan Mueller @ 2020-01-10 7:45 UTC (permalink / raw)
To: Randy Dunlap
Cc: Arnd Bergmann, Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange
In-Reply-To: <0ba976fa-fed1-4720-c039-043160dbce9e@infradead.org>
Am Freitag, 10. Januar 2020, 01:24:47 CET schrieb Randy Dunlap:
Hi Randy,
> Hi,
>
> On 1/9/20 12:34 AM, Stephan Müller wrote:
> > ---
> >
> > drivers/char/lrng/Kconfig | 11 +++++
> > drivers/char/lrng/Makefile | 1 +
> > drivers/char/lrng/lrng_jent.c | 87 +++++++++++++++++++++++++++++++++++
> > 3 files changed, 99 insertions(+)
> > create mode 100644 drivers/char/lrng/lrng_jent.c
> >
> > diff --git a/drivers/char/lrng/lrng_jent.c b/drivers/char/lrng/lrng_jent.c
> > new file mode 100644
> > index 000000000000..97c0d192e9c8
> > --- /dev/null
> > +++ b/drivers/char/lrng/lrng_jent.c
> > @@ -0,0 +1,87 @@
> > +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
> > +/*
> > + * LRNG Fast Noise Source: Jitter RNG
> > + *
> > + * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
> > + */
> > +
> > +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> > +
> > +#include <linux/types.h>
> > +#include <crypto/internal/jitterentropy.h>
> > +
> > +#include "lrng_internal.h"
> > +
> > +/*
> > + * Estimated entropy of data is a 16th of
> > LRNG_DRNG_SECURITY_STRENGTH_BITS. + * Albeit a full entropy assessment is
> > provided for the noise source indicating + * that it provides high
> > entropy rates and considering that it deactivates + * when it detects
> > insufficient hardware, the chosen under estimation of + * entropy is
> > considered to be acceptable to all reviewers.
> > + */
> > +static u32 jitterrng = LRNG_DRNG_SECURITY_STRENGTH_BITS>>4;
> > +module_param(jitterrng, uint, 0644);
> > +MODULE_PARM_DESC(jitterrng, "Entropy in bits of 256 data bits from Jitter
> > " + "RNG noise source");
> > +
> > +/**
> > + * Get Jitter RNG entropy
> > + *
> > + * @outbuf buffer to store entropy
> > + * @outbuflen length of buffer
> > + * @return > 0 on success where value provides the added entropy in bits
> > + * 0 if no fast source was available
> > + */
>
> Don't begin the comment block with /**
> or convert it to Linux kernel-doc notation format.
Yes, the colons are missing and there is no @return. It will be fixed with the
next patch set.
Thank you.
>
> > +static struct rand_data *lrng_jent_state;
> > +
> > +u32 lrng_get_jent(u8 *outbuf, unsigned int outbuflen)
> > +{
> > + int ret;
> > + u32 ent_bits = jitterrng;
> > + unsigned long flags;
> > + static DEFINE_SPINLOCK(lrng_jent_lock);
> > + static int lrng_jent_initialized = 0;
> > +
> > + spin_lock_irqsave(&lrng_jent_lock, flags);
> > +
> > + if (!ent_bits || (lrng_jent_initialized == -1)) {
> > + spin_unlock_irqrestore(&lrng_jent_lock, flags);
> > + return 0;
> > + }
> > +
> > + if (!lrng_jent_initialized) {
> > + lrng_jent_state = jent_lrng_entropy_collector();
> > + if (!lrng_jent_state) {
> > + jitterrng = 0;
> > + lrng_jent_initialized = -1;
> > + spin_unlock_irqrestore(&lrng_jent_lock, flags);
> > + pr_info("Jitter RNG unusable on current system\n");
> > + return 0;
> > + }
> > + lrng_jent_initialized = 1;
> > + pr_debug("Jitter RNG working on current system\n");
> > + }
> > + ret = jent_read_entropy(lrng_jent_state, outbuf, outbuflen);
> > + spin_unlock_irqrestore(&lrng_jent_lock, flags);
> > +
> > + if (ret) {
> > + pr_debug("Jitter RNG failed with %d\n", ret);
> > + return 0;
> > + }
> > +
> > + /* Obtain entropy statement */
> > + if (outbuflen != LRNG_DRNG_SECURITY_STRENGTH_BYTES)
> > + ent_bits = (ent_bits * outbuflen<<3) /
> > + LRNG_DRNG_SECURITY_STRENGTH_BITS;
> > + /* Cap entropy to buffer size in bits */
> > + ent_bits = min_t(u32, ent_bits, outbuflen<<3);
> > + pr_debug("obtained %u bits of entropy from Jitter RNG noise source\n",
> > + ent_bits);
> > +
> > + return ent_bits;
> > +}
> > +
> > +u32 lrng_jent_entropylevel(void)
> > +{
> > + return min_t(u32, jitterrng, LRNG_DRNG_SECURITY_STRENGTH_BITS);
> > +}
>
> thanks.
Ciao
Stephan
^ permalink raw reply
* Re: [PATCH RFC 0/1] mount: universally disallow mounting over symlinks
From: Ian Kent @ 2020-01-10 6:20 UTC (permalink / raw)
To: Al Viro, Linus Torvalds
Cc: Aleksa Sarai, David Howells, Eric Biederman, stable,
Christian Brauner, Serge Hallyn, dev, Linux Containers, Linux API,
linux-fsdevel, Linux Kernel Mailing List
In-Reply-To: <20200110041523.GK8904@ZenIV.linux.org.uk>
On Fri, 2020-01-10 at 04:15 +0000, Al Viro wrote:
> On Thu, Jan 09, 2020 at 04:08:16PM -0800, Linus Torvalds wrote:
> > On Wed, Jan 8, 2020 at 1:34 PM Al Viro <viro@zeniv.linux.org.uk>
> > wrote:
> > > The point is, we'd never followed mounts on /proc/self/cwd et.al.
> > > I hadn't checked 2.0, but 2.1.100 ('97, before any changes from
> > > me)
> > > is that way.
> >
> > Hmm. If that's the case, maybe they should be marked implicitly as
> > O_PATH when opened?
>
> I thought you wanted O_PATH as starting point to have mounts
> traversed?
> Confused...
>
> > > Actually, scratch that - 2.0 behaves the same way
> > > (mountpoint crossing is done in iget() there; is that Minix
> > > influence
> > > or straight from the Lions' book?)
> >
> > I don't think I ever had access to Lions' - I've _seen_ a printout
> > of
> > it later, and obviously maybe others did,
> >
> > More likely it's from Maurice Bach: the Design of the Unix
> > Operating
> > System. I'm pretty sure that's where a lot of the FS layer stuff
> > came
> > from. Certainly the bad old buffer head interfaces, and quite
> > likely
> > the iget() stuff too.
> >
> > > 0.10: forward traversal in iget(), back traversal in
> > > fs/namei.c:find_entry()
> >
> > Whee, you _really_ went back in time.
> >
> > So I did too.
> >
> > And looking at that code in iget(), I doubt it came from anywhere.
> > Christ. It's just looping over a fixed-size array, both when
> > finding
> > the inode, and finding the superblock.
> >
> > Cute, but unbelievably stupid. It was a more innocent time.
> >
> > In other words, I think you can chalk it up to just me, because
> > blaming anybody else for that garbage would be very very unfair
> > indeed
> > ;)
>
> See
> https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/sys/sys/iget.c
> Exactly the same algorithm, complete with linear searches over those
> fixed-sized array.
>
> <grabs Bach> Right, he simply transcribes v7 iget().
>
> So I suspect that you are right - your variant of iget was pretty
> much
> one-to-one implementation of Bach's description of v7 iget.
>
> Your namei wasn't - Bach has 'if the entry points to root and you are
> in the root and name is "..", find mount table entry (by device
> number),
> drop your directory inode, grab the inode of mountpount and restart
> the search for ".." in there', which gives back traversals to
> arbitrary
> depth. And v7 namei() (as Bach mentions) uses iget() for starting
> point
> as well as for each component. You kept pointers instead, which is
> where
> the other difference has come from (no mount traversal at the
> starting
> point)...
>
> Actually, I've misread your code in 0.10 - it does unlimited forward
> traversals; it's back traversals that go only one level. The forward
> ones got limited to one level in 0.95, but then mount-over-root had
> been banned all along. I'd read the pre-dcache variant of iget(),
> seen it go pretty much all the way back to beginning and hadn't
> sorted out the 0.12 -> 0.95 transition...
>
> > > How would your proposal deal with access("/proc/self/fd/42/foo",
> > > MAY_READ)
> > > vs. faccessat(42, "foo", MAY_READ)?
> >
> > I think that in a perfect world, the O_PATH'ness of '42' would be
> > the
> > deciding factor. Wouldn't those be the best and most consistent
> > semantics?
> >
> > And then 'cwd'/'root' always have the O_PATH behavior.
>
> See above - unless I'm misparsing you, you wanted mount traversals in
> the
> starting point if it's ...at() with O_PATH fd. With O_PATH open()
> not
> doing them.
>
> For cwd and root the situation is opposite - we do NOT traverse
> mounts
> for those. And that's really too late to change.
>
> > > The latter would trigger automount,
> > > the former would not... Or would you extend that to "traverse
> > > mounts
> > > upon following procfs links, if the file in question had been
> > > opened with
> > > O_PATH"?
> >
> > Exactly.
> >
> > But you know what? I do not believe this is all that important, and
> > I
> > doubt it will matter to anybody.
>
> FWIW, digging through the automount-related parts of that stuff has
> caught several fun issues. One (and I'm rather embarrassed by it)
> should've been caught back in commit 8aef18845266 (VFS: Fix vfsmount
> overput on simultaneous automount). To quote the commit message:
> The problem is that lock_mount() drops the caller's reference to
> the
> mountpoint's vfsmount in the case where it finds something
> already mounted on
> the mountpoint as it transits to the mounted filesystem and
> replaces path->mnt
> with the new mountpoint vfsmount.
>
> During a pathwalk, however, we don't take a reference on the
> vfsmount if it is
> the same as the one in the nameidata struct, but do_add_mount()
> doesn't know
> this.
> At which point I should've gone "what the fuck?" - lock_mount() does,
> indeed,
> drop path->mnt in this situation and replaces it with the whatever's
> come to
> cover it. For mount(2) that's the right thing to do - we _want_ to
> mount
> on top of whatever we have at the mountpoint. For automounts we very
> much
> don't want that - it's either "mount right on top of the automount
> trigger"
> or discard whatever we'd been about to mount and walk into whatever's
> got
> mounted there (presumably the same thing triggered by another
> process).
> We kinda-sorta get that effect, but in a very convoluted way:
> do_add_mount()
> will refuse to mount something on top of itself -
> /* Refuse the same filesystem on the same mount point */
> err = -EBUSY;
> if (path->mnt->mnt_sb == newmnt->mnt.mnt_sb &&
> path->mnt->mnt_root == path->dentry)
> goto unlock;
> which will end up with -EBUSY returned (and recognized by
> follow_automount()).
>
> First of all, that's unreliable. If somebody not only has triggered
> that
> automount, but managed to _mount_ something else on top (for example,
> has triggered it by lookup of mountpoint-to-be in mount(2)), we'll
> end
> up not triggering that check. In which case we'll get something like
> nfs referral point under nfs automounted there under tmpfs from
> explicit
> overmount under same nfs mount we'd automounted there - identical to
> what's
> been buried under tmpfs. It's hard to hit, but not impossibly so.
>
> What's more, the whole solution is a kludge - the root of problem is
> that lock_mount() is the wrong thing to do in case of
> finish_automount().
> We don't want to go into whatever's overmounting us there, both for
> the reasons above *and* because it's a PITA for the caller. So the
> right solution is
> * lift lock_mount() call from do_add_mount() into its callers
> (all 2 of them); while we are at it, lift unlock_mount() as well
> (makes for simpler failure exits in do_add_mount()).
> * replace the call of lock_mount() in finish_automount()
> with variant that doesn't do "unlock, walk deeper and retry locking",
> returning ERR_PTR(-EBUSY) in such case.
> * get rid of the kludge introduced in that commit. Better
> yet, don't bother with traversing into the covering mount in case
> of success - let the caller of follow_automount() do that. Which
> eliminates the need to pass need_mntput to the sucker and suggests
> an even better solution - have this analogue of lock_mount()
> return NULL instead of ERR_PTR(-EBUSY) and treat it in
> finish_automount()
> as "OK, discard what we wanted to mount and return 0". That gets
> rid of the entire
> err = finish_automount(mnt, path);
> switch (err) {
> case -EBUSY:
> /* Someone else made a mount here whilst we were busy
> */
> return 0;
> case 0:
> path_put(path);
> path->mnt = mnt;
> path->dentry = dget(mnt->mnt_root);
> return 0;
> default:
> return err;
> }
> chunk in follow_automount() - it would just be
> return finish_automount(mnt, path);
>
> Another thing (in the same area) is not a bug per se, but...
> after the call of ->d_automount() we have this:
> if (IS_ERR(mnt)) {
> /*
> * The filesystem is allowed to return -EISDIR here
> to indicate
> * it doesn't want to automount. For instance,
> autofs would do
> * this so that its userspace daemon can mount on
> this dentry.
> *
> * However, we can only permit this if it's a
> terminal point in
> * the path being looked up; if it wasn't then the
> remainder of
> * the path is inaccessible and we should say so.
> */
> if (PTR_ERR(mnt) == -EISDIR && (nd->flags &
> LOOKUP_PARENT))
> return -EREMOTE;
> return PTR_ERR(mnt);
> }
> Except that not a single instance of ->d_automount() has ever
> returned
> -EISDIR. Certainly not autofs one, despite the what the comment
> says.
> That chunk has come from dhowells, back when the whole mount trap
> series
> had been merged. After talking that thing over (fun: trying to
> figure
> out what had been intended nearly 9 years ago, when people involved
> are
> in UK, US east coast and AU west coast respectively. The only way it
> could suck more would've been if I were on the west coast - then all
> timezone deltas would be 8-hour ones)... looks like it's a rudiment
> of plans that got superseded during the series development, nobody
> quite remembers exact details. Conclusion: it's not even dead, it's
> stillborn; bury it.
Yeah, autofs ->d_automount() doesn't return -EISDIR, by the time
we get there it's not relevant any more, so that check looks
redundant. I'm not aware of any other fs automount implementation
that needs that EISDIR pass-thru function.
I didn't notice it at the time of the merge, sorry about that.
While we're at it that:
if (!path->dentry->d_op || !path->dentry->d_op->d_automount)
return -EREMOTE;
at the top of follow_automount() isn't going to be be relevant
for autofs because ->d_automount() really must always be defined
for it.
But, at the time of the merge, I didn't object to it because
there were (are) other file systems that use the VFS automount
function which may accidentally not define the method.
>
> Unfortunately, there are other interesting questions related to
> autofs-specific bits (->d_manage()) and the timezone-related fun
> is, of course, still there. I hope to sort that out today or
> tomorrow, at least enough to do a reasonable set of backportable
> fixes to put in front of follow_managed()/step_into() queue.
> Oh, well...
Yeah, I know it slows you down but I kink-off like having a chance
to look at what's going and think about your questions before trying
to answer them, rather than replying prematurely, as I usually do ...
It's been a bit of a busy day so far but I'm getting to look into
the questions you've asked.
Ian
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox