* [PATCH for v4.2 0/3] membarrier system call @ 2015-05-16 23:48 ` Mathieu Desnoyers 0 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-16 23:48 UTC (permalink / raw) To: Andrew Morton, Paul E . McKenney Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA, linux-api-u79uwXL29TY76Z2rM5mHXA, Mathieu Desnoyers Hi, At this point, all we're awaiting for is formal Acked-by by affected maintainers. Andrew, should you eventually pick it up into your tree ? Or perhaps it should go through Paul McKenney's tree, given that it uses synchronize_sched() ? The only change since last post is a layout change in the patch changelog, so I'm not even bumping the patch version (kept at v18). Thanks, Mathieu Mathieu Desnoyers (2): sys_membarrier(): system-wide memory barrier (generic, x86) (v18) selftests: enhance membarrier syscall test Pranith Kumar (1): selftests: add membarrier syscall test MAINTAINERS | 8 ++ arch/x86/syscalls/syscall_32.tbl | 1 + arch/x86/syscalls/syscall_64.tbl | 1 + include/linux/syscalls.h | 2 + include/uapi/asm-generic/unistd.h | 4 +- include/uapi/linux/Kbuild | 1 + include/uapi/linux/membarrier.h | 53 +++++++++ init/Kconfig | 12 ++ kernel/Makefile | 1 + kernel/membarrier.c | 66 +++++++++++ kernel/sys_ni.c | 3 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/membarrier/.gitignore | 1 + tools/testing/selftests/membarrier/Makefile | 11 ++ .../testing/selftests/membarrier/membarrier_test.c | 121 ++++++++++++++++++++ 15 files changed, 285 insertions(+), 1 deletions(-) create mode 100644 include/uapi/linux/membarrier.h create mode 100644 kernel/membarrier.c create mode 100644 tools/testing/selftests/membarrier/.gitignore create mode 100644 tools/testing/selftests/membarrier/Makefile create mode 100644 tools/testing/selftests/membarrier/membarrier_test.c -- 1.7.7.3 ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH for v4.2 0/3] membarrier system call @ 2015-05-16 23:48 ` Mathieu Desnoyers 0 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-16 23:48 UTC (permalink / raw) To: Andrew Morton, Paul E . McKenney Cc: linux-kernel, linux-api, Mathieu Desnoyers Hi, At this point, all we're awaiting for is formal Acked-by by affected maintainers. Andrew, should you eventually pick it up into your tree ? Or perhaps it should go through Paul McKenney's tree, given that it uses synchronize_sched() ? The only change since last post is a layout change in the patch changelog, so I'm not even bumping the patch version (kept at v18). Thanks, Mathieu Mathieu Desnoyers (2): sys_membarrier(): system-wide memory barrier (generic, x86) (v18) selftests: enhance membarrier syscall test Pranith Kumar (1): selftests: add membarrier syscall test MAINTAINERS | 8 ++ arch/x86/syscalls/syscall_32.tbl | 1 + arch/x86/syscalls/syscall_64.tbl | 1 + include/linux/syscalls.h | 2 + include/uapi/asm-generic/unistd.h | 4 +- include/uapi/linux/Kbuild | 1 + include/uapi/linux/membarrier.h | 53 +++++++++ init/Kconfig | 12 ++ kernel/Makefile | 1 + kernel/membarrier.c | 66 +++++++++++ kernel/sys_ni.c | 3 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/membarrier/.gitignore | 1 + tools/testing/selftests/membarrier/Makefile | 11 ++ .../testing/selftests/membarrier/membarrier_test.c | 121 ++++++++++++++++++++ 15 files changed, 285 insertions(+), 1 deletions(-) create mode 100644 include/uapi/linux/membarrier.h create mode 100644 kernel/membarrier.c create mode 100644 tools/testing/selftests/membarrier/.gitignore create mode 100644 tools/testing/selftests/membarrier/Makefile create mode 100644 tools/testing/selftests/membarrier/membarrier_test.c -- 1.7.7.3 ^ permalink raw reply [flat|nested] 10+ messages in thread
[parent not found: <1431820100-17040-1-git-send-email-mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org>]
* [PATCH for v4.2 v18 1/3] sys_membarrier(): system-wide memory barrier (generic, x86) 2015-05-16 23:48 ` Mathieu Desnoyers @ 2015-05-16 23:48 ` Mathieu Desnoyers -1 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-16 23:48 UTC (permalink / raw) To: Andrew Morton, Paul E . McKenney Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA, linux-api-u79uwXL29TY76Z2rM5mHXA, Mathieu Desnoyers, KOSAKI Motohiro, Steven Rostedt, Nicholas Miell, Linus Torvalds, Ingo Molnar, Alan Cox, Lai Jiangshan, Stephen Hemminger, Thomas Gleixner, Peter Zijlstra, David Howells, Pranith Kumar, Michael Kerrisk Here is an implementation of a new system call, sys_membarrier(), which executes a memory barrier on all threads running on the system. It is implemented by calling synchronize_sched(). It can be used to distribute the cost of user-space memory barriers asymmetrically by transforming pairs of memory barriers into pairs consisting of sys_membarrier() and a compiler barrier. For synchronization primitives that distinguish between read-side and write-side (e.g. userspace RCU [1], rwlocks), the read-side can be accelerated significantly by moving the bulk of the memory barrier overhead to the write-side. It is based on kernel v4.1-rc2. To explain the benefit of this scheme, let's introduce two example threads: Thread A (non-frequent, e.g. executing liburcu synchronize_rcu()) Thread B (frequent, e.g. executing liburcu rcu_read_lock()/rcu_read_unlock()) In a scheme where all smp_mb() in thread A are ordering memory accesses with respect to smp_mb() present in Thread B, we can change each smp_mb() within Thread A into calls to sys_membarrier() and each smp_mb() within Thread B into compiler barriers "barrier()". Before the change, we had, for each smp_mb() pairs: Thread A Thread B previous mem accesses previous mem accesses smp_mb() smp_mb() following mem accesses following mem accesses After the change, these pairs become: Thread A Thread B prev mem accesses prev mem accesses sys_membarrier() barrier() follow mem accesses follow mem accesses As we can see, there are two possible scenarios: either Thread B memory accesses do not happen concurrently with Thread A accesses (1), or they do (2). 1) Non-concurrent Thread A vs Thread B accesses: Thread A Thread B prev mem accesses sys_membarrier() follow mem accesses prev mem accesses barrier() follow mem accesses In this case, thread B accesses will be weakly ordered. This is OK, because at that point, thread A is not particularly interested in ordering them with respect to its own accesses. 2) Concurrent Thread A vs Thread B accesses Thread A Thread B prev mem accesses prev mem accesses sys_membarrier() barrier() follow mem accesses follow mem accesses In this case, thread B accesses, which are ensured to be in program order thanks to the compiler barrier, will be "upgraded" to full smp_mb() by synchronize_sched(). * Benchmarks On Intel Xeon E5405 (8 cores) (one thread is calling sys_membarrier, the other 7 threads are busy looping) 1000 non-expedited sys_membarrier calls in 33s = 33 milliseconds/call. * User-space user of this system call: Userspace RCU library Both the signal-based and the sys_membarrier userspace RCU schemes permit us to remove the memory barrier from the userspace RCU rcu_read_lock() and rcu_read_unlock() primitives, thus significantly accelerating them. These memory barriers are replaced by compiler barriers on the read-side, and all matching memory barriers on the write-side are turned into an invocation of a memory barrier on all active threads in the process. By letting the kernel perform this synchronization rather than dumbly sending a signal to every process threads (as we currently do), we diminish the number of unnecessary wake ups and only issue the memory barriers on active threads. Non-running threads do not need to execute such barrier anyway, because these are implied by the scheduler context switches. Results in liburcu: Operations in 10s, 6 readers, 2 writers: memory barriers in reader: 1701557485 reads, 2202847 writes signal-based scheme: 9830061167 reads, 6700 writes sys_membarrier: 9952759104 reads, 425 writes sys_membarrier (dyn. check): 7970328887 reads, 425 writes The dynamic sys_membarrier availability check adds some overhead to the read-side compared to the signal-based scheme, but besides that, sys_membarrier slightly outperforms the signal-based scheme. However, this non-expedited sys_membarrier implementation has a much slower grace period than signal and memory barrier schemes. Besides diminishing the number of wake-ups, one major advantage of the membarrier system call over the signal-based scheme is that it does not need to reserve a signal. This plays much more nicely with libraries, and with processes injected into for tracing purposes, for which we cannot expect that signals will be unused by the application. An expedited version of this system call can be added later on to speed up the grace period. Its implementation will likely depend on reading the cpu_curr()->mm without holding each CPU's rq lock. This patch adds the system call to x86 and to asm-generic. [1] http://urcu.so Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org> Reviewed-by: Paul E. McKenney <paulmck-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> Reviewed-by: Josh Triplett <josh-iaAMLnmF4UmaiuxdJuQwMA@public.gmane.org> CC: KOSAKI Motohiro <kosaki.motohiro-+CUm20s59erQFUHtdCDX3A@public.gmane.org> CC: Steven Rostedt <rostedt-nx8X9YLhiw1AfugRpC6u6w@public.gmane.org> CC: Nicholas Miell <nmiell-Wuw85uim5zDR7s880joybQ@public.gmane.org> CC: Linus Torvalds <torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> CC: Ingo Molnar <mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> CC: Alan Cox <gnomes-qBU/x9rampVanCEyBjwyrvXRex20P6io@public.gmane.org> CC: Lai Jiangshan <laijs-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> CC: Stephen Hemminger <stephen-OTpzqLSitTUnbdJkjeBofR2eb7JE58TQ@public.gmane.org> CC: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> CC: Thomas Gleixner <tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org> CC: Peter Zijlstra <peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> CC: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> CC: Pranith Kumar <bobby.prani-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> CC: Michael Kerrisk <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> CC: linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org --- membarrier(2) man page: --------------- snip ------------------- MEMBARRIER(2) Linux Programmer's Manual MEMBARRIER(2) NAME membarrier - issue memory barriers on a set of threads SYNOPSIS #include <linux/membarrier.h> int membarrier(int cmd, int flags); DESCRIPTION The cmd argument is one of the following: MEMBARRIER_CMD_QUERY Query the set of supported commands. It returns a bitmask of supported commands. MEMBARRIER_CMD_SHARED Execute a memory barrier on all threads running on the system. Upon return from system call, the caller thread is ensured that all running threads have passed through a state where all memory accesses to user-space addresses match program order between entry to and return from the system call (non-running threads are de facto in such a state). This covers threads from all pro‐ cesses running on the system. This command returns 0. The flags argument needs to be 0. For future extensions. All memory accesses performed in program order from each targeted thread is guaranteed to be ordered with respect to sys_membarrier(). If we use the semantic "barrier()" to represent a compiler barrier forcing memory accesses to be performed in program order across the barrier, and smp_mb() to represent explicit memory barriers forcing full memory ordering across the barrier, we have the following ordering table for each pair of barrier(), sys_membarrier() and smp_mb(): The pair ordering is detailed as (O: ordered, X: not ordered): barrier() smp_mb() sys_membarrier() barrier() X X O smp_mb() X O O sys_membarrier() O O O RETURN VALUE On success, these system calls return zero. On error, -1 is returned, and errno is set appropriately. For a given command, with flags argument set to 0, this system call is guaranteed to always return the same value until reboot. ERRORS ENOSYS System call is not implemented. EINVAL Invalid arguments. Linux 2015-04-15 MEMBARRIER(2) --------------- snip ------------------- Changes since v17: - Update commit message. Changes since v16: - Update documentation. - Add man page to changelog. - Build sys_membarrier on !CONFIG_SMP. It allows userspace applications to not care about the number of processors on the system. Based on recommendations from Stephen Hemminger and Steven Rostedt. - Check that flags argument is 0, update documentation to require it. Changes since v15: - Add flags argument in addition to cmd. - Update documentation. Changes since v14: - Take care of Thomas Gleixner's comments. Changes since v13: - Move to kernel/membarrier.c. - Remove MEMBARRIER_PRIVATE flag. - Add MAINTAINERS file entry. Changes since v12: - Remove _FLAG suffix from uapi flags. - Add Expert menuconfig option CONFIG_MEMBARRIER (default=y). - Remove EXPEDITED mode. Only implement non-expedited for now, until reading the cpu_curr()->mm can be done without holding the CPU's rq lock. Changes since v11: - 5 years have passed. - Rebase on v3.19 kernel. - Add futex-alike PRIVATE vs SHARED semantic: private for per-process barriers, non-private for memory mappings shared between processes. - Simplify user API. - Code refactoring. Changes since v10: - Apply Randy's comments. - Rebase on 2.6.34-rc4 -tip. Changes since v9: - Clean up #ifdef CONFIG_SMP. Changes since v8: - Go back to rq spin locks taken by sys_membarrier() rather than adding memory barriers to the scheduler. It implies a potential RoS (reduction of service) if sys_membarrier() is executed in a busy-loop by a user, but nothing more than what is already possible with other existing system calls, but saves memory barriers in the scheduler fast path. - re-add the memory barrier comments to x86 switch_mm() as an example to other architectures. - Update documentation of the memory barriers in sys_membarrier and switch_mm(). - Append execution scenarios to the changelog showing the purpose of each memory barrier. Changes since v7: - Move spinlock-mb and scheduler related changes to separate patches. - Add support for sys_membarrier on x86_32. - Only x86 32/64 system calls are reserved in this patch. It is planned to incrementally reserve syscall IDs on other architectures as these are tested. Changes since v6: - Remove some unlikely() not so unlikely. - Add the proper scheduler memory barriers needed to only use the RCU read lock in sys_membarrier rather than take each runqueue spinlock: - Move memory barriers from per-architecture switch_mm() to schedule() and finish_lock_switch(), where they clearly document that all data protected by the rq lock is guaranteed to have memory barriers issued between the scheduler update and the task execution. Replacing the spin lock acquire/release barriers with these memory barriers imply either no overhead (x86 spinlock atomic instruction already implies a full mb) or some hopefully small overhead caused by the upgrade of the spinlock acquire/release barriers to more heavyweight smp_mb(). - The "generic" version of spinlock-mb.h declares both a mapping to standard spinlocks and full memory barriers. Each architecture can specialize this header following their own need and declare CONFIG_HAVE_SPINLOCK_MB to use their own spinlock-mb.h. - Note: benchmarks of scheduler overhead with specialized spinlock-mb.h implementations on a wide range of architecture would be welcome. Changes since v5: - Plan ahead for extensibility by introducing mandatory/optional masks to the "flags" system call parameter. Past experience with accept4(), signalfd4(), eventfd2(), epoll_create1(), dup3(), pipe2(), and inotify_init1() indicates that this is the kind of thing we want to plan for. Return -EINVAL if the mandatory flags received are unknown. - Create include/linux/membarrier.h to define these flags. - Add MEMBARRIER_QUERY optional flag. Changes since v4: - Add "int expedited" parameter, use synchronize_sched() in the non-expedited case. Thanks to Lai Jiangshan for making us consider seriously using synchronize_sched() to provide the low-overhead membarrier scheme. - Check num_online_cpus() == 1, quickly return without doing nothing. Changes since v3a: - Confirm that each CPU indeed runs the current task's ->mm before sending an IPI. Ensures that we do not disturb RT tasks in the presence of lazy TLB shootdown. - Document memory barriers needed in switch_mm(). - Surround helper functions with #ifdef CONFIG_SMP. Changes since v2: - simply send-to-many to the mm_cpumask. It contains the list of processors we have to IPI to (which use the mm), and this mask is updated atomically. Changes since v1: - Only perform the IPI in CONFIG_SMP. - Only perform the IPI if the process has more than one thread. - Only send IPIs to CPUs involved with threads belonging to our process. - Adaptative IPI scheme (single vs many IPI with threshold). - Issue smp_mb() at the beginning and end of the system call. --- MAINTAINERS | 8 ++++ arch/x86/syscalls/syscall_32.tbl | 1 + arch/x86/syscalls/syscall_64.tbl | 1 + include/linux/syscalls.h | 2 + include/uapi/asm-generic/unistd.h | 4 ++- include/uapi/linux/Kbuild | 1 + include/uapi/linux/membarrier.h | 53 +++++++++++++++++++++++++++++ init/Kconfig | 12 +++++++ kernel/Makefile | 1 + kernel/membarrier.c | 66 +++++++++++++++++++++++++++++++++++++ kernel/sys_ni.c | 3 ++ 11 files changed, 151 insertions(+), 1 deletions(-) create mode 100644 include/uapi/linux/membarrier.h create mode 100644 kernel/membarrier.c diff --git a/MAINTAINERS b/MAINTAINERS index 781e099..fcb63d4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6370,6 +6370,14 @@ W: http://www.mellanox.com Q: http://patchwork.ozlabs.org/project/netdev/list/ F: drivers/net/ethernet/mellanox/mlx4/en_* +MEMBARRIER SUPPORT +M: Mathieu Desnoyers <mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org> +M: "Paul E. McKenney" <paulmck-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> +L: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org +S: Supported +F: kernel/membarrier.c +F: include/uapi/linux/membarrier.h + MEMORY MANAGEMENT L: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org W: http://www.linux-mm.org diff --git a/arch/x86/syscalls/syscall_32.tbl b/arch/x86/syscalls/syscall_32.tbl index ef8187f..e63ad61 100644 --- a/arch/x86/syscalls/syscall_32.tbl +++ b/arch/x86/syscalls/syscall_32.tbl @@ -365,3 +365,4 @@ 356 i386 memfd_create sys_memfd_create 357 i386 bpf sys_bpf 358 i386 execveat sys_execveat stub32_execveat +359 i386 membarrier sys_membarrier diff --git a/arch/x86/syscalls/syscall_64.tbl b/arch/x86/syscalls/syscall_64.tbl index 9ef32d5..87f3cd6 100644 --- a/arch/x86/syscalls/syscall_64.tbl +++ b/arch/x86/syscalls/syscall_64.tbl @@ -329,6 +329,7 @@ 320 common kexec_file_load sys_kexec_file_load 321 common bpf sys_bpf 322 64 execveat stub_execveat +323 common membarrier sys_membarrier # # x32-specific system call numbers start at 512 to avoid cache impact diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index 76d1e38..51a9054 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -884,4 +884,6 @@ asmlinkage long sys_execveat(int dfd, const char __user *filename, const char __user *const __user *argv, const char __user *const __user *envp, int flags); +asmlinkage long sys_membarrier(int cmd, int flags); + #endif diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h index e016bd9..8da542a 100644 --- a/include/uapi/asm-generic/unistd.h +++ b/include/uapi/asm-generic/unistd.h @@ -709,9 +709,11 @@ __SYSCALL(__NR_memfd_create, sys_memfd_create) __SYSCALL(__NR_bpf, sys_bpf) #define __NR_execveat 281 __SC_COMP(__NR_execveat, sys_execveat, compat_sys_execveat) +#define __NR_membarrier 282 +__SYSCALL(__NR_membarrier, sys_membarrier) #undef __NR_syscalls -#define __NR_syscalls 282 +#define __NR_syscalls 283 /* * All syscalls below here should go away really, diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild index 1a0006a..7bcc827 100644 --- a/include/uapi/linux/Kbuild +++ b/include/uapi/linux/Kbuild @@ -250,6 +250,7 @@ header-y += mdio.h header-y += media.h header-y += media-bus-format.h header-y += mei.h +header-y += membarrier.h header-y += memfd.h header-y += mempolicy.h header-y += meye.h diff --git a/include/uapi/linux/membarrier.h b/include/uapi/linux/membarrier.h new file mode 100644 index 0000000..e0b108b --- /dev/null +++ b/include/uapi/linux/membarrier.h @@ -0,0 +1,53 @@ +#ifndef _UAPI_LINUX_MEMBARRIER_H +#define _UAPI_LINUX_MEMBARRIER_H + +/* + * linux/membarrier.h + * + * membarrier system call API + * + * Copyright (c) 2010, 2015 Mathieu Desnoyers <mathieu.desnoyers@efficios.com> + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +/** + * enum membarrier_cmd - membarrier system call command + * @MEMBARRIER_CMD_QUERY: Query the set of supported commands. It returns + * a bitmask of valid commands. + * @MEMBARRIER_CMD_SHARED: Execute a memory barrier on all running threads. + * Upon return from system call, the caller thread + * is ensured that all running threads have passed + * through a state where all memory accesses to + * user-space addresses match program order between + * entry to and return from the system call + * (non-running threads are de facto in such a + * state). This covers threads from all processes + * running on the system. This command returns 0. + * + * Command to be passed to the membarrier system call. The commands need to + * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to + * the value 0. + */ +enum membarrier_cmd { + MEMBARRIER_CMD_QUERY = 0, + MEMBARRIER_CMD_SHARED = (1 << 0), +}; + +#endif /* _UAPI_LINUX_MEMBARRIER_H */ diff --git a/init/Kconfig b/init/Kconfig index dc24dec..307e406 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1583,6 +1583,18 @@ config PCI_QUIRKS bugs/quirks. Disable this only if your target machine is unaffected by PCI quirks. +config MEMBARRIER + bool "Enable membarrier() system call" if EXPERT + default y + help + Enable the membarrier() system call that allows issuing memory + barriers across all running threads, which can be used to distribute + the cost of user-space memory barriers asymmetrically by transforming + pairs of memory barriers into pairs consisting of membarrier() and a + compiler barrier. + + If unsure, say Y. + config EMBEDDED bool "Embedded system" option allnoconfig_y diff --git a/kernel/Makefile b/kernel/Makefile index 60c302c..05191fd 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -98,6 +98,7 @@ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o obj-$(CONFIG_JUMP_LABEL) += jump_label.o obj-$(CONFIG_CONTEXT_TRACKING) += context_tracking.o obj-$(CONFIG_TORTURE_TEST) += torture.o +obj-$(CONFIG_MEMBARRIER) += membarrier.o $(obj)/configs.o: $(obj)/config_data.h diff --git a/kernel/membarrier.c b/kernel/membarrier.c new file mode 100644 index 0000000..a20b279 --- /dev/null +++ b/kernel/membarrier.c @@ -0,0 +1,66 @@ +/* + * Copyright (C) 2010, 2015 Mathieu Desnoyers <mathieu.desnoyers@efficios.com> + * + * membarrier system call + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include <linux/syscalls.h> +#include <linux/membarrier.h> + +/* + * Bitmask made from a "or" of all commands within enum membarrier_cmd, + * except MEMBARRIER_CMD_QUERY. + */ +#define MEMBARRIER_CMD_BITMASK (MEMBARRIER_CMD_SHARED) + +/** + * sys_membarrier - issue memory barriers on a set of threads + * @cmd: Takes command values defined in enum membarrier_cmd. + * @flags: Currently needs to be 0. For future extensions. + * + * If this system call is not implemented, -ENOSYS is returned. If the + * command specified does not exist, or if the command argument is invalid, + * this system call returns -EINVAL. For a given command, with flags argument + * set to 0, this system call is guaranteed to always return the same value + * until reboot. + * + * All memory accesses performed in program order from each targeted thread + * is guaranteed to be ordered with respect to sys_membarrier(). If we use + * the semantic "barrier()" to represent a compiler barrier forcing memory + * accesses to be performed in program order across the barrier, and + * smp_mb() to represent explicit memory barriers forcing full memory + * ordering across the barrier, we have the following ordering table for + * each pair of barrier(), sys_membarrier() and smp_mb(): + * + * The pair ordering is detailed as (O: ordered, X: not ordered): + * + * barrier() smp_mb() sys_membarrier() + * barrier() X X O + * smp_mb() X O O + * sys_membarrier() O O O + */ +SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) +{ + if (flags) + return -EINVAL; + switch (cmd) { + case MEMBARRIER_CMD_QUERY: + return MEMBARRIER_CMD_BITMASK; + case MEMBARRIER_CMD_SHARED: + if (num_online_cpus() > 1) + synchronize_sched(); + return 0; + default: + return -EINVAL; + } +} diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c index 7995ef5..eb4fde0 100644 --- a/kernel/sys_ni.c +++ b/kernel/sys_ni.c @@ -243,3 +243,6 @@ cond_syscall(sys_bpf); /* execveat */ cond_syscall(sys_execveat); + +/* membarrier */ +cond_syscall(sys_membarrier); -- 1.7.7.3 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH for v4.2 v18 1/3] sys_membarrier(): system-wide memory barrier (generic, x86) @ 2015-05-16 23:48 ` Mathieu Desnoyers 0 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-16 23:48 UTC (permalink / raw) To: Andrew Morton, Paul E . McKenney Cc: linux-kernel, linux-api, Mathieu Desnoyers, KOSAKI Motohiro, Steven Rostedt, Nicholas Miell, Linus Torvalds, Ingo Molnar, Alan Cox, Lai Jiangshan, Stephen Hemminger, Thomas Gleixner, Peter Zijlstra, David Howells, Pranith Kumar, Michael Kerrisk Here is an implementation of a new system call, sys_membarrier(), which executes a memory barrier on all threads running on the system. It is implemented by calling synchronize_sched(). It can be used to distribute the cost of user-space memory barriers asymmetrically by transforming pairs of memory barriers into pairs consisting of sys_membarrier() and a compiler barrier. For synchronization primitives that distinguish between read-side and write-side (e.g. userspace RCU [1], rwlocks), the read-side can be accelerated significantly by moving the bulk of the memory barrier overhead to the write-side. It is based on kernel v4.1-rc2. To explain the benefit of this scheme, let's introduce two example threads: Thread A (non-frequent, e.g. executing liburcu synchronize_rcu()) Thread B (frequent, e.g. executing liburcu rcu_read_lock()/rcu_read_unlock()) In a scheme where all smp_mb() in thread A are ordering memory accesses with respect to smp_mb() present in Thread B, we can change each smp_mb() within Thread A into calls to sys_membarrier() and each smp_mb() within Thread B into compiler barriers "barrier()". Before the change, we had, for each smp_mb() pairs: Thread A Thread B previous mem accesses previous mem accesses smp_mb() smp_mb() following mem accesses following mem accesses After the change, these pairs become: Thread A Thread B prev mem accesses prev mem accesses sys_membarrier() barrier() follow mem accesses follow mem accesses As we can see, there are two possible scenarios: either Thread B memory accesses do not happen concurrently with Thread A accesses (1), or they do (2). 1) Non-concurrent Thread A vs Thread B accesses: Thread A Thread B prev mem accesses sys_membarrier() follow mem accesses prev mem accesses barrier() follow mem accesses In this case, thread B accesses will be weakly ordered. This is OK, because at that point, thread A is not particularly interested in ordering them with respect to its own accesses. 2) Concurrent Thread A vs Thread B accesses Thread A Thread B prev mem accesses prev mem accesses sys_membarrier() barrier() follow mem accesses follow mem accesses In this case, thread B accesses, which are ensured to be in program order thanks to the compiler barrier, will be "upgraded" to full smp_mb() by synchronize_sched(). * Benchmarks On Intel Xeon E5405 (8 cores) (one thread is calling sys_membarrier, the other 7 threads are busy looping) 1000 non-expedited sys_membarrier calls in 33s = 33 milliseconds/call. * User-space user of this system call: Userspace RCU library Both the signal-based and the sys_membarrier userspace RCU schemes permit us to remove the memory barrier from the userspace RCU rcu_read_lock() and rcu_read_unlock() primitives, thus significantly accelerating them. These memory barriers are replaced by compiler barriers on the read-side, and all matching memory barriers on the write-side are turned into an invocation of a memory barrier on all active threads in the process. By letting the kernel perform this synchronization rather than dumbly sending a signal to every process threads (as we currently do), we diminish the number of unnecessary wake ups and only issue the memory barriers on active threads. Non-running threads do not need to execute such barrier anyway, because these are implied by the scheduler context switches. Results in liburcu: Operations in 10s, 6 readers, 2 writers: memory barriers in reader: 1701557485 reads, 2202847 writes signal-based scheme: 9830061167 reads, 6700 writes sys_membarrier: 9952759104 reads, 425 writes sys_membarrier (dyn. check): 7970328887 reads, 425 writes The dynamic sys_membarrier availability check adds some overhead to the read-side compared to the signal-based scheme, but besides that, sys_membarrier slightly outperforms the signal-based scheme. However, this non-expedited sys_membarrier implementation has a much slower grace period than signal and memory barrier schemes. Besides diminishing the number of wake-ups, one major advantage of the membarrier system call over the signal-based scheme is that it does not need to reserve a signal. This plays much more nicely with libraries, and with processes injected into for tracing purposes, for which we cannot expect that signals will be unused by the application. An expedited version of this system call can be added later on to speed up the grace period. Its implementation will likely depend on reading the cpu_curr()->mm without holding each CPU's rq lock. This patch adds the system call to x86 and to asm-generic. [1] http://urcu.so Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Reviewed-by: Josh Triplett <josh@joshtriplett.org> CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> CC: Steven Rostedt <rostedt@goodmis.org> CC: Nicholas Miell <nmiell@comcast.net> CC: Linus Torvalds <torvalds@linux-foundation.org> CC: Ingo Molnar <mingo@redhat.com> CC: Alan Cox <gnomes@lxorguk.ukuu.org.uk> CC: Lai Jiangshan <laijs@cn.fujitsu.com> CC: Stephen Hemminger <stephen@networkplumber.org> CC: Andrew Morton <akpm@linux-foundation.org> CC: Thomas Gleixner <tglx@linutronix.de> CC: Peter Zijlstra <peterz@infradead.org> CC: David Howells <dhowells@redhat.com> CC: Pranith Kumar <bobby.prani@gmail.com> CC: Michael Kerrisk <mtk.manpages@gmail.com> CC: linux-api@vger.kernel.org --- membarrier(2) man page: --------------- snip ------------------- MEMBARRIER(2) Linux Programmer's Manual MEMBARRIER(2) NAME membarrier - issue memory barriers on a set of threads SYNOPSIS #include <linux/membarrier.h> int membarrier(int cmd, int flags); DESCRIPTION The cmd argument is one of the following: MEMBARRIER_CMD_QUERY Query the set of supported commands. It returns a bitmask of supported commands. MEMBARRIER_CMD_SHARED Execute a memory barrier on all threads running on the system. Upon return from system call, the caller thread is ensured that all running threads have passed through a state where all memory accesses to user-space addresses match program order between entry to and return from the system call (non-running threads are de facto in such a state). This covers threads from all pro‐ cesses running on the system. This command returns 0. The flags argument needs to be 0. For future extensions. All memory accesses performed in program order from each targeted thread is guaranteed to be ordered with respect to sys_membarrier(). If we use the semantic "barrier()" to represent a compiler barrier forcing memory accesses to be performed in program order across the barrier, and smp_mb() to represent explicit memory barriers forcing full memory ordering across the barrier, we have the following ordering table for each pair of barrier(), sys_membarrier() and smp_mb(): The pair ordering is detailed as (O: ordered, X: not ordered): barrier() smp_mb() sys_membarrier() barrier() X X O smp_mb() X O O sys_membarrier() O O O RETURN VALUE On success, these system calls return zero. On error, -1 is returned, and errno is set appropriately. For a given command, with flags argument set to 0, this system call is guaranteed to always return the same value until reboot. ERRORS ENOSYS System call is not implemented. EINVAL Invalid arguments. Linux 2015-04-15 MEMBARRIER(2) --------------- snip ------------------- Changes since v17: - Update commit message. Changes since v16: - Update documentation. - Add man page to changelog. - Build sys_membarrier on !CONFIG_SMP. It allows userspace applications to not care about the number of processors on the system. Based on recommendations from Stephen Hemminger and Steven Rostedt. - Check that flags argument is 0, update documentation to require it. Changes since v15: - Add flags argument in addition to cmd. - Update documentation. Changes since v14: - Take care of Thomas Gleixner's comments. Changes since v13: - Move to kernel/membarrier.c. - Remove MEMBARRIER_PRIVATE flag. - Add MAINTAINERS file entry. Changes since v12: - Remove _FLAG suffix from uapi flags. - Add Expert menuconfig option CONFIG_MEMBARRIER (default=y). - Remove EXPEDITED mode. Only implement non-expedited for now, until reading the cpu_curr()->mm can be done without holding the CPU's rq lock. Changes since v11: - 5 years have passed. - Rebase on v3.19 kernel. - Add futex-alike PRIVATE vs SHARED semantic: private for per-process barriers, non-private for memory mappings shared between processes. - Simplify user API. - Code refactoring. Changes since v10: - Apply Randy's comments. - Rebase on 2.6.34-rc4 -tip. Changes since v9: - Clean up #ifdef CONFIG_SMP. Changes since v8: - Go back to rq spin locks taken by sys_membarrier() rather than adding memory barriers to the scheduler. It implies a potential RoS (reduction of service) if sys_membarrier() is executed in a busy-loop by a user, but nothing more than what is already possible with other existing system calls, but saves memory barriers in the scheduler fast path. - re-add the memory barrier comments to x86 switch_mm() as an example to other architectures. - Update documentation of the memory barriers in sys_membarrier and switch_mm(). - Append execution scenarios to the changelog showing the purpose of each memory barrier. Changes since v7: - Move spinlock-mb and scheduler related changes to separate patches. - Add support for sys_membarrier on x86_32. - Only x86 32/64 system calls are reserved in this patch. It is planned to incrementally reserve syscall IDs on other architectures as these are tested. Changes since v6: - Remove some unlikely() not so unlikely. - Add the proper scheduler memory barriers needed to only use the RCU read lock in sys_membarrier rather than take each runqueue spinlock: - Move memory barriers from per-architecture switch_mm() to schedule() and finish_lock_switch(), where they clearly document that all data protected by the rq lock is guaranteed to have memory barriers issued between the scheduler update and the task execution. Replacing the spin lock acquire/release barriers with these memory barriers imply either no overhead (x86 spinlock atomic instruction already implies a full mb) or some hopefully small overhead caused by the upgrade of the spinlock acquire/release barriers to more heavyweight smp_mb(). - The "generic" version of spinlock-mb.h declares both a mapping to standard spinlocks and full memory barriers. Each architecture can specialize this header following their own need and declare CONFIG_HAVE_SPINLOCK_MB to use their own spinlock-mb.h. - Note: benchmarks of scheduler overhead with specialized spinlock-mb.h implementations on a wide range of architecture would be welcome. Changes since v5: - Plan ahead for extensibility by introducing mandatory/optional masks to the "flags" system call parameter. Past experience with accept4(), signalfd4(), eventfd2(), epoll_create1(), dup3(), pipe2(), and inotify_init1() indicates that this is the kind of thing we want to plan for. Return -EINVAL if the mandatory flags received are unknown. - Create include/linux/membarrier.h to define these flags. - Add MEMBARRIER_QUERY optional flag. Changes since v4: - Add "int expedited" parameter, use synchronize_sched() in the non-expedited case. Thanks to Lai Jiangshan for making us consider seriously using synchronize_sched() to provide the low-overhead membarrier scheme. - Check num_online_cpus() == 1, quickly return without doing nothing. Changes since v3a: - Confirm that each CPU indeed runs the current task's ->mm before sending an IPI. Ensures that we do not disturb RT tasks in the presence of lazy TLB shootdown. - Document memory barriers needed in switch_mm(). - Surround helper functions with #ifdef CONFIG_SMP. Changes since v2: - simply send-to-many to the mm_cpumask. It contains the list of processors we have to IPI to (which use the mm), and this mask is updated atomically. Changes since v1: - Only perform the IPI in CONFIG_SMP. - Only perform the IPI if the process has more than one thread. - Only send IPIs to CPUs involved with threads belonging to our process. - Adaptative IPI scheme (single vs many IPI with threshold). - Issue smp_mb() at the beginning and end of the system call. --- MAINTAINERS | 8 ++++ arch/x86/syscalls/syscall_32.tbl | 1 + arch/x86/syscalls/syscall_64.tbl | 1 + include/linux/syscalls.h | 2 + include/uapi/asm-generic/unistd.h | 4 ++- include/uapi/linux/Kbuild | 1 + include/uapi/linux/membarrier.h | 53 +++++++++++++++++++++++++++++ init/Kconfig | 12 +++++++ kernel/Makefile | 1 + kernel/membarrier.c | 66 +++++++++++++++++++++++++++++++++++++ kernel/sys_ni.c | 3 ++ 11 files changed, 151 insertions(+), 1 deletions(-) create mode 100644 include/uapi/linux/membarrier.h create mode 100644 kernel/membarrier.c diff --git a/MAINTAINERS b/MAINTAINERS index 781e099..fcb63d4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6370,6 +6370,14 @@ W: http://www.mellanox.com Q: http://patchwork.ozlabs.org/project/netdev/list/ F: drivers/net/ethernet/mellanox/mlx4/en_* +MEMBARRIER SUPPORT +M: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> +M: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> +L: linux-kernel@vger.kernel.org +S: Supported +F: kernel/membarrier.c +F: include/uapi/linux/membarrier.h + MEMORY MANAGEMENT L: linux-mm@kvack.org W: http://www.linux-mm.org diff --git a/arch/x86/syscalls/syscall_32.tbl b/arch/x86/syscalls/syscall_32.tbl index ef8187f..e63ad61 100644 --- a/arch/x86/syscalls/syscall_32.tbl +++ b/arch/x86/syscalls/syscall_32.tbl @@ -365,3 +365,4 @@ 356 i386 memfd_create sys_memfd_create 357 i386 bpf sys_bpf 358 i386 execveat sys_execveat stub32_execveat +359 i386 membarrier sys_membarrier diff --git a/arch/x86/syscalls/syscall_64.tbl b/arch/x86/syscalls/syscall_64.tbl index 9ef32d5..87f3cd6 100644 --- a/arch/x86/syscalls/syscall_64.tbl +++ b/arch/x86/syscalls/syscall_64.tbl @@ -329,6 +329,7 @@ 320 common kexec_file_load sys_kexec_file_load 321 common bpf sys_bpf 322 64 execveat stub_execveat +323 common membarrier sys_membarrier # # x32-specific system call numbers start at 512 to avoid cache impact diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index 76d1e38..51a9054 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -884,4 +884,6 @@ asmlinkage long sys_execveat(int dfd, const char __user *filename, const char __user *const __user *argv, const char __user *const __user *envp, int flags); +asmlinkage long sys_membarrier(int cmd, int flags); + #endif diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h index e016bd9..8da542a 100644 --- a/include/uapi/asm-generic/unistd.h +++ b/include/uapi/asm-generic/unistd.h @@ -709,9 +709,11 @@ __SYSCALL(__NR_memfd_create, sys_memfd_create) __SYSCALL(__NR_bpf, sys_bpf) #define __NR_execveat 281 __SC_COMP(__NR_execveat, sys_execveat, compat_sys_execveat) +#define __NR_membarrier 282 +__SYSCALL(__NR_membarrier, sys_membarrier) #undef __NR_syscalls -#define __NR_syscalls 282 +#define __NR_syscalls 283 /* * All syscalls below here should go away really, diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild index 1a0006a..7bcc827 100644 --- a/include/uapi/linux/Kbuild +++ b/include/uapi/linux/Kbuild @@ -250,6 +250,7 @@ header-y += mdio.h header-y += media.h header-y += media-bus-format.h header-y += mei.h +header-y += membarrier.h header-y += memfd.h header-y += mempolicy.h header-y += meye.h diff --git a/include/uapi/linux/membarrier.h b/include/uapi/linux/membarrier.h new file mode 100644 index 0000000..e0b108b --- /dev/null +++ b/include/uapi/linux/membarrier.h @@ -0,0 +1,53 @@ +#ifndef _UAPI_LINUX_MEMBARRIER_H +#define _UAPI_LINUX_MEMBARRIER_H + +/* + * linux/membarrier.h + * + * membarrier system call API + * + * Copyright (c) 2010, 2015 Mathieu Desnoyers <mathieu.desnoyers@efficios.com> + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +/** + * enum membarrier_cmd - membarrier system call command + * @MEMBARRIER_CMD_QUERY: Query the set of supported commands. It returns + * a bitmask of valid commands. + * @MEMBARRIER_CMD_SHARED: Execute a memory barrier on all running threads. + * Upon return from system call, the caller thread + * is ensured that all running threads have passed + * through a state where all memory accesses to + * user-space addresses match program order between + * entry to and return from the system call + * (non-running threads are de facto in such a + * state). This covers threads from all processes + * running on the system. This command returns 0. + * + * Command to be passed to the membarrier system call. The commands need to + * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to + * the value 0. + */ +enum membarrier_cmd { + MEMBARRIER_CMD_QUERY = 0, + MEMBARRIER_CMD_SHARED = (1 << 0), +}; + +#endif /* _UAPI_LINUX_MEMBARRIER_H */ diff --git a/init/Kconfig b/init/Kconfig index dc24dec..307e406 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1583,6 +1583,18 @@ config PCI_QUIRKS bugs/quirks. Disable this only if your target machine is unaffected by PCI quirks. +config MEMBARRIER + bool "Enable membarrier() system call" if EXPERT + default y + help + Enable the membarrier() system call that allows issuing memory + barriers across all running threads, which can be used to distribute + the cost of user-space memory barriers asymmetrically by transforming + pairs of memory barriers into pairs consisting of membarrier() and a + compiler barrier. + + If unsure, say Y. + config EMBEDDED bool "Embedded system" option allnoconfig_y diff --git a/kernel/Makefile b/kernel/Makefile index 60c302c..05191fd 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -98,6 +98,7 @@ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o obj-$(CONFIG_JUMP_LABEL) += jump_label.o obj-$(CONFIG_CONTEXT_TRACKING) += context_tracking.o obj-$(CONFIG_TORTURE_TEST) += torture.o +obj-$(CONFIG_MEMBARRIER) += membarrier.o $(obj)/configs.o: $(obj)/config_data.h diff --git a/kernel/membarrier.c b/kernel/membarrier.c new file mode 100644 index 0000000..a20b279 --- /dev/null +++ b/kernel/membarrier.c @@ -0,0 +1,66 @@ +/* + * Copyright (C) 2010, 2015 Mathieu Desnoyers <mathieu.desnoyers@efficios.com> + * + * membarrier system call + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include <linux/syscalls.h> +#include <linux/membarrier.h> + +/* + * Bitmask made from a "or" of all commands within enum membarrier_cmd, + * except MEMBARRIER_CMD_QUERY. + */ +#define MEMBARRIER_CMD_BITMASK (MEMBARRIER_CMD_SHARED) + +/** + * sys_membarrier - issue memory barriers on a set of threads + * @cmd: Takes command values defined in enum membarrier_cmd. + * @flags: Currently needs to be 0. For future extensions. + * + * If this system call is not implemented, -ENOSYS is returned. If the + * command specified does not exist, or if the command argument is invalid, + * this system call returns -EINVAL. For a given command, with flags argument + * set to 0, this system call is guaranteed to always return the same value + * until reboot. + * + * All memory accesses performed in program order from each targeted thread + * is guaranteed to be ordered with respect to sys_membarrier(). If we use + * the semantic "barrier()" to represent a compiler barrier forcing memory + * accesses to be performed in program order across the barrier, and + * smp_mb() to represent explicit memory barriers forcing full memory + * ordering across the barrier, we have the following ordering table for + * each pair of barrier(), sys_membarrier() and smp_mb(): + * + * The pair ordering is detailed as (O: ordered, X: not ordered): + * + * barrier() smp_mb() sys_membarrier() + * barrier() X X O + * smp_mb() X O O + * sys_membarrier() O O O + */ +SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) +{ + if (flags) + return -EINVAL; + switch (cmd) { + case MEMBARRIER_CMD_QUERY: + return MEMBARRIER_CMD_BITMASK; + case MEMBARRIER_CMD_SHARED: + if (num_online_cpus() > 1) + synchronize_sched(); + return 0; + default: + return -EINVAL; + } +} diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c index 7995ef5..eb4fde0 100644 --- a/kernel/sys_ni.c +++ b/kernel/sys_ni.c @@ -243,3 +243,6 @@ cond_syscall(sys_bpf); /* execveat */ cond_syscall(sys_execveat); + +/* membarrier */ +cond_syscall(sys_membarrier); -- 1.7.7.3 ^ permalink raw reply related [flat|nested] 10+ messages in thread
[parent not found: <1431820100-17040-2-git-send-email-mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org>]
* Re: [PATCH for v4.2 v18 1/3] sys_membarrier(): system-wide memory barrier (generic, x86) 2015-05-16 23:48 ` Mathieu Desnoyers @ 2015-05-29 22:40 ` Andrew Morton -1 siblings, 0 replies; 10+ messages in thread From: Andrew Morton @ 2015-05-29 22:40 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Paul E . McKenney, linux-kernel-u79uwXL29TY76Z2rM5mHXA, linux-api-u79uwXL29TY76Z2rM5mHXA, KOSAKI Motohiro, Steven Rostedt, Nicholas Miell, Linus Torvalds, Ingo Molnar, Alan Cox, Lai Jiangshan, Stephen Hemminger, Thomas Gleixner, Peter Zijlstra, David Howells, Pranith Kumar, Michael Kerrisk On Sat, 16 May 2015 19:48:18 -0400 Mathieu Desnoyers <mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org> wrote: > Here is an implementation of a new system call, sys_membarrier(), which > executes a memory barrier on all threads running on the system. It is > implemented by calling synchronize_sched(). It can be used to distribute > the cost of user-space memory barriers asymmetrically by transforming > pairs of memory barriers into pairs consisting of sys_membarrier() and a > compiler barrier. For synchronization primitives that distinguish > between read-side and write-side (e.g. userspace RCU [1], rwlocks), the > read-side can be accelerated significantly by moving the bulk of the > memory barrier overhead to the write-side. > > ... > It would be nice to hear about the real world value of this syscall to our users. I'm seeing test results for a microbenchmark but so what. What actual applications or application classes are calling for this and what results can they expect to see? > > membarrier(2) man page: > --------------- snip ------------------- > MEMBARRIER(2) Linux Programmer's Manual MEMBARRIER(2) > > NAME > membarrier - issue memory barriers on a set of threads > > SYNOPSIS > #include <linux/membarrier.h> > > int membarrier(int cmd, int flags); > > DESCRIPTION > The cmd argument is one of the following: > > MEMBARRIER_CMD_QUERY > Query the set of supported commands. It returns a bitmask of > supported commands. > > MEMBARRIER_CMD_SHARED > Execute a memory barrier on all threads running on the system. > Upon return from system call, the caller thread is ensured that > all running threads have passed through a state where all memory > accesses to user-space addresses match program order between > entry to and return from the system call (non-running threads > are de facto in such a state). This covers threads from all pro___ > cesses running on the system. This command returns 0. > > The flags argument needs to be 0. For future extensions. > > All memory accesses performed in program order from each targeted > thread is guaranteed to be ordered with respect to sys_membarrier(). If > we use the semantic "barrier()" to represent a compiler barrier forcing > memory accesses to be performed in program order across the barrier, > and smp_mb() to represent explicit memory barriers forcing full memory > ordering across the barrier, we have the following ordering table for > each pair of barrier(), sys_membarrier() and smp_mb(): > > The pair ordering is detailed as (O: ordered, X: not ordered): > > barrier() smp_mb() sys_membarrier() > barrier() X X O > smp_mb() X O O > sys_membarrier() O O O > > RETURN VALUE > On success, these system calls return zero. On error, -1 is returned, > and errno is set appropriately. For a given command, with flags > argument set to 0, this system call is guaranteed to always return the > same value until reboot. I suggest "with flags argument set to MEMBARRIER_CMD_QUERY" here. > > ERRORS > ENOSYS System call is not implemented. > > EINVAL Invalid arguments. > > ... > > +SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > +{ > + if (flags) > + return -EINVAL; I'm not a huge fan of this "add a flags arg to syscalls" rule. Is there any realistic expectation that we'll ever *use* this thing? If not, why add it? You may as well put an unlikely() in there btw. > + switch (cmd) { > + case MEMBARRIER_CMD_QUERY: > + return MEMBARRIER_CMD_BITMASK; > + case MEMBARRIER_CMD_SHARED: > + if (num_online_cpus() > 1) > + synchronize_sched(); > + return 0; > + default: > + return -EINVAL; > + } > +} ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH for v4.2 v18 1/3] sys_membarrier(): system-wide memory barrier (generic, x86) @ 2015-05-29 22:40 ` Andrew Morton 0 siblings, 0 replies; 10+ messages in thread From: Andrew Morton @ 2015-05-29 22:40 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Paul E . McKenney, linux-kernel, linux-api, KOSAKI Motohiro, Steven Rostedt, Nicholas Miell, Linus Torvalds, Ingo Molnar, Alan Cox, Lai Jiangshan, Stephen Hemminger, Thomas Gleixner, Peter Zijlstra, David Howells, Pranith Kumar, Michael Kerrisk On Sat, 16 May 2015 19:48:18 -0400 Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > Here is an implementation of a new system call, sys_membarrier(), which > executes a memory barrier on all threads running on the system. It is > implemented by calling synchronize_sched(). It can be used to distribute > the cost of user-space memory barriers asymmetrically by transforming > pairs of memory barriers into pairs consisting of sys_membarrier() and a > compiler barrier. For synchronization primitives that distinguish > between read-side and write-side (e.g. userspace RCU [1], rwlocks), the > read-side can be accelerated significantly by moving the bulk of the > memory barrier overhead to the write-side. > > ... > It would be nice to hear about the real world value of this syscall to our users. I'm seeing test results for a microbenchmark but so what. What actual applications or application classes are calling for this and what results can they expect to see? > > membarrier(2) man page: > --------------- snip ------------------- > MEMBARRIER(2) Linux Programmer's Manual MEMBARRIER(2) > > NAME > membarrier - issue memory barriers on a set of threads > > SYNOPSIS > #include <linux/membarrier.h> > > int membarrier(int cmd, int flags); > > DESCRIPTION > The cmd argument is one of the following: > > MEMBARRIER_CMD_QUERY > Query the set of supported commands. It returns a bitmask of > supported commands. > > MEMBARRIER_CMD_SHARED > Execute a memory barrier on all threads running on the system. > Upon return from system call, the caller thread is ensured that > all running threads have passed through a state where all memory > accesses to user-space addresses match program order between > entry to and return from the system call (non-running threads > are de facto in such a state). This covers threads from all pro___ > cesses running on the system. This command returns 0. > > The flags argument needs to be 0. For future extensions. > > All memory accesses performed in program order from each targeted > thread is guaranteed to be ordered with respect to sys_membarrier(). If > we use the semantic "barrier()" to represent a compiler barrier forcing > memory accesses to be performed in program order across the barrier, > and smp_mb() to represent explicit memory barriers forcing full memory > ordering across the barrier, we have the following ordering table for > each pair of barrier(), sys_membarrier() and smp_mb(): > > The pair ordering is detailed as (O: ordered, X: not ordered): > > barrier() smp_mb() sys_membarrier() > barrier() X X O > smp_mb() X O O > sys_membarrier() O O O > > RETURN VALUE > On success, these system calls return zero. On error, -1 is returned, > and errno is set appropriately. For a given command, with flags > argument set to 0, this system call is guaranteed to always return the > same value until reboot. I suggest "with flags argument set to MEMBARRIER_CMD_QUERY" here. > > ERRORS > ENOSYS System call is not implemented. > > EINVAL Invalid arguments. > > ... > > +SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > +{ > + if (flags) > + return -EINVAL; I'm not a huge fan of this "add a flags arg to syscalls" rule. Is there any realistic expectation that we'll ever *use* this thing? If not, why add it? You may as well put an unlikely() in there btw. > + switch (cmd) { > + case MEMBARRIER_CMD_QUERY: > + return MEMBARRIER_CMD_BITMASK; > + case MEMBARRIER_CMD_SHARED: > + if (num_online_cpus() > 1) > + synchronize_sched(); > + return 0; > + default: > + return -EINVAL; > + } > +} ^ permalink raw reply [flat|nested] 10+ messages in thread
[parent not found: <20150529154036.9695b6c153ed37aed3343f3b-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>]
* Re: [PATCH for v4.2 v18 1/3] sys_membarrier(): system-wide memory barrier (generic, x86) 2015-05-29 22:40 ` Andrew Morton @ 2015-05-31 12:53 ` Mathieu Desnoyers -1 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-31 12:53 UTC (permalink / raw) To: Andrew Morton Cc: Paul E. McKenney, linux-kernel-u79uwXL29TY76Z2rM5mHXA, linux-api, KOSAKI Motohiro, rostedt, Nicholas Miell, Linus Torvalds, Ingo Molnar, One Thousand Gnomes, Lai Jiangshan, Stephen Hemminger, Thomas Gleixner, Peter Zijlstra, David Howells, Pranith Kumar, Michael Kerrisk ----- On May 30, 2015, at 12:40 AM, Andrew Morton akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org wrote: > On Sat, 16 May 2015 19:48:18 -0400 Mathieu Desnoyers > <mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org> wrote: > >> Here is an implementation of a new system call, sys_membarrier(), which >> executes a memory barrier on all threads running on the system. It is >> implemented by calling synchronize_sched(). It can be used to distribute >> the cost of user-space memory barriers asymmetrically by transforming >> pairs of memory barriers into pairs consisting of sys_membarrier() and a >> compiler barrier. For synchronization primitives that distinguish >> between read-side and write-side (e.g. userspace RCU [1], rwlocks), the >> read-side can be accelerated significantly by moving the bulk of the >> memory barrier overhead to the write-side. >> >> ... >> > > It would be nice to hear about the real world value of this syscall to > our users. I'm seeing test results for a microbenchmark but so what. > What actual applications or application classes are calling for this and > what results can they expect to see? AFAIK, the existing open source applications that would be improved by this system call are as follows: * Through Userspace RCU library (http://urcu.so) - DNS server (Knot DNS) https://www.knot-dns.cz/ - Network sniffer (http://netsniff-ng.org/) - Distributed object storage (https://sheepdog.github.io/sheepdog/) - User-space tracing (http://lttng.org) - Network storage system (https://www.gluster.org/) Those projects use RCU in userspace to increase read-side speed and scalability compared to locking. Especially in the case of RCU used by libraries, sys_membarrier can speed up the read-side by moving the bulk of the memory barrier cost to synchronize_rcu(). * Direct users of sys_membarrier - core dotnet garbage collector (https://github.com/dotnet/coreclr/issues/198) Microsoft core dotnet GC developers are planning to use the mprotect() side-effect of issuing memory barriers through IPIs as a way to implement Windows FlushProcessWriteBuffers() on Linux. They are referring to sys_membarrier in their github thread, specifically stating that sys_membarrier() is what they are looking for. > >> >> membarrier(2) man page: >> --------------- snip ------------------- >> MEMBARRIER(2) Linux Programmer's Manual MEMBARRIER(2) >> >> NAME >> membarrier - issue memory barriers on a set of threads >> >> SYNOPSIS >> #include <linux/membarrier.h> >> >> int membarrier(int cmd, int flags); >> >> DESCRIPTION >> The cmd argument is one of the following: >> >> MEMBARRIER_CMD_QUERY >> Query the set of supported commands. It returns a bitmask of >> supported commands. >> >> MEMBARRIER_CMD_SHARED >> Execute a memory barrier on all threads running on the system. >> Upon return from system call, the caller thread is ensured that >> all running threads have passed through a state where all memory >> accesses to user-space addresses match program order between >> entry to and return from the system call (non-running threads >> are de facto in such a state). This covers threads from all pro___ >> cesses running on the system. This command returns 0. >> >> The flags argument needs to be 0. For future extensions. >> >> All memory accesses performed in program order from each targeted >> thread is guaranteed to be ordered with respect to sys_membarrier(). If >> we use the semantic "barrier()" to represent a compiler barrier forcing >> memory accesses to be performed in program order across the barrier, >> and smp_mb() to represent explicit memory barriers forcing full memory >> ordering across the barrier, we have the following ordering table for >> each pair of barrier(), sys_membarrier() and smp_mb(): >> >> The pair ordering is detailed as (O: ordered, X: not ordered): >> >> barrier() smp_mb() sys_membarrier() >> barrier() X X O >> smp_mb() X O O >> sys_membarrier() O O O >> >> RETURN VALUE >> On success, these system calls return zero. On error, -1 is returned, >> and errno is set appropriately. For a given command, with flags >> argument set to 0, this system call is guaranteed to always return the >> same value until reboot. > > I suggest "with flags argument set to MEMBARRIER_CMD_QUERY" here. No, the enum is for the "cmd" argument (see above) not the flags argument. We really mean flags = 0 (the value) here. > >> >> ERRORS >> ENOSYS System call is not implemented. >> >> EINVAL Invalid arguments. >> >> ... >> >> +SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> +{ >> + if (flags) >> + return -EINVAL; > > I'm not a huge fan of this "add a flags arg to syscalls" rule. Is > there any realistic expectation that we'll ever *use* this thing? If > not, why add it? I can see this system call evolve in a few ways in the future, such as having an expedited version (using IPIs), targeting the local thread group, and targeting all threads mapping a specific shared memory mapping. I guess that the cmd argument should be enough to cover that, but in doubt, it might be better to keep a flags argument there for future needs we might be overlooking right now, so we never end up needing a sys_membarrier2 system call. > > You may as well put an unlikely() in there btw. Will do. Thanks! Mathieu > >> + switch (cmd) { >> + case MEMBARRIER_CMD_QUERY: >> + return MEMBARRIER_CMD_BITMASK; >> + case MEMBARRIER_CMD_SHARED: >> + if (num_online_cpus() > 1) >> + synchronize_sched(); >> + return 0; >> + default: >> + return -EINVAL; >> + } > > +} -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH for v4.2 v18 1/3] sys_membarrier(): system-wide memory barrier (generic, x86) @ 2015-05-31 12:53 ` Mathieu Desnoyers 0 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-31 12:53 UTC (permalink / raw) To: Andrew Morton Cc: Paul E. McKenney, linux-kernel, linux-api, KOSAKI Motohiro, rostedt, Nicholas Miell, Linus Torvalds, Ingo Molnar, One Thousand Gnomes, Lai Jiangshan, Stephen Hemminger, Thomas Gleixner, Peter Zijlstra, David Howells, Pranith Kumar, Michael Kerrisk ----- On May 30, 2015, at 12:40 AM, Andrew Morton akpm@linux-foundation.org wrote: > On Sat, 16 May 2015 19:48:18 -0400 Mathieu Desnoyers > <mathieu.desnoyers@efficios.com> wrote: > >> Here is an implementation of a new system call, sys_membarrier(), which >> executes a memory barrier on all threads running on the system. It is >> implemented by calling synchronize_sched(). It can be used to distribute >> the cost of user-space memory barriers asymmetrically by transforming >> pairs of memory barriers into pairs consisting of sys_membarrier() and a >> compiler barrier. For synchronization primitives that distinguish >> between read-side and write-side (e.g. userspace RCU [1], rwlocks), the >> read-side can be accelerated significantly by moving the bulk of the >> memory barrier overhead to the write-side. >> >> ... >> > > It would be nice to hear about the real world value of this syscall to > our users. I'm seeing test results for a microbenchmark but so what. > What actual applications or application classes are calling for this and > what results can they expect to see? AFAIK, the existing open source applications that would be improved by this system call are as follows: * Through Userspace RCU library (http://urcu.so) - DNS server (Knot DNS) https://www.knot-dns.cz/ - Network sniffer (http://netsniff-ng.org/) - Distributed object storage (https://sheepdog.github.io/sheepdog/) - User-space tracing (http://lttng.org) - Network storage system (https://www.gluster.org/) Those projects use RCU in userspace to increase read-side speed and scalability compared to locking. Especially in the case of RCU used by libraries, sys_membarrier can speed up the read-side by moving the bulk of the memory barrier cost to synchronize_rcu(). * Direct users of sys_membarrier - core dotnet garbage collector (https://github.com/dotnet/coreclr/issues/198) Microsoft core dotnet GC developers are planning to use the mprotect() side-effect of issuing memory barriers through IPIs as a way to implement Windows FlushProcessWriteBuffers() on Linux. They are referring to sys_membarrier in their github thread, specifically stating that sys_membarrier() is what they are looking for. > >> >> membarrier(2) man page: >> --------------- snip ------------------- >> MEMBARRIER(2) Linux Programmer's Manual MEMBARRIER(2) >> >> NAME >> membarrier - issue memory barriers on a set of threads >> >> SYNOPSIS >> #include <linux/membarrier.h> >> >> int membarrier(int cmd, int flags); >> >> DESCRIPTION >> The cmd argument is one of the following: >> >> MEMBARRIER_CMD_QUERY >> Query the set of supported commands. It returns a bitmask of >> supported commands. >> >> MEMBARRIER_CMD_SHARED >> Execute a memory barrier on all threads running on the system. >> Upon return from system call, the caller thread is ensured that >> all running threads have passed through a state where all memory >> accesses to user-space addresses match program order between >> entry to and return from the system call (non-running threads >> are de facto in such a state). This covers threads from all pro___ >> cesses running on the system. This command returns 0. >> >> The flags argument needs to be 0. For future extensions. >> >> All memory accesses performed in program order from each targeted >> thread is guaranteed to be ordered with respect to sys_membarrier(). If >> we use the semantic "barrier()" to represent a compiler barrier forcing >> memory accesses to be performed in program order across the barrier, >> and smp_mb() to represent explicit memory barriers forcing full memory >> ordering across the barrier, we have the following ordering table for >> each pair of barrier(), sys_membarrier() and smp_mb(): >> >> The pair ordering is detailed as (O: ordered, X: not ordered): >> >> barrier() smp_mb() sys_membarrier() >> barrier() X X O >> smp_mb() X O O >> sys_membarrier() O O O >> >> RETURN VALUE >> On success, these system calls return zero. On error, -1 is returned, >> and errno is set appropriately. For a given command, with flags >> argument set to 0, this system call is guaranteed to always return the >> same value until reboot. > > I suggest "with flags argument set to MEMBARRIER_CMD_QUERY" here. No, the enum is for the "cmd" argument (see above) not the flags argument. We really mean flags = 0 (the value) here. > >> >> ERRORS >> ENOSYS System call is not implemented. >> >> EINVAL Invalid arguments. >> >> ... >> >> +SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> +{ >> + if (flags) >> + return -EINVAL; > > I'm not a huge fan of this "add a flags arg to syscalls" rule. Is > there any realistic expectation that we'll ever *use* this thing? If > not, why add it? I can see this system call evolve in a few ways in the future, such as having an expedited version (using IPIs), targeting the local thread group, and targeting all threads mapping a specific shared memory mapping. I guess that the cmd argument should be enough to cover that, but in doubt, it might be better to keep a flags argument there for future needs we might be overlooking right now, so we never end up needing a sys_membarrier2 system call. > > You may as well put an unlikely() in there btw. Will do. Thanks! Mathieu > >> + switch (cmd) { >> + case MEMBARRIER_CMD_QUERY: >> + return MEMBARRIER_CMD_BITMASK; >> + case MEMBARRIER_CMD_SHARED: >> + if (num_online_cpus() > 1) >> + synchronize_sched(); >> + return 0; >> + default: >> + return -EINVAL; >> + } > > +} -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH for v4.2 2/3] selftests: add membarrier syscall test 2015-05-16 23:48 ` Mathieu Desnoyers (?) (?) @ 2015-05-16 23:48 ` Mathieu Desnoyers -1 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-16 23:48 UTC (permalink / raw) To: Andrew Morton, Paul E . McKenney Cc: linux-kernel, linux-api, Pranith Kumar, Michael Ellerman, Mathieu Desnoyers From: Pranith Kumar <bobby.prani@gmail.com> This patch adds a self test for the membarrier system call. CC: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Pranith Kumar <bobby.prani@gmail.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> --- tools/testing/selftests/Makefile | 1 + tools/testing/selftests/membarrier/.gitignore | 1 + tools/testing/selftests/membarrier/Makefile | 11 +++ .../testing/selftests/membarrier/membarrier_test.c | 71 ++++++++++++++++++++ 4 files changed, 84 insertions(+), 0 deletions(-) create mode 100644 tools/testing/selftests/membarrier/.gitignore create mode 100644 tools/testing/selftests/membarrier/Makefile create mode 100644 tools/testing/selftests/membarrier/membarrier_test.c diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 95abddc..73824b1 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -5,6 +5,7 @@ TARGETS += exec TARGETS += firmware TARGETS += ftrace TARGETS += kcmp +TARGETS += membarrier TARGETS += memfd TARGETS += memory-hotplug TARGETS += mount diff --git a/tools/testing/selftests/membarrier/.gitignore b/tools/testing/selftests/membarrier/.gitignore new file mode 100644 index 0000000..020c44f4 --- /dev/null +++ b/tools/testing/selftests/membarrier/.gitignore @@ -0,0 +1 @@ +membarrier_test diff --git a/tools/testing/selftests/membarrier/Makefile b/tools/testing/selftests/membarrier/Makefile new file mode 100644 index 0000000..877a503 --- /dev/null +++ b/tools/testing/selftests/membarrier/Makefile @@ -0,0 +1,11 @@ +CFLAGS += -g -I../../../../usr/include/ + +all: + $(CC) $(CFLAGS) membarrier_test.c -o membarrier_test + +TEST_PROGS := membarrier_test + +include ../lib.mk + +clean: + $(RM) membarrier_test diff --git a/tools/testing/selftests/membarrier/membarrier_test.c b/tools/testing/selftests/membarrier/membarrier_test.c new file mode 100644 index 0000000..3c9f217 --- /dev/null +++ b/tools/testing/selftests/membarrier/membarrier_test.c @@ -0,0 +1,71 @@ +#define _GNU_SOURCE +#define __EXPORTED_HEADERS__ + +#include <linux/membarrier.h> +#include <asm-generic/unistd.h> +#include <sys/syscall.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> + +#include "../kselftest.h" + +static int sys_membarrier(int cmd, int flags) +{ + return syscall(__NR_membarrier, cmd, flags); +} + +static void test_membarrier_fail(void) +{ + int cmd = -1, flags = 0; + + if (sys_membarrier(cmd, flags) != -1) { + printf("membarrier: Should fail but passed\n"); + ksft_exit_fail(); + } +} + +static void test_membarrier_success(void) +{ + int flags = 0; + + if (sys_membarrier(MEMBARRIER_CMD_SHARED, flags) != 0) { + printf("membarrier: Executing MEMBARRIER failed, %s\n", + strerror(errno)); + ksft_exit_fail(); + } + + printf("membarrier: MEMBARRIER_CMD_SHARED success\n"); +} + +static void test_membarrier(void) +{ + test_membarrier_fail(); + test_membarrier_success(); +} + +static int test_membarrier_exists(void) +{ + int flags = 0; + + if (sys_membarrier(MEMBARRIER_CMD_QUERY, flags)) + return 0; + + return 1; +} + +int main(int argc, char **argv) +{ + printf("membarrier: MEMBARRIER_CMD_QUERY "); + if (test_membarrier_exists()) { + printf("syscall implemented\n"); + test_membarrier(); + } else { + printf("syscall not implemented!\n"); + return ksft_exit_fail(); + } + + printf("membarrier: tests done!\n"); + + return ksft_exit_pass(); +} -- 1.7.7.3 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH for v4.2 3/3] selftests: enhance membarrier syscall test 2015-05-16 23:48 ` Mathieu Desnoyers ` (2 preceding siblings ...) (?) @ 2015-05-16 23:48 ` Mathieu Desnoyers -1 siblings, 0 replies; 10+ messages in thread From: Mathieu Desnoyers @ 2015-05-16 23:48 UTC (permalink / raw) To: Andrew Morton, Paul E . McKenney Cc: linux-kernel, linux-api, Mathieu Desnoyers, Michael Ellerman, Pranith Kumar Update the membarrier syscall self-test to match the membarrier interface. Extend coverage of the interface. Consider ENOSYS as a "SKIP" test, since it is a valid configuration, but does not allow testing the system call. CC: Michael Ellerman <mpe@ellerman.id.au> CC: Pranith Kumar <bobby.prani@gmail.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> --- .../testing/selftests/membarrier/membarrier_test.c | 100 +++++++++++++++----- 1 files changed, 75 insertions(+), 25 deletions(-) diff --git a/tools/testing/selftests/membarrier/membarrier_test.c b/tools/testing/selftests/membarrier/membarrier_test.c index 3c9f217..dde3125 100644 --- a/tools/testing/selftests/membarrier/membarrier_test.c +++ b/tools/testing/selftests/membarrier/membarrier_test.c @@ -10,62 +10,112 @@ #include "../kselftest.h" +enum test_membarrier_status { + TEST_MEMBARRIER_PASS = 0, + TEST_MEMBARRIER_FAIL, + TEST_MEMBARRIER_SKIP, +}; + static int sys_membarrier(int cmd, int flags) { return syscall(__NR_membarrier, cmd, flags); } -static void test_membarrier_fail(void) +static enum test_membarrier_status test_membarrier_cmd_fail(void) { int cmd = -1, flags = 0; if (sys_membarrier(cmd, flags) != -1) { - printf("membarrier: Should fail but passed\n"); - ksft_exit_fail(); + printf("membarrier: Wrong command should fail but passed.\n"); + return TEST_MEMBARRIER_FAIL; + } + return TEST_MEMBARRIER_PASS; +} + +static enum test_membarrier_status test_membarrier_flags_fail(void) +{ + int cmd = MEMBARRIER_CMD_QUERY, flags = 1; + + if (sys_membarrier(cmd, flags) != -1) { + printf("membarrier: Wrong flags should fail but passed.\n"); + return TEST_MEMBARRIER_FAIL; } + return TEST_MEMBARRIER_PASS; } -static void test_membarrier_success(void) +static enum test_membarrier_status test_membarrier_success(void) { - int flags = 0; + int cmd = MEMBARRIER_CMD_SHARED, flags = 0; - if (sys_membarrier(MEMBARRIER_CMD_SHARED, flags) != 0) { - printf("membarrier: Executing MEMBARRIER failed, %s\n", + if (sys_membarrier(cmd, flags) != 0) { + printf("membarrier: Executing MEMBARRIER_CMD_SHARED failed. %s.\n", strerror(errno)); - ksft_exit_fail(); + return TEST_MEMBARRIER_FAIL; } - printf("membarrier: MEMBARRIER_CMD_SHARED success\n"); + printf("membarrier: MEMBARRIER_CMD_SHARED success.\n"); + return TEST_MEMBARRIER_PASS; } -static void test_membarrier(void) +static enum test_membarrier_status test_membarrier(void) { - test_membarrier_fail(); - test_membarrier_success(); + enum test_membarrier_status status; + + status = test_membarrier_cmd_fail(); + if (status) + return status; + status = test_membarrier_flags_fail(); + if (status) + return status; + status = test_membarrier_success(); + if (status) + return status; + return TEST_MEMBARRIER_PASS; } -static int test_membarrier_exists(void) +static enum test_membarrier_status test_membarrier_query(void) { - int flags = 0; - - if (sys_membarrier(MEMBARRIER_CMD_QUERY, flags)) - return 0; + int flags = 0, ret; - return 1; + printf("membarrier MEMBARRIER_CMD_QUERY "); + ret = sys_membarrier(MEMBARRIER_CMD_QUERY, flags); + if (ret < 0) { + printf("failed. %s.\n", strerror(errno)); + switch (errno) { + case ENOSYS: + /* + * It is valid to build a kernel with + * CONFIG_MEMBARRIER=n. However, this skips the tests. + */ + return TEST_MEMBARRIER_SKIP; + case EINVAL: + default: + return TEST_MEMBARRIER_FAIL; + } + } + if (!(ret & MEMBARRIER_CMD_SHARED)) { + printf("command MEMBARRIER_CMD_SHARED is not supported.\n"); + return TEST_MEMBARRIER_FAIL; + } + printf("syscall available.\n"); + return TEST_MEMBARRIER_PASS; } int main(int argc, char **argv) { - printf("membarrier: MEMBARRIER_CMD_QUERY "); - if (test_membarrier_exists()) { - printf("syscall implemented\n"); - test_membarrier(); - } else { - printf("syscall not implemented!\n"); + switch (test_membarrier_query()) { + case TEST_MEMBARRIER_FAIL: return ksft_exit_fail(); + case TEST_MEMBARRIER_SKIP: + return ksft_exit_skip(); + } + switch (test_membarrier()) { + case TEST_MEMBARRIER_FAIL: + return ksft_exit_fail(); + case TEST_MEMBARRIER_SKIP: + return ksft_exit_skip(); } printf("membarrier: tests done!\n"); - return ksft_exit_pass(); } -- 1.7.7.3 ^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2015-05-31 12:53 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-05-16 23:48 [PATCH for v4.2 0/3] membarrier system call Mathieu Desnoyers
2015-05-16 23:48 ` Mathieu Desnoyers
[not found] ` <1431820100-17040-1-git-send-email-mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org>
2015-05-16 23:48 ` [PATCH for v4.2 v18 1/3] sys_membarrier(): system-wide memory barrier (generic, x86) Mathieu Desnoyers
2015-05-16 23:48 ` Mathieu Desnoyers
[not found] ` <1431820100-17040-2-git-send-email-mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org>
2015-05-29 22:40 ` Andrew Morton
2015-05-29 22:40 ` Andrew Morton
[not found] ` <20150529154036.9695b6c153ed37aed3343f3b-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
2015-05-31 12:53 ` Mathieu Desnoyers
2015-05-31 12:53 ` Mathieu Desnoyers
2015-05-16 23:48 ` [PATCH for v4.2 2/3] selftests: add membarrier syscall test Mathieu Desnoyers
2015-05-16 23:48 ` [PATCH for v4.2 3/3] selftests: enhance " Mathieu Desnoyers
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.