From: Christoph Lameter <cl@linux.com>
To: akpm@linux-foundation.org
Cc: Pekka Enberg <penberg@cs.helsinki.fi>,
linux-kernel@vger.kernel.org,
Eric Dumazet <eric.dumazet@gmail.com>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Tejun Heo <tj@kernel.org>,
linux-mm@kvack.org
Subject: [thisops uV3 03/18] percpu: Generic support for this_cpu_add,sub,dec,inc_return
Date: Tue, 30 Nov 2010 13:07:10 -0600 [thread overview]
Message-ID: <20101130190842.869940433@linux.com> (raw)
In-Reply-To: 20101130190707.457099608@linux.com
[-- Attachment #1: this_cpu_add_dec_return --]
[-- Type: text/plain, Size: 4093 bytes --]
Introduce generic support for this_cpu_add_return etc.
The fallback is to realize these operations with simpler __this_cpu_ops.
Reviewed-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Christoph Lameter <cl@linux.com>
---
include/linux/percpu.h | 77 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
Index: linux-2.6/include/linux/percpu.h
===================================================================
--- linux-2.6.orig/include/linux/percpu.h 2010-11-29 10:19:28.000000000 -0600
+++ linux-2.6/include/linux/percpu.h 2010-11-29 10:24:57.000000000 -0600
@@ -240,6 +240,25 @@ extern void __bad_size_call_parameter(vo
pscr_ret__; \
})
+#define __pcpu_size_call_return2(stem, pcp, ...) \
+({ \
+ typeof(pcp) ret__; \
+ __verify_pcpu_ptr(&(pcp)); \
+ switch(sizeof(pcp)) { \
+ case 1: ret__ = stem##1(pcp, __VA_ARGS__); \
+ break; \
+ case 2: ret__ = stem##2(pcp, __VA_ARGS__); \
+ break; \
+ case 4: ret__ = stem##4(pcp, __VA_ARGS__); \
+ break; \
+ case 8: ret__ = stem##8(pcp, __VA_ARGS__); \
+ break; \
+ default: \
+ __bad_size_call_parameter();break; \
+ } \
+ ret__; \
+})
+
#define __pcpu_size_call(stem, variable, ...) \
do { \
__verify_pcpu_ptr(&(variable)); \
@@ -529,6 +548,64 @@ do { \
# define __this_cpu_xor(pcp, val) __pcpu_size_call(__this_cpu_xor_, (pcp), (val))
#endif
+#define _this_cpu_generic_add_return(pcp, val) \
+({ \
+ typeof(pcp) ret__; \
+ preempt_disable(); \
+ __this_cpu_add(pcp, val); \
+ ret__ = __this_cpu_read(pcp); \
+ preempt_enable(); \
+ ret__; \
+})
+
+#ifndef this_cpu_add_return
+# ifndef this_cpu_add_return_1
+# define this_cpu_add_return_1(pcp, val) _this_cpu_generic_add_return(pcp, val)
+# endif
+# ifndef this_cpu_add_return_2
+# define this_cpu_add_return_2(pcp, val) _this_cpu_generic_add_return(pcp, val)
+# endif
+# ifndef this_cpu_add_return_4
+# define this_cpu_add_return_4(pcp, val) _this_cpu_generic_add_return(pcp, val)
+# endif
+# ifndef this_cpu_add_return_8
+# define this_cpu_add_return_8(pcp, val) _this_cpu_generic_add_return(pcp, val)
+# endif
+# define this_cpu_add_return(pcp, val) __pcpu_size_call_return2(this_cpu_add_return_, pcp, val)
+#endif
+
+#define this_cpu_sub_return(pcp, val) this_cpu_add_return(pcp, -(val))
+#define this_cpu_inc_return(pcp) this_cpu_add_return(pcp, 1)
+#define this_cpu_dec_return(pcp) this_cpu_add_return(pcp, -1)
+
+#define __this_cpu_generic_add_return(pcp, val) \
+({ \
+ typeof(pcp) ret__; \
+ __this_cpu_add(pcp, val); \
+ ret__ = __this_cpu_read(pcp); \
+ ret__; \
+})
+
+#ifndef __this_cpu_add_return
+# ifndef __this_cpu_add_return_1
+# define __this_cpu_add_return_1(pcp, val) __this_cpu_generic_add_return(pcp, val)
+# endif
+# ifndef __this_cpu_add_return_2
+# define __this_cpu_add_return_2(pcp, val) __this_cpu_generic_add_return(pcp, val)
+# endif
+# ifndef __this_cpu_add_return_4
+# define __this_cpu_add_return_4(pcp, val) __this_cpu_generic_add_return(pcp, val)
+# endif
+# ifndef __this_cpu_add_return_8
+# define __this_cpu_add_return_8(pcp, val) __this_cpu_generic_add_return(pcp, val)
+# endif
+# define __this_cpu_add_return(pcp, val) __pcpu_size_call_return2(this_cpu_add_return_, pcp, val)
+#endif
+
+#define __this_cpu_sub_return(pcp, val) this_cpu_add_return(pcp, -(val))
+#define __this_cpu_inc_return(pcp) this_cpu_add_return(pcp, 1)
+#define __this_cpu_dec_return(pcp) this_cpu_add_return(pcp, -1)
+
/*
* IRQ safe versions of the per cpu RMW operations. Note that these operations
* are *not* safe against modification of the same variable from another
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2010-11-30 19:09 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-30 19:07 [thisops uV3 00/18] Upgrade of this_cpu_ops V3 Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 01/18] percpucounter: Optimize __percpu_counter_add a bit through the use of this_cpu() options Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 02/18] vmstat: Optimize zone counter modifications through the use of this cpu operations Christoph Lameter
2010-11-30 19:07 ` Christoph Lameter [this message]
2010-11-30 19:07 ` [thisops uV3 04/18] x86: Support for this_cpu_add,sub,dec,inc_return Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 05/18] x86: Use this_cpu_inc_return for nmi counter Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 06/18] vmstat: Use this_cpu_inc_return for vm statistics Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 07/18] highmem: Use this_cpu_xx_return() operations Christoph Lameter
2010-11-30 19:13 ` Eric Dumazet
2010-11-30 19:19 ` Peter Zijlstra
2010-11-30 19:26 ` Christoph Lameter
2010-11-30 19:29 ` Eric Dumazet
2010-11-30 19:38 ` Peter Zijlstra
2010-11-30 19:53 ` Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 08/18] Taskstats: Use this_cpu_ops Christoph Lameter
2010-12-01 18:06 ` Michael Holzheu
2010-12-01 18:13 ` Christoph Lameter
2010-12-06 14:32 ` Balbir Singh
2010-12-07 14:39 ` Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 09/18] fs: Use this_cpu_xx operations in buffer.c Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 10/18] x86: Use this_cpu_ops to optimize code Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 11/18] x86: Use this_cpu_ops for current_cpu_data accesses Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 12/18] Core: Replace __get_cpu_var with __this_cpu_read if not used for an address Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 13/18] drivers: " Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 14/18] lguest: Use this_cpu_ops Christoph Lameter
2010-12-06 7:46 ` Rusty Russell
2010-12-06 15:54 ` Christoph Lameter
2010-12-06 16:27 ` Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 15/18] Xen: " Christoph Lameter
2010-11-30 20:53 ` Jeremy Fitzhardinge
2010-11-30 21:03 ` Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 16/18] kprobes: " Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 17/18] Connector: Use this_cpu operations Christoph Lameter
2010-11-30 19:07 ` [thisops uV3 18/18] Fakekey: Simplify speakup_fake_key_pressed through this_cpu_ops Christoph Lameter
2010-11-30 20:05 ` [extra] timers: Use this_cpu_read Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101130190842.869940433@linux.com \
--to=cl@linux.com \
--cc=akpm@linux-foundation.org \
--cc=eric.dumazet@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=penberg@cs.helsinki.fi \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).