From: Peter Zijlstra <peterz@infradead.org>
To: Linus Torvalds <torvalds@linux-foundation.org>,
Ingo Molnar <mingo@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>, Peter Anvin <hpa@zytor.com>,
Mike Galbraith <bitbucket@online.de>,
Thomas Gleixner <tglx@linutronix.de>,
Arjan van de Ven <arjan@linux.intel.com>,
Frederic Weisbecker <fweisbec@gmail.com>,
linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org,
Peter Zijlstra <peterz@infradead.org>
Subject: [PATCH 00/11] preempt_count rework -v3
Date: Tue, 17 Sep 2013 11:10:46 +0200 [thread overview]
Message-ID: <20130917082838.218329307@infradead.org> (raw)
These patches optimize preempt_enable by firstly folding the preempt and
need_resched tests into one -- this should work for all architectures. And
secondly by providing per-arch preempt_count implementations; with x86 using
per-cpu preempt_count for fastest access.
These patches have been boot tested on CONFIG_PREEMPT=y x86_64 and survive
building a x86_64-defconfig kernel.
text data bss filename
11387014 1454776 1187840 defconfig-build/vmlinux.before
11352294 1454776 1187840 defconfig-build/vmlinux.after
void kick_process(struct task_struct *p)
{
int cpu;
preempt_disable();
cpu = task_cpu(p);
if ((cpu != smp_processor_id()) && task_curr(p))
smp_send_reschedule(cpu);
preempt_enable();
}
before:
ffffffff81073f00 <kick_process>:
ffffffff81073f00: 55 push %rbp
I ffffffff81073f01: 65 48 8b 04 25 f0 b7 mov %gs:0xb7f0,%rax
ffffffff81073f08: 00 00
I ffffffff81073f0a: 83 80 44 e0 ff ff 01 addl $0x1,-0x1fbc(%rax)
ffffffff81073f11: 48 89 e5 mov %rsp,%rbp
ffffffff81073f14: 48 8b 47 08 mov 0x8(%rdi),%rax
ffffffff81073f18: 8b 50 18 mov 0x18(%rax),%edx
ffffffff81073f1b: 65 8b 04 25 1c b0 00 mov %gs:0xb01c,%eax
ffffffff81073f22: 00
ffffffff81073f23: 39 c2 cmp %eax,%edx
ffffffff81073f25: 74 1b je ffffffff81073f42 <kick_process+0x42>
ffffffff81073f27: 89 d1 mov %edx,%ecx
ffffffff81073f29: 48 c7 c0 40 2c 01 00 mov $0x12c40,%rax
ffffffff81073f30: 48 8b 0c cd 00 25 cc mov -0x7e33db00(,%rcx,8),%rcx
ffffffff81073f37: 81
ffffffff81073f38: 48 3b bc 08 00 08 00 cmp 0x800(%rax,%rcx,1),%rdi
ffffffff81073f3f: 00
ffffffff81073f40: 74 2e je ffffffff81073f70 <kick_process+0x70>
D ffffffff81073f42: 65 48 8b 04 25 f0 b7 mov %gs:0xb7f0,%rax
ffffffff81073f49: 00 00
D ffffffff81073f4b: 83 a8 44 e0 ff ff 01 subl $0x1,-0x1fbc(%rax)
D ffffffff81073f52: 48 8b 80 38 e0 ff ff mov -0x1fc8(%rax),%rax
D ffffffff81073f59: a8 08 test $0x8,%al
D ffffffff81073f5b: 75 02 jne ffffffff81073f5f <kick_process+0x5f>
ffffffff81073f5d: 5d pop %rbp
ffffffff81073f5e: c3 retq
D ffffffff81073f5f: e8 9c 6f 75 00 callq ffffffff817caf00 <preempt_schedule>
ffffffff81073f64: 5d pop %rbp
ffffffff81073f65: c3 retq
ffffffff81073f66: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
ffffffff81073f6d: 00 00 00
ffffffff81073f70: 89 d7 mov %edx,%edi
ffffffff81073f72: ff 15 a8 99 ba 00 callq *0xba99a8(%rip) # ffffffff81c1d920 <smp_ops+0x20>
ffffffff81073f78: eb c8 jmp ffffffff81073f42 <kick_process+0x42>
ffffffff81073f7a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1)
after:
ffffffff8106f5e0 <kick_process>:
ffffffff8106f5e0: 55 push %rbp
I ffffffff8106f5e1: 65 ff 04 25 e0 b7 00 incl %gs:0xb7e0
ffffffff8106f5e8: 00
ffffffff8106f5e9: 48 89 e5 mov %rsp,%rbp
ffffffff8106f5ec: 48 8b 47 08 mov 0x8(%rdi),%rax
ffffffff8106f5f0: 8b 50 18 mov 0x18(%rax),%edx
ffffffff8106f5f3: 65 8b 04 25 1c b0 00 mov %gs:0xb01c,%eax
ffffffff8106f5fa: 00
ffffffff8106f5fb: 39 c2 cmp %eax,%edx
ffffffff8106f5fd: 74 1b je ffffffff8106f61a <kick_process+0x3a>
ffffffff8106f5ff: 89 d1 mov %edx,%ecx
ffffffff8106f601: 48 c7 c0 40 2c 01 00 mov $0x12c40,%rax
ffffffff8106f608: 48 8b 0c cd 00 25 cc mov -0x7e33db00(,%rcx,8),%rcx
ffffffff8106f60f: 81
ffffffff8106f610: 48 3b bc 08 00 08 00 cmp 0x800(%rax,%rcx,1),%rdi
ffffffff8106f617: 00
ffffffff8106f618: 74 26 je ffffffff8106f640 <kick_process+0x60>
D ffffffff8106f61a: 65 ff 0c 25 e0 b7 00 decl %gs:0xb7e0
ffffffff8106f621: 00
D ffffffff8106f622: 74 0c je ffffffff8106f630 <kick_process+0x50>
ffffffff8106f624: 5d pop %rbp
ffffffff8106f625: c3 retq
ffffffff8106f626: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
ffffffff8106f62d: 00 00 00
D ffffffff8106f630: e8 fb b4 f9 ff callq ffffffff8100ab30 <___preempt_schedule>
ffffffff8106f635: 5d pop %rbp
ffffffff8106f636: c3 retq
ffffffff8106f637: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
ffffffff8106f63e: 00 00
ffffffff8106f640: 89 d7 mov %edx,%edi
ffffffff8106f642: ff 15 d8 e2 ba 00 callq *0xbae2d8(%rip) # ffffffff81c1d920 <smp_ops+0x20>
ffffffff8106f648: eb d0 jmp ffffffff8106f61a <kick_process+0x3a>
ffffffff8106f64a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1)
next reply other threads:[~2013-09-17 9:10 UTC|newest]
Thread overview: 85+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-17 9:10 Peter Zijlstra [this message]
2013-09-17 9:10 ` [PATCH 00/11] preempt_count rework -v3 Peter Zijlstra
2013-09-17 9:10 ` [PATCH 01/11] x86: Use asm goto to implement better modify_and_test() functions Peter Zijlstra
2013-09-18 18:44 ` Linus Torvalds
[not found] ` <4ec87843-c29a-401a-a54f-2cd4d61fba62@email.android.com>
2013-09-19 8:31 ` Andi Kleen
2013-09-19 8:31 ` Andi Kleen
2013-09-19 9:39 ` Ingo Molnar
2013-09-20 4:43 ` H. Peter Anvin
2013-09-17 9:10 ` [PATCH 02/11] sched, rcu: Make RCU use resched_cpu() Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 14:40 ` Peter Zijlstra
2013-09-23 16:55 ` Paul E. McKenney
2013-09-23 21:18 ` Paul E. McKenney
2013-09-24 8:07 ` Peter Zijlstra
2013-09-24 13:37 ` Paul E. McKenney
2013-09-17 9:10 ` [PATCH 03/11] sched: Remove {set,clear}_need_resched Peter Zijlstra
2013-09-17 9:10 ` [PATCH 04/11] sched, idle: Fix the idle polling state logic Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 9:10 ` [PATCH 05/11] sched: Introduce preempt_count accessor functions Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 9:10 ` [PATCH 06/11] sched: Add NEED_RESCHED to the preempt_count Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 9:10 ` [PATCH 07/11] sched, arch: Create asm/preempt.h Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 9:10 ` [PATCH 08/11] sched: Create more preempt_count accessors Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 9:10 ` [PATCH 09/11] sched: Extract the basic add/sub preempt_count modifiers Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 9:10 ` [PATCH 10/11] sched, x86: Provide a per-cpu preempt_count implementation Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 9:10 ` [PATCH 11/11] sched, x86: Optimize the preempt_schedule() call Peter Zijlstra
2013-09-17 9:10 ` Peter Zijlstra
2013-09-17 20:23 ` Peter Zijlstra
2013-09-17 10:53 ` [PATCH 00/11] preempt_count rework -v3 Ingo Molnar
2013-09-17 11:22 ` Peter Zijlstra
2013-09-17 18:53 ` [patch 0/6] Make all preempt_count related constants generic Thomas Gleixner
2013-09-17 18:53 ` Thomas Gleixner
2013-09-17 18:53 ` [patch 1/6] hardirq: Make hardirq bits generic Thomas Gleixner
2013-09-17 20:00 ` Geert Uytterhoeven
2013-09-17 21:24 ` Thomas Gleixner
2013-09-17 21:24 ` Thomas Gleixner
2013-09-18 14:06 ` Thomas Gleixner
2013-09-18 14:06 ` Thomas Gleixner
2013-09-19 15:14 ` Thomas Gleixner
2013-09-19 15:14 ` Thomas Gleixner
2013-09-19 17:02 ` Andreas Schwab
2013-09-19 18:19 ` Geert Uytterhoeven
2013-09-20 9:26 ` Thomas Gleixner
2013-09-20 9:26 ` Thomas Gleixner
2013-11-04 12:06 ` Thomas Gleixner
2013-11-04 19:44 ` Geert Uytterhoeven
2013-11-04 19:44 ` Geert Uytterhoeven
2013-11-06 17:23 ` Thomas Gleixner
2013-11-06 17:23 ` Thomas Gleixner
2013-11-07 14:12 ` Geert Uytterhoeven
2013-11-07 14:12 ` Geert Uytterhoeven
2013-11-07 16:39 ` Thomas Gleixner
2013-09-17 18:53 ` [patch 2/6] h8300: Use schedule_preempt_irq Thomas Gleixner
2013-09-17 18:53 ` Thomas Gleixner
2013-09-20 17:41 ` Guenter Roeck
2013-09-20 17:41 ` Guenter Roeck
2013-09-20 21:46 ` Thomas Gleixner
2013-09-17 18:53 ` [patch 3/6] m32r: Use preempt_schedule_irq Thomas Gleixner
2013-09-17 18:53 ` Thomas Gleixner
2013-09-17 18:53 ` [patch 4/6] ia64: " Thomas Gleixner
2013-11-20 19:59 ` Tony Luck
2013-11-20 20:57 ` Thomas Gleixner
2013-11-21 11:41 ` Thomas Gleixner
2013-11-21 12:39 ` Frederic Weisbecker
2013-11-21 13:06 ` Peter Zijlstra
2013-11-21 13:06 ` Peter Zijlstra
2013-11-21 13:30 ` Thomas Gleixner
2013-11-21 13:30 ` Thomas Gleixner
2013-11-21 18:57 ` Tony Luck
2013-11-21 18:57 ` Tony Luck
2013-11-26 18:37 ` Tony Luck
2013-11-26 18:58 ` Peter Zijlstra
2013-11-26 18:58 ` Peter Zijlstra
2013-11-27 13:36 ` Ingo Molnar
2013-09-17 18:53 ` [patch 5/6] sparc: " Thomas Gleixner
2013-09-17 22:54 ` David Miller
2013-09-17 23:23 ` Thomas Gleixner
2013-09-18 0:12 ` David Miller
2013-09-17 18:53 ` [patch 6/6] preempt: Make PREEMPT_ACTIVE generic Thomas Gleixner
2013-09-18 10:48 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130917082838.218329307@infradead.org \
--to=peterz@infradead.org \
--cc=ak@linux.intel.com \
--cc=arjan@linux.intel.com \
--cc=bitbucket@online.de \
--cc=fweisbec@gmail.com \
--cc=hpa@zytor.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).