From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752894AbbDBMZw (ORCPT ); Thu, 2 Apr 2015 08:25:52 -0400 Received: from terminus.zytor.com ([198.137.202.10]:60430 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752522AbbDBMZs (ORCPT ); Thu, 2 Apr 2015 08:25:48 -0400 Date: Thu, 2 Apr 2015 05:25:05 -0700 From: tip-bot for Denys Vlasenko Message-ID: Cc: hpa@zytor.com, mingo@kernel.org, keescook@chromium.org, linux-kernel@vger.kernel.org, wad@chromium.org, ast@plumgrid.com, tglx@linutronix.de, dvlasenk@redhat.com, luto@amacapital.net, torvalds@linux-foundation.org, oleg@redhat.com, bp@alien8.de, rostedt@goodmis.org, fweisbec@gmail.com Reply-To: mingo@kernel.org, hpa@zytor.com, ast@plumgrid.com, wad@chromium.org, linux-kernel@vger.kernel.org, keescook@chromium.org, dvlasenk@redhat.com, luto@amacapital.net, tglx@linutronix.de, fweisbec@gmail.com, rostedt@goodmis.org, bp@alien8.de, torvalds@linux-foundation.org, oleg@redhat.com In-Reply-To: <1427821211-25099-1-git-send-email-dvlasenk@redhat.com> References: <1427821211-25099-1-git-send-email-dvlasenk@redhat.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:x86/asm] x86/asm/entry/64: Do not TRACE_IRQS fast SYSRET64 path Git-Commit-ID: 4416c5a6dacdddd55378e7011f9c8720d2a7470f X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: 4416c5a6dacdddd55378e7011f9c8720d2a7470f Gitweb: http://git.kernel.org/tip/4416c5a6dacdddd55378e7011f9c8720d2a7470f Author: Denys Vlasenko AuthorDate: Tue, 31 Mar 2015 19:00:03 +0200 Committer: Ingo Molnar CommitDate: Wed, 1 Apr 2015 13:17:38 +0200 x86/asm/entry/64: Do not TRACE_IRQS fast SYSRET64 path SYSRET code path has a small irq-off block. On this code path, TRACE_IRQS_ON can't be called right before interrupts are enabled for real, we can't clobber registers there. So current code does it earlier, in a safe place. But with this, TRACE_IRQS_OFF/ON frames just two fast instructions, which is ridiculous: now most of irq-off block is _outside_ of the framing. Do the same thing that we do on SYSCALL entry: do not track this irq-off block, it is very small to ever cause noticeable irq latency. Be careful: make sure that "jnz int_ret_from_sys_call_irqs_off" now does invoke TRACE_IRQS_OFF - move int_ret_from_sys_call_irqs_off label before TRACE_IRQS_OFF. Signed-off-by: Denys Vlasenko Cc: Alexei Starovoitov Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Frederic Weisbecker Cc: H. Peter Anvin Cc: Kees Cook Cc: Linus Torvalds Cc: Oleg Nesterov Cc: Steven Rostedt Cc: Will Drewry Link: http://lkml.kernel.org/r/1427821211-25099-1-git-send-email-dvlasenk@redhat.com Signed-off-by: Ingo Molnar --- arch/x86/kernel/entry_64.S | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index 6f251a5..f6e37de 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S @@ -269,8 +269,11 @@ system_call_fastpath: * Has incompletely filled pt_regs. */ LOCKDEP_SYS_EXIT + /* + * We do not frame this tiny irq-off block with TRACE_IRQS_OFF/ON, + * it is too small to ever cause noticeable irq latency. + */ DISABLE_INTERRUPTS(CLBR_NONE) - TRACE_IRQS_OFF /* * We must check ti flags with interrupts (or at least preemption) @@ -284,10 +287,7 @@ system_call_fastpath: jnz int_ret_from_sys_call_irqs_off /* Go to the slow path */ CFI_REMEMBER_STATE - /* - * sysretq will re-enable interrupts: - */ - TRACE_IRQS_ON + RESTORE_C_REGS_EXCEPT_RCX_R11 movq RIP(%rsp),%rcx CFI_REGISTER rip,rcx @@ -298,6 +298,7 @@ system_call_fastpath: * 64bit SYSRET restores rip from rcx, * rflags from r11 (but RF and VM bits are forced to 0), * cs and ss are loaded from MSRs. + * Restoration of rflags re-enables interrupts. */ USERGS_SYSRET64 @@ -346,8 +347,8 @@ tracesys_phase2: */ GLOBAL(int_ret_from_sys_call) DISABLE_INTERRUPTS(CLBR_NONE) +int_ret_from_sys_call_irqs_off: /* jumps come here from the irqs-off SYSRET path */ TRACE_IRQS_OFF -int_ret_from_sys_call_irqs_off: movl $_TIF_ALLWORK_MASK,%edi /* edi: mask to check */ GLOBAL(int_with_check)