From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.linuxfoundation.org ([140.211.169.12]:56252 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753420AbdGJOpC (ORCPT ); Mon, 10 Jul 2017 10:45:02 -0400 Subject: Patch "x86/uaccess: Optimize copy_user_enhanced_fast_string() for short strings" has been added to the 4.11-stable tree To: pabeni@redhat.com, bp@alien8.de, brgerst@gmail.com, dvlasenk@redhat.com, gnomes@lxorguk.ukuu.org.uk, gregkh@linuxfoundation.org, hannes@stressinduktion.org, hpa@zytor.com, jpoimboe@redhat.com, keescook@chromium.org, luto@kernel.org, mgorman@techsingularity.net, mingo@kernel.org, peterz@infradead.org, tglx@linutronix.de, torvalds@linux-foundation.org Cc: , From: Date: Mon, 10 Jul 2017 16:44:58 +0200 Message-ID: <1499697898144208@kroah.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ANSI_X3.4-1968 Content-Transfer-Encoding: 8bit Sender: stable-owner@vger.kernel.org List-ID: This is a note to let you know that I've just added the patch titled x86/uaccess: Optimize copy_user_enhanced_fast_string() for short strings to the 4.11-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary The filename of the patch is: x86-uaccess-optimize-copy_user_enhanced_fast_string-for-short-strings.patch and it can be found in the queue-4.11 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let know about it. >>From 236222d39347e0e486010f10c1493e83dbbdfba8 Mon Sep 17 00:00:00 2001 From: Paolo Abeni Date: Thu, 29 Jun 2017 15:55:58 +0200 Subject: x86/uaccess: Optimize copy_user_enhanced_fast_string() for short strings From: Paolo Abeni commit 236222d39347e0e486010f10c1493e83dbbdfba8 upstream. According to the Intel datasheet, the REP MOVSB instruction exposes a pretty heavy setup cost (50 ticks), which hurts short string copy operations. This change tries to avoid this cost by calling the explicit loop available in the unrolled code for strings shorter than 64 bytes. The 64 bytes cutoff value is arbitrary from the code logic point of view - it has been selected based on measurements, as the largest value that still ensures a measurable gain. Micro benchmarks of the __copy_from_user() function with lengths in the [0-63] range show this performance gain (shorter the string, larger the gain): - in the [55%-4%] range on Intel Xeon(R) CPU E5-2690 v4 - in the [72%-9%] range on Intel Core i7-4810MQ Other tested CPUs - namely Intel Atom S1260 and AMD Opteron 8216 - show no difference, because they do not expose the ERMS feature bit. Signed-off-by: Paolo Abeni Acked-by: Linus Torvalds Cc: Alan Cox Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Hannes Frederic Sowa Cc: Josh Poimboeuf Cc: Kees Cook Cc: Peter Zijlstra Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/4533a1d101fd460f80e21329a34928fad521c1d4.1498744345.git.pabeni@redhat.com [ Clarified the changelog. ] Signed-off-by: Ingo Molnar Signed-off-by: Greg Kroah-Hartman Signed-off-by: Mel Gorman diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S index c5959576c315..020f75cc8cf6 100644 --- a/arch/x86/lib/copy_user_64.S +++ b/arch/x86/lib/copy_user_64.S @@ -37,7 +37,7 @@ ENTRY(copy_user_generic_unrolled) movl %edx,%ecx andl $63,%edx shrl $6,%ecx - jz 17f + jz .L_copy_short_string 1: movq (%rsi),%r8 2: movq 1*8(%rsi),%r9 3: movq 2*8(%rsi),%r10 @@ -58,7 +58,8 @@ ENTRY(copy_user_generic_unrolled) leaq 64(%rdi),%rdi decl %ecx jnz 1b -17: movl %edx,%ecx +.L_copy_short_string: + movl %edx,%ecx andl $7,%edx shrl $3,%ecx jz 20f @@ -174,6 +175,8 @@ EXPORT_SYMBOL(copy_user_generic_string) */ ENTRY(copy_user_enhanced_fast_string) ASM_STAC + cmpl $64,%edx + jb .L_copy_short_string /* less then 64 bytes, avoid the costly 'rep' */ movl %edx,%ecx 1: rep movsb Patches currently in stable-queue which might be from pabeni@redhat.com are queue-4.11/x86-uaccess-optimize-copy_user_enhanced_fast_string-for-short-strings.patch