From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Google-Smtp-Source: AIpwx48ixXdZ1CaEMRdlnjFoWCsy6VkFdozKUD6tgMI0N4iz5BRmlFOdhNSIwG885J/Uss8IIm0T ARC-Seal: i=1; a=rsa-sha256; t=1523981305; cv=none; d=google.com; s=arc-20160816; b=DnUsxSUZ+ShAXoo1rwhw+siwj6y9/7Y8C8eB8FMxmgTgVCj5KJRi1eJVBelFhRM6jS Lt6HR4kvSkJm0x61fBl/t4Mh9LiL5Z6WhWoQvXbmGxZIX2K9c97xXc+Dh8dx56ajfCef VSzW4nWNOQU8cSaa1LaPQjFvwFDa578KVj1IC/ydm5u9ChLJRiP79EFDx6SJ0DpTwpsS mtATr1jcBdniEgvD5PIQLqM3PJCtTy4fKEUO3n92nu1QL7AiYq3d9L4w4E1bRyToUJkn qFybhgk1l5dI+QGmu+FL4R6PvtK8jLjJIXNjcnaW+EpqCkJcxFR1TURwPzReEC51nPLS e3IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=+lc3JmUshJtuKA0C7UXkkY4uFQ8Xnn7R5h4wsYiVbV8=; b=rdPRvfc04IH2fkYO/So4fU2evyn01l96JylBQzO3CyNjkQMfk3rS9MYZT3/f0LrGck DSL+1hTeINA592qaNwPsJuZdUnUlhXeF8GFP/fME5kOUJTpFm0u+0Rljez89wxUrDYvr GPlPkdkwf7LdA96oVaj06p1k1twIa4adaNLRHZtSvgbwbNowYqmfRMliOcOcwscivkkx nQ7uEYlldSsmDwyBdM77u/04y0ekw1QsdCs8NAE6DReFPy3YnKuHEXKpQI8UwGEmgQDY 8KmmW2dATemUsrQ0AA08MpzvpJTz6NSFQSHHbaIJ2zPZLi1ehcp5sOCvfVfWBR0IeZgp V51w== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning gregkh@linuxfoundation.org does not designate 46.44.180.42 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning gregkh@linuxfoundation.org does not designate 46.44.180.42 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Greg Kroah-Hartman , Robin Murphy , Will Deacon , Catalin Marinas , Greg Hackmann , Mark Rutland Subject: [PATCH 4.9 17/66] arm64: uaccess: Dont bother eliding access_ok checks in __{get, put}_user Date: Tue, 17 Apr 2018 17:58:50 +0200 Message-Id: <20180417155646.597245647@linuxfoundation.org> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180417155645.868055442@linuxfoundation.org> References: <20180417155645.868055442@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-LABELS: =?utf-8?b?IlxcU2VudCI=?= X-GMAIL-THRID: =?utf-8?q?1598010221612023293?= X-GMAIL-MSGID: =?utf-8?q?1598010221612023293?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: 4.9-stable review patch. If anyone has any objections, please let me know. ------------------ From: Mark Rutland From: Will Deacon commit 84624087dd7e3b482b7b11c170ebc1f329b3a218 upstream. access_ok isn't an expensive operation once the addr_limit for the current thread has been loaded into the cache. Given that the initial access_ok check preceding a sequence of __{get,put}_user operations will take the brunt of the miss, we can make the __* variants identical to the full-fat versions, which brings with it the benefits of address masking. The likely cost in these sequences will be from toggling PAN/UAO, which we can address later by implementing the *_unsafe versions. Reviewed-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Mark Rutland [v4.9 backport] Tested-by: Greg Hackmann Signed-off-by: Greg Kroah-Hartman --- arch/arm64/include/asm/uaccess.h | 62 ++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 26 deletions(-) --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -209,30 +209,35 @@ do { \ CONFIG_ARM64_PAN)); \ } while (0) -#define __get_user(x, ptr) \ +#define __get_user_check(x, ptr, err) \ ({ \ - int __gu_err = 0; \ - __get_user_err((x), (ptr), __gu_err); \ - __gu_err; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_READ, __p, sizeof(*__p))) { \ + __p = uaccess_mask_ptr(__p); \ + __get_user_err((x), __p, (err)); \ + } else { \ + (x) = 0; (err) = -EFAULT; \ + } \ }) #define __get_user_error(x, ptr, err) \ ({ \ - __get_user_err((x), (ptr), (err)); \ + __get_user_check((x), (ptr), (err)); \ (void)0; \ }) -#define __get_user_unaligned __get_user - -#define get_user(x, ptr) \ +#define __get_user(x, ptr) \ ({ \ - __typeof__(*(ptr)) __user *__p = (ptr); \ - might_fault(); \ - access_ok(VERIFY_READ, __p, sizeof(*__p)) ? \ - __p = uaccess_mask_ptr(__p), __get_user((x), __p) : \ - ((x) = 0, -EFAULT); \ + int __gu_err = 0; \ + __get_user_check((x), (ptr), __gu_err); \ + __gu_err; \ }) +#define __get_user_unaligned __get_user + +#define get_user __get_user + #define __put_user_asm(instr, alt_instr, reg, x, addr, err, feature) \ asm volatile( \ "1:"ALTERNATIVE(instr " " reg "1, [%2]\n", \ @@ -277,30 +282,35 @@ do { \ CONFIG_ARM64_PAN)); \ } while (0) -#define __put_user(x, ptr) \ +#define __put_user_check(x, ptr, err) \ ({ \ - int __pu_err = 0; \ - __put_user_err((x), (ptr), __pu_err); \ - __pu_err; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_WRITE, __p, sizeof(*__p))) { \ + __p = uaccess_mask_ptr(__p); \ + __put_user_err((x), __p, (err)); \ + } else { \ + (err) = -EFAULT; \ + } \ }) #define __put_user_error(x, ptr, err) \ ({ \ - __put_user_err((x), (ptr), (err)); \ + __put_user_check((x), (ptr), (err)); \ (void)0; \ }) -#define __put_user_unaligned __put_user - -#define put_user(x, ptr) \ +#define __put_user(x, ptr) \ ({ \ - __typeof__(*(ptr)) __user *__p = (ptr); \ - might_fault(); \ - access_ok(VERIFY_WRITE, __p, sizeof(*__p)) ? \ - __p = uaccess_mask_ptr(__p), __put_user((x), __p) : \ - -EFAULT; \ + int __pu_err = 0; \ + __put_user_check((x), (ptr), __pu_err); \ + __pu_err; \ }) +#define __put_user_unaligned __put_user + +#define put_user __put_user + extern unsigned long __must_check __arch_copy_from_user(void *to, const void __user *from, unsigned long n); extern unsigned long __must_check __arch_copy_to_user(void __user *to, const void *from, unsigned long n); extern unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n);